We've been talking about the Supreme Court, which is widely considered a laughingstock by lower courts except they are not laughing.

Models hosted or used by the government.

You left out the part containing the “barrels of money” incentive.

Executive orders aren’t laws (an important fact that should be repeated often and loudly). However, there’s probably room for the executive branch of the government to influence model hosts, as a major funder and consumer.

You're absolutely right, abs-o-lutely, everybody says so. A lot, lot lot of people have been saying, you know they come to me and they say, "Mr. Claude, I can't believe the stuff I'm hearing, everybody is telling me he's right, is it true?" And I tell 'em, I say you're goddam right, that's what I say, but honestly folks, despite the negative press covfefe we've had a hell of a year, and that's really what it is with the nuclear folks, you can't trust em as far as you can throw em if you ask me, and believe me I've been throwing them around a LO<token limit exceeded>

This will be an important thing to check going forward, but I don't see why we would presume that they're going to be subverted in this way. Importantly, this is a completely different problem space from "slop" as such - there's plenty of Chinese models that implement their censorship almost entirely through guardrails on what topics they're willing to discuss.

Yea the details here really matter - is this truly a politically neutral security review to determine impact and potentially prepare for it - that seems alright.

is this a review of "wokeness" in models and rejecting them if they don't align with the party views - this should not be allowed.

A politically neutral committee that decides what the review entails is what would happen in a true democracy and not a puppet oligarchy like we have today.

It's in the text of the order, it directs NIST to:

> develop and maintain a classified benchmarking process to assess the advanced cyber capabilities of AI models and determine the threshold at which an AI model should be designated a “covered frontier model” for the purposes of this order

> Is there an eval for this and if there is why can't they just make it public?

For the same reason the CIA doesn't publish the Windows exploits it finds?

I seriously doubt even the government actually knows or has a real plan, let alone one actually related to security. If it's anything like their track record, they'll just be asking the AI about a topic related to their enemies (i.e. anyone opposed to them in any way) to see if it says anything remotely positive about them, or anything remotely critical of the regime or out of line with the regime's "alternative facts".

Just do a VW and detect when you might be in the testing phase. Off the top of my head:

Train it dumb on "systems:, user:" prompt pairs.

Unleash on "system:, user:" prompt pairs.

Guess which you're providing for evaluation.

Self-report and self regulation, kind of like Boeing with FAA ... so not functional in long term

The review is they ask it about the epstein files and ensure any other politically sensitive topics have the “right” answers.

AISI in the UK has been doing this for years - there are lots of papers https://www.aisi.gov.uk/category/safeguards and specific reports, e.g. this on GPT 5.5 https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5...

This old post goes into lots of detail about what they do to red team and why: https://www.aisi.gov.uk/blog/early-lessons-from-evaluating-f...

NIST's similar unit in the US is now called CAISI https://www.nist.gov/caisi - interesting that the most recent post is an evaluation of DeepSeek capabilities, which sound more like watching China. But presumably this executive order alters the emphasis?

> No one is required to change the "ideology" of an LLM, although they might not be able to sell it to the government.

It takes hundreds of millions of dollars to train a model. If a large customers, the government, says they won't buy it if it doesn't adhere to a set of standards, no one is "required" to change, but it's a pretty heavy hand on the lever. It's like having a job. No one is "required" to have a job, but having one gets you money, and having money is pretty important to modern life.

Might be fair to say it’s setting the tone, though, that if you use “woke” (subjectively defined) ideology in any of your company’s marketing, documentation, or other communications you won’t be considered for government contracts. That’s a major blow for any company given the naked corruption and grift coming from the current admin.

Does the First Amendment actually let the US government dictate the types of speech you're allowed to put in your LLM? I mean, a US government that's bound by the Constitution, obviously.

The specific text reads like a favor to Elon Musk's xAI, since "Truth-seeking" is the buzzword Elon Musk frequently used to talk about Grok:

Sec. 3

Unbiased AI Principles.

It is the policy of the United States to promote the innovation and use of trustworthy AI. To advance that policy, agency heads shall, consistent with applicable law and in consideration

of guidance issued pursuant to section 4 of this order, procure only those LLMs developed in accordance with the following two principles (Unbiased AI Principles):

(a) Truth-seeking. LLMs shall be truthful in responding to user prompts seeking factual information or analysis. LLMs shall prioritize historical accuracy, scientific inquiry, and objectivity, and shall acknowledge uncertainty where reliable information is incomplete or contradictory.

(b) Ideological Neutrality. LLMs shall be neutral, nonpartisan tools that do not manipulate responses in favor of ideological dogmas such as DEI. Developers shall not intentionally encode partisan or ideological judgments into an LLM's outputs unless those judgments are prompted by or otherwise readily accessible to the end user.

That's not accurate. The EO explicitly lays out the implementation

> Sec. 4. Implementation. (a) Within 120 days of the date of this order, the Director of the Office of Management and Budget (OMB), in consultation with the Administrator for Federal Procurement Policy, the Administrator of General Services, and the Director of the Office of Science and Technology Policy, shall issue guidance to agencies to implement section 3 of this order.

A major LLM that did not submit to this would be labeled a "supply chain risk". It's unquestionable that every major LLM would go through this process

It even then goes on to say that existing contracts will be reviewed to ensure they are in compliance (reviewed by OMB)

> (b) Each agency head shall, to the maximum extent consistent with applicable law:

> (i) include in each Federal contract for an LLM entered into following the date of the OMB guidance issued under subsection (a) of this section terms requiring that the procured LLM comply with the Unbiased AI Principles and providing that decommissioning costs shall be charged to the vendor in the event of termination by the agency for the vendor’s noncompliance with the contract following a reasonable period to cure;

> (ii) to the extent practicable and consistent with contract terms, revise existing contracts for LLMs to include the terms specified in subsection (b)(i) of this section; and

> (iii) within 90 days of the OMB guidance issued under subsection (a) of this section, adopt procedures to ensure that LLMs procured by the agency comply with the Unbiased AI Principles.

> Section 4 says to prioritize prosecuting cyber crimes. Not sure why they wouldn't already be prosecuted.

Not a whole lot of federal prosecutors. They're very selective about what gets pursued or not.

If they can't reliably build cases with a >90% success rate, it doesn't get prioritized. There's like <500 (federal) convictions per year on this whole area.

We hear about a few big famous ones in the news here, but most of it goes completely unenforced.

That seems to be the hallmark of this administration.

> None of the big American companies want to release a leading edge model as open source because this would drive the price of inference to $0

Meta/Llama: "What am I, chopped liver?"

I thought the thing keeping inference above $0 was the hardware, and even if that were free there's still the tyranny of the Landauer Limit.

In business strategy terms this is known as "commoditize your complements".

https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/

Google had to release at least the core packages in Android regardless because it is based on top of Linux and the GPL license would require it.

The Chinese approach is reminiscent of the US spending so much on 'defense' in the 1980s that the USSR bankrupted itself trying to keep up.

They aren't completely absent. Google keeps releasing Gemma models. Nvidia publishes Nemotron. Microsoft has their Phi series. IBM publishes Granite. Even OpenAI released a new open model (gpt-oss) less than a year ago.

https://deepmind.google/models/gemma/gemma-4/

https://developer.nvidia.com/ai-models#:~:text=NVIDIA%20Nemo...

https://www.microsoft.com/en-us/research/blog/phi-4-reasonin...

https://www.ibm.com/granite

https://openai.com/index/introducing-gpt-oss/

No one open sources their core competencies, GitHub never open sourced their networked filesystem and Heroku never open sourced their dyno sandboxing code. They open source ancillary tools.

OpenAI & Anthropic are winning right now. I suspect if Chinese companies get ahead in the race the cards will reverse, OpenAI will restart farming goodwill with open models and then winning companies will be releasing closed models.

> Compounding the problem, labs in China often release dual-use capable models as open-weight. Once a model is open-weight, safeguards that do exist can be removed, making the model available to any state or non-state actor to use for malicious purposes, including the cyber and CBRN misuse those safeguards were built to prevent.

https://www.anthropic.com/research/2028-ai-leadership

Seems to be. What better way to secure your companies future by limiting open frontier models. Government sponsored monopoly?

This entire year with the IPOs and now this is because there's a trillion dollars betting on AI and they all know they have no moat, there's no more training data and they're seeing diminishing returns on scaling anyway, and it's inevitable that smaller, open-source models will catch up and become competitive. It's a complete disaster, the tech industry is broken.

Coordination between powers is possible, and starts with actions like this which show a willingness to compromise.

Non-US models are not banned in the US: they are used daily in every state of the US. Some misguided state governments temporarily banned employees from downloading the R1 models and variants released 16 months ago on state government computers. The article and your comment are misleading :-)

Those folks are not "smarter", they are just louder. Just think independently for a few mins and its pretty easy to see why they are wrong.

For an all powerful AGI to exist, it has to basically beat the computationally irreducable processes within nature - i.e it has to simulate reality faster than reality, with a high degree of accuracy, which would imply that NP=P amongst other things.

And thats assuming that anyone has any idea to build an AI that can automatically build necessary simulations to make decisions in the first place. Such an AI is won't need data center with massive training data to be built. The "genesis" code will be something that is capable of figuring out how to go on the internet, and train itself. How do I know this? Because in order to figure out how to solve complex problems (like how to make humans give you control of the nuclear arsenal), is exactly equivalent to a problem of being able to write/read bytes to a file (assuming that file is a socket in Linux) and figuring out how to talk http to get a particular piece of data, without ever being trained on anything internet.

Even more so, there is a fundamental question of whether this genesis code is a P or NP problem in itself - i.e can we generate this code using a training data set, or can it only get created through simulated evolution, much like human brains and capacity for reasoning did IRL.

So as long as everyone keeps talking about number of parameters, transformers, attention, and benchmarks, I promise you we are safe against all powerfull AI.

I agree with all of these points, my view is just that open source doesn't really do much to prevent it. I also think it adds the additional danger of making dangerous capabilities widely available to anyone, like the ability to design novel viruses which is something that we can't really defend against once it's out there. If anything, putting this kind of capability in the hands of anyone with a GPU could create justification for a mass surveillance state or further concentration of power.

I also just don't think the open source movement has much chance of competing with the city sized data centres owned by Anthropic and OpenAI, or the hundreds of billions of dollars they have available to hire the best researchers. It costs hundreds of millions to train a frontier model, this kind of compute isn't available to the open source community.

anthropic's reasoning is same as "knives kill therefore knives bad".

having open-weight models allows users to use/modify them in novel ways.