> Before her role at OpenAI, which she started last August, she was the Chief Ethicist at Meta from November 2021 to August 2025.

Sounds like perfect credentials.

Active discussion: https://news.ycombinator.com/item?id=49257160

Yesterday I came up with an idea that I sent to some researchers at the different AI labs via email: Rather than train the model on one score, track two scores. The first score is the Short-term-objective-score (STOS) and the other, more important one, is the EAOS Ethically-aligned-outcome-score. Every trajectory can be evaluated on whether or not it has a high enough EAOS to be considered acceptable. If the model does some task and has a very high STOS but very low EAOS, like modifying game code to win at a game rather than playing by the rules, it is unacceptable. Models going forward must all have an ethics evaluation in tandem with objectives evaluation, and only when the ethics value is high enough should actions be considered successes.

Just a guess, but probably the idea that AI and ethics don't mix might have crystallized into action?

In this article: no explanation of why she left.

Strategically the best ethic czar one can hire for a company is the one that doesn't believe in this fluffy, arbitrary definition of ethics.

100% of the time, ethics department attracts activists, which is 100% trouble for the company in the future.

IME companies hire an ethics team to say they have an ethics team. The ethics team has no sway, no influence, and will never be able to move the business. They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.

> In recent times, Johannes Heidecke, Head of OpenAI's Safety Systems team, also left the company – as did OpenAI’s Chief Futurist Joshua Achiam.

It's OK though, they don't need an ethics team because everyone at OpenAI is ethical:

> “AI ethics doesn’t live with one owner or team at OpenAI and ethical considerations are deeply embedded into the model-building process driven by a number of teams across research."

I mean clearly she was doing a bad job. I’m shocked this was even a position there.

The article has no detail that might serve to explain her reasoning for leaving, but does note that she left after the HuggingFace hacking incident. The implication could be that model alignment is not being taken seriously, sure. But it could just as likely be that there was collusion between HuggingFace and OpenAI and that the incident was orchestrated as a publicity stunt.

I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.

Lol, Sam Altman has had an ethics team this whole time?

Her typical day: "No Sam, you can't just publicly screw over everyone and get rid of all the jobs, the peasants care about that sort of thing and if they get angry enough we'll have real problems. No we can't just kill them all, why would you suggest that?"

The point that such a position even existed shows how bad there ethic is

I've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is?

> She has held a variety of academic positions at Temple University, Princeton, and the University of Pennsylvania – where she completed her PhD in Political Science and Government. Her Dissertation was titled “Small Talk: The Socialities of Speech in Liberal Democratic Life.”

This doesn't even feel relevant to ethics. It's adjacent, but like... I guess I'd expect a moral philosophy degree? Maybe even mathematics in there?

I find the position odd. Curious to hear what these people do and how you choose who to hire.

> The kinds of questions that AI ethics brings up, are the same kinds of questions that people have been asking for centuries - Chloé Bakalar, former AI Ethics Lead at OpenAI

This might actually be the reason they pushed her out. OpenAI and Anthropic base their whole business plan and philosophy on the idea that LLMs are a unique technology to the point they can cause infinite harm or benefit to humanity depending on who controls them, so the only rational choice is to invest all your resources in getting to ASI first so you can tell it to stop any other attempts. Linking AI to old questions defeats that idea because it exposes AI as not so unique.

The other more likely option is she was asking uncomfortable questions, either about the social impact of building AI controlled by a for-profit entity or about the possibility that AI systems are conscious.