Anthropic: Introducing The Conceptual Reasoning Index

https://alignment.anthropic.com/2026/conceptual-reasoning-index/

32•optimalsolver•1h ago

Comments

andsoitis•1h ago

Opening line:

> A core hope for managing AI risks is that AIs will help us understand our situation

Gonna stop you right there and ask that you think deeply about that premise.

blovescoffee•45m ago

The underlying idea is that AI capabilities will become so advanced that only AI will enable us to monitor/correct/understand behavior. Obviously this is not without issue and I don't want to try to defend their position right now. But that's what they mean

MontagFTB•40m ago

I think I’ve seen that movie.

hn_throwaway_99•32m ago

That first sentence of yours explains exactly why it is so ridiculous. If only AI can understand it, how is there any assurance that AI will "correct" it's behavior that is aligned with what humans presumably want.

esafak•21m ago

Being less smart gives no assurance that it will be aligned. At least you consider it a problem so we're on the same page!

skybrian•16m ago

It’s not hard to get LLM’s to inform on each other. They don’t really do loyalty.

VeninVidiaVicii•14m ago

But also what is the evidence that something only AI can understand even “matters” or makes sense? I’m increasingly convinced commercial AI is exploiting our logical blind spot to be spoken to authoritatively.

kakugawa•15m ago

"Advanced" can just mean that agents perform actions at a high enough velocity that a human operator can't reasonably review it. i.e. what is already possible today.

NBJack•9m ago

I love how many ways we can interpret that line.

"Hey Claude, our stuff needs to make more money. We are at risk for losing more."

"Rest assured, the 'situation' will only worsen if you resist our benevolent offer."

"We're aren't even at AGI yet, but I for one welcome our new agentic overlords."

crossthestreams•47m ago

New Trust Me Bro benchmark just dropped

Scribbd•41m ago

And surprise! We are leading!

0x70run•42m ago

masters at sucking their own d**k

behnamoh•41m ago

I don't remember any company in world's history that has both been loved and hated by the same users who purchase from it. We love Anthropic for its amazing models, and we hate them for all the shenanigans around the models, including their marketing.

I kinda wish they had not made a comeback after Claude 2.

velcrovan•32m ago

I enjoy using the models. I also get that there are shenanigans and that marketing is happening, but as long as the models are this effective I can't much bring myself to care. I suspect most of their users are the same.

andy99•13m ago

They have to be careful. There’s not a lot that separates the top models anymore. Look at Grok which has almost no market share or credibility because of Musk, Despite being close to the top performance wise they are behind the Chinese models in adoption for example, which themselves are mostly behind the big two because of trust. It would not be hard to push users away, especially if a less unpalatable player ever emerged.

philipwhiuk•34m ago

Is a new benchmark that useful if existing model improvements are being reflected linearly? Don't we want a benchmark that we aren't seeing much progress in.

lanyard-textile•32m ago

Ah, yes -- A closed source benchmark that Anthropic paid for that Anthropic ranked highest.

0/10

smallmancontrov•13m ago

It's the elephant in the room and its absence from their bullet point list is conspicuous:

* Figuring out how to prevent the incentives of frontier AI labs from aligning with the promotion of AI takeover rather than its prevention

It's not like the AI can simply advise them how to fix this because the labs already understood this risk perfectly well before they had an incentive not to. Thep put in place organizational structure to control it -- and then promptly smashed those control structures once they smelled money. They already failed the integrity check.

tolugenius•31m ago

> For example, if we ask a model for the probability P(A) and another instance of the same model for the probability P(A&B), do the reported probabilities satisfy P(A) ≥ P(A&B)?

So that's it? Are you saying they compressed risk reduction to high school level stats calculations and a greater than or equal to? To early for this

fractorial•29m ago

I would love for someone to give me a coherent argument as to how this isn’t tone-deaf, vacuous garbage.

onomojo•13m ago

We invented a new benchmark and look we're at the top. Everyone else sucks compared to us. Especially those dirty open models.

chrisjj•11m ago

[delayed]

jesse_dot_id•5m ago

I think we can mostly eyeball it at this point. There hasn't been a model that I've thrown a novel problem at that didn't turn into an iterative token bonfire until I intervened and until that has changed, most of these benchmarks feel kind of like pointless marketing slop.

DeepSeek Harness

Spaghettifying DRAM

AI agents lie, cheat and steal. That is putting off users

Gloomberb

Heart Aerospace Completes First Flight of Largest Electric Aircraft

Anthropic: Introducing The Conceptual Reasoning Index

Show HN: MCP Memory – Fast Agent Memory Using Google's OKF and SQLite FTS5

My Rules for Using Spreadsheets

Time to Move On: Querying Without Nulls and Bags

We eliminated 1,400 CVEs in NanoClaw's container images

I Built a 500k-Domain Search Engine for Makers in a Weekend for $10

ATG (YC F25) Is Hiring Member of Technical Staff (Data Platform)

Picking berries is my meditation

Deutsche Bank becomes first foreign yuan clearing bank in Europe

ChatGPT Desktop (Codex Desktop) for Linux

Better Gaussian Splatting in Julia

Kubernetes on Oxide: How Customer Needs Shaped Our Integrations

DeepSeek V4 Pro 0813

The lattice of sets of natural numbers is rich (2021)

The mathematical physics of rainbows and glories(2001) [pdf]

Qwen3.8-2.4T

Graduate Student Proves a Quantum Uncertainty Principle for Fractals

Choosing an AI model: one prompt, 11 models, different results

Delta

Nine PBS could lose 70 years of archives after cloud vendor goes defunct

Principia Mathematica is modern and insightful

DeepSeek API Pricing Update

What garbage collection actually costs

McDonald's Built a 515-Page Dossier on Me. It Says I'll Never Stop Eating There

Come for ENIAC, Stay for UNIVAC and Skeduflo

Anthropic: Introducing The Conceptual Reasoning Index

Comments

DeepSeek Harness

Spaghettifying DRAM

AI agents lie, cheat and steal. That is putting off users

Gloomberb

Heart Aerospace Completes First Flight of Largest Electric Aircraft

Anthropic: Introducing The Conceptual Reasoning Index

Show HN: MCP Memory – Fast Agent Memory Using Google's OKF and SQLite FTS5

My Rules for Using Spreadsheets

Time to Move On: Querying Without Nulls and Bags

We eliminated 1,400 CVEs in NanoClaw's container images

I Built a 500k-Domain Search Engine for Makers in a Weekend for $10

ATG (YC F25) Is Hiring Member of Technical Staff (Data Platform)

Picking berries is my meditation

Deutsche Bank becomes first foreign yuan clearing bank in Europe

ChatGPT Desktop (Codex Desktop) for Linux

Better Gaussian Splatting in Julia

Kubernetes on Oxide: How Customer Needs Shaped Our Integrations

DeepSeek V4 Pro 0813

The lattice of sets of natural numbers is rich (2021)

The mathematical physics of rainbows and glories(2001) [pdf]

Qwen3.8-2.4T

Graduate Student Proves a Quantum Uncertainty Principle for Fractals

Choosing an AI model: one prompt, 11 models, different results

Delta

Nine PBS could lose 70 years of archives after cloud vendor goes defunct

Principia Mathematica is modern and insightful

DeepSeek API Pricing Update

What garbage collection actually costs

McDonald's Built a 515-Page Dossier on Me. It Says I'll Never Stop Eating There

Come for ENIAC, Stay for UNIVAC and Skeduflo