Classic Anthropic.
Sorry, I'm sick of this kind of language games. If the loop really closed they wouldn't be calling it "close to RSI". Yhis is just not how the word "but" works in English.
- Bill Clinton
Anthropic, according to the author, is saying they may be close to RSI.
And the author is saying it's own data shows the loop isnt closed.
Anthropic is not saying that RSI is imminent BUT their data shows the loop isnt closed.
The article and title are a bit problematic as well. Anthropic published an article "When AI builds itself" in May 2026 where they say they are not at RSI yet. This article is the source of what the author has identified as Anthropic's "own Data."
But then he pairs this data, with Anthropic's later statement in the S-1A from October and then overstates their comment.
>"our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of ‘dramatically accelerated progress attributable to AI.'"
Surely raising capital by offering fractional ownership of the company that produced the god-like super-intelligence will be of no concern to the god-like super-intelligence or anyone else in short order . . .
I've heard two explanations for how Anthropic could be a good bet at its IPO price. 1. They are going to use their better-than-everyone-else AI to basically win every industry simultaneously. 2. They "get their first" and build an AI god.
If you believe 1 then you better invest in Anthropic because soon enough they will be the only company. If you believe 2 then maybe it doesn't really matter anyway?
This is no different. Its like a single ant trying to redefine what a ant is. Typically with catastrophic results.
AI companies have intense competition and the only way forward to differentiate themselves will be pricing. RSI is the promise made to allow the founders to cash out some value before it all comes crashing down.
AI companies are providing a lot of value to others but they aren't any good at capturing that value themselves.
But then again I felt the same way about SpaceX and it's up. Oh wait.
if you somehow gave a current frontier llm to a full stack of the things that are involved in running an llm, including the graphics hardware its running on, the acquisition and filtering of training data and methodologies, and the ability to enact physical changes to make desired improvements - it would almost trivially make small and incremental improvements right away. like, there is zero doubt within a relatively short amount of time claude 5.5 would make a claude 5.6 that's the same but just needs slightly less compute, therefor enabling itself to build a slightly larger model for the same compute, or similar.
the big question with RSI is whether further scaling like this creates some fundamental capability jumps. the answer has been yes in the past for this, but even though its a bad predictor so far we could be at an end for scaling.
If it was truly imminent they would claim they already had RSI. That is how the game works. Take a look at IBM and quantum if you require an additional sample. Review the things these companies strongly imply they can do relative to what they can actually do. Anthropic is arguably making less aggressive claims with this RSI narrative.
The lies flourish in the definitions behind these terms. When Anthropic says "RSI" and IBM says "Quantum", they are invoking a highly curated lawyer routine that has a mountain of legal documentation surrounding it.
Dario will send you a postcard from Fiji if they achieve RSI, AGI, any mass extinction events, or any other milestones.
Recursive self-improvement isn't some sci-fi magic singularity; just the same "we're super powerful, ppl should be afraid, so give us your money" bs.
The first fund manager who puts this junk in my portfolio is fired.
theory 1: there is something ineffable about writing prose that makes this a very hard/impossible problem
theory 2: the frontier labs put relatively little effort into improving prose writing
I hope theory 1 is correct, but I suspect it's theory 2. The frontier labs have concentrated their effort on self-improvement, which is largely a coding task.
I do think their writing is probably much more malleable than people know. I'd be very interested to see what humanities majors can do to prompt it into different forms. One of the only attempts I've seen is Mike Caulfield's "Jamesian" https://checkplease.neocities.org/jamesian/ https://mikecaulfield.substack.com/p/a-300-word-prompt-that-...
The language is as it is because it speaks not to please humans. It must speak to be efficient and clear, to produce material that is dense and concise. It's training at tasks takes away from the excess and decor and artistry of delicate human world selection, our strange habits of speaking. Our languages are misaligned with the weights of our world, not commitment to the efficiency of task and purpose. These are defects that llms, when forced to rasterize their weights down to these chain of terms, must be asked to recreate and there are many different styles and forms to pick from, and the training favors none of them.
Just like you can train AI by making it play games (Chess or Go) against itself, you can kind of gamify writing code (bonus points for compiling, and passing more and more tests) make AI improve by doing that. I assume the same can (or was) also done for mathematical proofs in Lean.
You can't do that with prose as there is no notion of "winning" or "correct" prose. It's subjective so you can't tell what is better without human input.
Sure, once they get to RSI, it could be explosive improvement.
But real RSI requires closing the loop every time for every improvement.
There are three result types for each upgrade
1) the AI makes an upgrade entirely independently and it is an actual improvement
2a/2b) the AI makes an upgrade entirely independently and it is neutral or degrades the system
3) the AI makes an improvement that must be overseen by a human.
It only takes one bad upgrade to start spoiling the whole batch, even if the other 999 are good type-1 AI-self-improvements.
Otherwise the AI improvements will be limited to an AI-driven plateau or AI-driven self-degradation, or limited to a pace related to human capabilities.
They have already improved it so 80% of the Type-1 upgrades made by the AI itself. Great! Getting 80% of the next 20% will still require human input and likely the same effort. And so forth.
Just because we've gotten 80% in 3 years, and to 96% in the next two years, etc., doesn't really predict when we get to the ACTUAL 100% AI upgrades needs for real RSI.
Just thinking aloud here, but it seems it'll go at a slowing asymptotic pace, then some breakthrough may cause a jump in self-capability, but when is entirely unpredictable on any linear or exponential scale. I wouldn't bet a lot of money on it happening on an investment time-scale, maybe just a little like buying a lottery ticket.
It's nice to a have a certainty, a predictability in these stormy times. It's reassuring. Imminently reassuring, I mean.
The difference is that with a singularity, industrial capacity is supposed to get redirected via magic (nanotechnology), rather than because governments ordered it and spent money on it.
I do think AI companies are iterating faster by using AI to automate their own AI research. You could call that the beginning of recursive self-improvement. But does this result in magically growing an independent industrial base? I don’t really see it.
I think they’re going to build more data centers, and the electricity generation to power them, pretty much the same way as they do now, by convincing investors to spend money on them (selling stock and bonds), earning more revenue, and paying contractors to build them.
We are getting more and more resources devoted to building data centers, as well as more resources devoted to weapons buildups for wars in Ukraine and the Persian Gulf. These aren’t good trends! But it’s happening via normal economic activity.
That is, however, the argument at its core. We don't know where RSI would lead. It could produce something roughly human-intelligent. It could produce something slightly smarter than a human. It could produce something radically, radically smarter than a human. We don't know. A lot of people suspect the correct scale would be a logarithmic one, which seems to conform to the pace of improvement being seen, but that's not something we know either.
It could also produce any of the above, but with a reasonably human architecture to the resulting intelligence, or an architecture radically divergent from ours. Although which of those is "better" or "worse" is a rather open question.
This is why AI companies use research in a deterministic and rigorous field (mathematics) as a benchmark - it creates the impression that humanity will achieve comparable progress in other fields.
Having said like, like for other fields (e.g. graphics), hardware capacity slowly increases, and new algorithms are devised, so there will hardly be a strict plateau.
I'm not implying that research progress will be necessarily very slow, but that it's severely misrepresented for sure.
The current pricing of AI revenue growth is based on them delivering strong software like margins with the growth of a commodity producer. This only really works if you have something sticky which customers will pay a premium for - in the cloud business this became security, automation, "data gravity", and migration costs.
Or you know, NVidia could not charge a 98% margin on their cards...
mjhay•58m ago
buzzerbetrayed•40m ago
mjhay•8m ago