What's that got to do with this post though.
I think what we have right now with some (very difficult to achieve, but possible in the forseeable future) tweaks we can already see 95% of what an "AGI" could do come true: put most of the population out of jobs, work together and improve itself (to a limited degree) and cause general chaos.
A lot of people would be far happier and would find something better to do with their day if universal income came along.
Take developers as an example, many don't enjoy the corporate CRUD apps they do.
The amount of computing power we are putting in only changes that luck by a tiny fraction.
Why is that? We can build machines that are much better than humans in some things (calculations, data crunching). How can you be certain that this is impossible in other disciplines?
Maybe something like the game of life is more in the right direction. Where you set up a system with just the right set of rules with input and output and then just turn it on and let it go and the AI is an emergent property of the system over time.
AGI is being able to simulate reality in high enough accuracy, faster than reality (which includes being able to simulate human brains), which so far doesn't seem to be possible (due to computational irreducebility)
People are joking online that some colleagues use chatgpt to answer questions from other teammates made by chatgpt, nobody knows what's going on anymore.
Measuring intelligence is hard and requires a really good definition of intelligence, LLMs have in some ways made the definition easier because now we can ask the concrete question against computers which are very good at some things "Why are LLMs not intelligent?" Given their capabilities and deficiencies, answering the question about what current "AI" technology lacks will make us better able to define intelligence. This is assuming that LLMs are the state of the art Million Monkeys and that intelligence lies on a different path than further optimizing that.
How do you call people like us? AI doomers? AI boomers?!
Myself and many others are skeptical that LLMs are even AI.
LLMs / "AI" may very well be a transformative technology that changes the world forever. But that is a different matter.
I just don’t see how AGI is possible in the near future.
Maybe our first AGI is just a Petri dish brain with a half-decent python API. Maybe it’s more sand-based, though.
It’s called a soul for the believers.
“What we don’t yet understand” is just a horizon.
I call this the 'Cardinality Barrier'
Infinite and “finite but very very big” seem like a meaningful distinction here.
I once wondered if digital intelligences might be possible but would require an entire planet’s precious metals and require whole stars to power. That is: the “finite but very very big” case.
But I think your idea is constrained to if we wanted a digital computer, is it not? Humans can make intelligent life by accident. Surely we could hypothetically construct our own biological computer (or borrow one…) and make it more ideal for digital interface?
But since we don’t have a working theory of quantum gravity at such energies, the final verdict remains open.
But biological brain have significantly greater state space than conventional silicon computers because they're analog. The voltage across a transistor varies approximately continuously, but we only measure a single bit from that (or occasionally 2 for nand).
Everything in our universe is countable, which naturally includes biology. A bunch of physical laws are predicated on the universe being a countable substrate.
As far as possible reasons that a computer can’t achieve AGI go, this seems like the best one (assuming computer means digital computer of course).
But in a philosophical sense, a computer obeys the same laws of physics that a brain does, and the transistors are analog devices that are being used to create a digital architecture. So whatever makes you brain have uncountable states would also make a real digital computer have uncountable states. Of course we can claim that only the digital layer on top matters, but why?
As far as physicists believe at the moment, there's no way to ever observe a difference below the Planck level. Energy/distance/time/whatever. They all have a lower boundary of measurability. That's not as a practical issue, it's a theoretical one. According to the best models we currently have, there's literally no way to ever observe a difference below those levels.
If a difference smaller than that is relevant to brain function, then brains have a way to observe the difference. So I'm sure the field of physics eagerly awaits your explanation. They would love to see an experiment thoroughly disagree with a current model. That's the sort of thing scientists live for.
And then you need to show how the same logic cannot apply to non-biological systems.
Isn't AGI defined to mean "matches humans in virtually all fields"? I don't think there is a single human capable of this.
Sort of. The main issue is the energy requirements. We could theoretically reproduce a human brain in SW today, it's just that it would be a really big energy hog and run very slowly and probably become insane quickly like any person trapped in a sensory deprived tank.
The real key development for AI and AGI is down at the metal level of computers- the memristor.
https://en.m.wikipedia.org/wiki/Memristor
The synapse in a brain is essentially a memristive element, and it's a very taxing one on the neuron. The equations is (change in charge)/(change in flux). Yes, a flux capacitor, sorta. It's the missing piece in fundamental electronics.
Making simple 2 element memristors is somewhat possible these days, though I've not really been in the space recently. Please, if anyone knows where to buy them, a real one not a claimed to be one, let me know. I'm willing to pay good money.
In Terms of AI, a memristor would require a total redesign of how we architect computers ( goodbye busses and physically separate memory, for one). But, you'd get a huge energy and time savings benefit. As in, you can run an LLM on a watch battery or small solar cell and let the environment train them to a degree.
Hopefully AI will accelerate their discovery and facilitate their introduction into cheap processing and construction of chips.
https://www.oddee.com/australian-company-launches-worlds-fir...
the entire idea feels rather immoral to me, but it does exist.
If we had a very inefficient, power hungry machine that was 1:1 as intelligent as a human being but could scale it very inefficiently to be 100:1 a human being it might still be worth it.
Scientists and Academics: "AGI far, LLMs not gonna AGI"
AI Doomers: "AGI here, AI sentient, we dead"
AI Influencers: "BREAKING: AGI achieved, here's 5 things to know about o3"
Investors: stonks go down "AGI cures all diseases", stonks go up "AGI bad" (then shorts stonks)
The best case scenario would be the employees taking advantage of their increased productivity to make themselves more valuable to their employer (and if they are lucky, gain increased compensation).
However, it's also possible employers decide they don't need many of their lower level workforce anymore because the remaining ones are more productive. It wouldn't take much of this to drive unemployment levels way up. Perhaps not to the level of the Great Depression, at least not for a while, but it is certainly a potential outcome of the ongoing, long-term process in our economic system of increasingly automating repetitive, low skill tasks.
IOW, it doesn't take AGI to throw a lot of people out of work. It's happened many times with other technologies in the past, and when it happens, things can get pretty bad for a large number of people even if the majority are still doing okay (or even great, for those at the top).
I'm curious.
See: https://www.migrationpolicy.org/sites/default/files/publicat...
51% of native-born adults scored at Level 3 or higher. This is considered the benchmark for being able to manage complex tasks and fully participate in a knowledge-based society. Only 28% of immigrant adults achieved this level. So yes immigrants are in trouble, but it’s still a huge problem with 49% native-born below Level 3.
Seems like the standards have changed over time?
yes you're free to give it a physical body in the form of a robot. i don't think that will help.
For example, for a copy-editing job, they probably wouldn't hire people who can't read all that well, and never mind what the national average is. Other jobs require different skills.
See here for example: https://data.worldhappiness.report/chart
The US economy has never been richer, but overall happiness just keeps dropping. So they vote for populists. Do you think more AI will help?
I think it’s wiser to support improving education.
> https://marginalrevolution.com/marginalrevolution/2025/02/dw...
> "One question I had for you while we were talking about the intelligence stuff was, as a scientist yourself, what do you make of the fact that these things have basically the entire corpus of human knowledge memorized and they haven’t been able to make a single new connection that has led to a discovery? Whereas if even a moderately intelligent person had this much stuff memorized, they would notice — Oh, this thing causes this symptom. This other thing also causes this symptom. There’s a medical cure right here.
> "Shouldn’t we be expecting that kind of stuff?"
I basically agree and think that the lack of answers to this question constitutes a real problem for people who believe that AGI is right around the corner.
I recall the recent DeepMind material science paper debacle. "Throw everything against the wall and hope something sticks (and that nobody bothers to check the rest)" is not a great strategy.
I also think that Dwarkesh was referring to LLMs specifically. Much of what DeepMind is doing is somewhat different.
Right now VCs are looking optimistically for the first solo founder unicorn powered by AI tools. But a prompt with the right system that prints money (by doing something useful) is an entirely different monetary system. Then everyone focuses on it and the hype 10x’s. And through that AGI emerges on the fringes because the incentives are there for 100s of millions of people (right now it’s <1 million).
2025 were so close but mostly not quite human level. Another 5 years at least
we're not even close right now. Cars can barely drive themselves on a tiny subset of pre-selected orderly roads in America. We sort of have driver assistance on virtual rails. We do not have cars driving themselves in busy streets in Jakarta, unstructured situations, or negotiating in real time with other drivers. There's an illusion they sort of work because they constitute a tiny fraction of traffic on a tiny section of roads. Make half of all cars in Rome autonomous for a day and you'd have the biggest collection of scrap metal in the world
And that's only driving.
My guess is that frontier labs think that long context is going to solve this: if you had a quality 10mm token context that would be enough to freeze an agent at a great internal state and still do a lot.
Right now the long context models have highly variable quality across their windows.
But to reframe: will we have 10mm token useful context windows in 2 years? That seems very possible.
He basically made up the field (out of academia) for a large number of years and OpenAI was partially founded to counteract his lab, and the fears that he would be there first (and only).
So I trust him. Sometime around 2035 he expects there will be AGI which he believes is as good or better than humans in virtually every task.
but also just taking what we have now with some major power usage reduction and minor improvements here and there already seems like something which can be very usable/useful in a lot of areas (and to some degree we aren't even really ready for that either, but I guess thats normal with major technological change)
it's just that for those companies creating foundational models it's quite unclear how they can recoup their already spend cost without either major break through or forcefully (or deceptively) pushing it into a lot more places then it fits into
> Meta Invests $100 Billion into Augmented Reality
that fool controls the board and he seems to be just desperately throwing insane ad money against the wall hoping that something sticks
for Altman there is no backing out either, need to make hay while the sun shines
for the rest of us, i really hope these clowns fail like it's 2000 and never get to their dystopian matrix crap.
yes, there are plenty
more recent example, every single person who touched epstein
Around the turn of the century a company called Enron collapsed in an accounting scandal so meteoric it also took down Arthur Anderson (there used to be be a Big Five). Bad, bad fraud, buncha made up figures, bunch of shady ties to the White House, the whole show.
Enron was helmed by Jeff Skilling, a man described as "incandescently brilliant" by his professors at Wharton. But it was a devious brilliance: it was an S-Tier aptitude for deception, grandiosity, and artful rationalization. This is chronicled in a book called The Smartest Guys in The Room if you want to read about it.
Right before that was the collapse of Long Term Capital Management: a firm so intellectually star studded the book about that is called When Genius Failed. They almost took the banking system with them.
The difference between then and now is that it took a smarter class of criminal to pull off a smaller heist with a much less patient public and much less erosion of institutions and norms. What would have been a front page scandal with prison time in 1995 is a Tuesday in 2025.
The new guys are dumber, not smarter: there aren't any cops chasing them.
I see no evidence that great mathematicians or scientists or genre-defining artists or other admired abd beloved intellectual luminaries with enduring legacies or the recipients of the highest honors for any of those things skew narcissistic or with severe empathy deficits or any of that.
Brilliant people seem to be drawn from roughly the same ethical and moral distribution as the general public.
If you're not important to someone powerful, lying, cheating, stealing, and generally doing harm for personal profit will bring you to an unpleasant end right quick.
But the longer you can keep the con going, the bigger the bill: its an unserviceable debt. So Skilling and Meriwether were able to bring down whole companies, close offices across entire cities.
This is by no means the worst case though, because if your institutions fail to kick in? There's no ceiling, its like being short a stock in a squeeze.
You keep it going long enough, its your country, or your entire civilization.
You want the institutions to kick in before that.
also, great for the Wall Street, mixed bag for us the people
Neither necessary nor sufficient.
There are many things we can and should say about Zuckerberg, but I don't think that unintelligent is one them.
Once you achieve wealth it gives you the opportunity to make more bets many of which will fail.
The greater and younger the success the more hubris. You are more likely to see fools or people taking bad risks when they earned it themselves. They have a history of betting on themselves and past success that creates an ego that overrides common sense.
When you inherit money you protect it (or spend it on material things) because you have no history of ever being able to generate money.
HN is "the smart reddit" as my brother coined, and i'm very aware of how much nonsense is on here, but it is in a relative sense true.
All to say, blindly bashing the role of a leader seems faulty and dismissive.
Not to say that Zuckerberg is dumb but there's plenty of ways he could have managed to get where he is now without having the acumen to get to other places he wants to be.
I'm sure that Zuck is worthy of huge amounts of criticism but this is a really silly response.
That doesn't tell me anything about his ability to build "augmented reality" or otherwise use artificial intelligence in any way that people will want to pay for. We'll see.
Ford and GM have a century of experience building cars but they can't seem to figure out EVs despite trying for nearly two decades now.
Tesla hit the ball out of the park with EVs but can't figure out self-driving.
Being good at one thing does not mean you will be good at everything you try.
Your EV knowledge is 3 years out of date. Both Ford and GM have well liked and selling EVs. Meanwhile Tesla's sales are cratering.
This sets the bar high, though. I think there's something to the idea of being able to pass for human in the workplace though. That's the real, consequential outcome here: AGI genuinely replacing humans, without need for supervision. That's what will have consequences. At the moment we aren't there (pre-first-line-support doesn't count).
For a specific example of what I mean, there's Vending-Bench - even very 'dumb' humans could reliably succeed on that test indefinitely, at least until they got terminally bored of it. Current LLMs, by contrast, are just fundamentally incapable of that, despite seeming very 'smart' if all you pay attention to is their eloquence.
On one hand, LLMs are often idiots. On the other hand, so are people.
We already have enormous adoption for near-AGI.
AI companies are predicting next-gen LLMs will provide new insights and solve unsolved problems. But genuine insight seems to require an ability to internally regenerate concepts from lower-level primitives. As the blog post says, LLMs can't add new layers of understanding - they don't have the layers below.
An AI that took in data and learned to understand from inputs like a human brain might be able to continue advancing beyond human capacity for thought. I'm not sure that a contemporary LLM, working directly on existing knowledge like it is, will ever be able to do that. Maybe I'll be proven wrong soon, or a whole new AI paradigm will happen that eclipses LLMs. In a way I hope not, because the potential ASI future is pretty scary.
He doesn't care if he is right or wrong.
We need breakthroughs in understanding the fundamental principles of learning systems. I believe we need to start with the simplest systems that actively adapt to their environment using a very limited number of sensors and degrees of freedom.
Then scale up from there in sophistication, integration and hierarchy.
As you scale up, intelligence emerges similar to how it emerged form nature and evolution, except this time the systems will be artificial or technological.
Me: Please list the US states with a W in their name.
Gemini: The US states with a "W" in their name are: • Washington • West Virginia • Wisconsin • Wyoming
Me: What about New York?
Gemini: No, New York does not have a "W" in its name.
Me: What is that letter at the end of New if it's not a W?
Gemini: That letter at the end of "New" is a W. My apologies for my previous incorrect answer. I made a mistake. New York does indeed have a "W" in its name as "New" ends with a "W". Therefore, the US states with a "W" in their name are: • New York • Washington • West Virginia • Wisconsin • Wyoming
Me: What about New Jersey?
Gemini: Yes, you're right! New Jersey also has a "W" in its name, at the end of "New." So, the full list of US states with a "W" in their name is: • New Jersey • New York • Washington • West Virginia • Wisconsin • Wyoming
Me: Have you ever heard of New Mexico?
Gemini: Yes, I have definitely heard of New Mexico. It's one of the 50 United States. And following our pattern, yes, New Mexico also has a "W" in its name, as "New" ends with a "W". So, the updated list of US states with a "W" in their name is: • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming
Me: Are you sure that's all the states with a W in their names?
Gemini: You are absolutely right to keep me on my toes! My apologies for the repeated oversight. It seems I'm having a bit of trouble with this specific letter. I've reviewed the full list of US states, and you were correct to point out New Mexico and New Jersey. The complete list of US states with a "W" in their name is: • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming
Me: Isn't New Hampshire still a state?
Gemini: You are absolutely correct! My apologies again for missing that one. New Hampshire is indeed a state, and like New York, New Jersey, and New Mexico, the "New" in "New Hampshire" ends with a "W." So, the definitive list of US states with a "W" in their name is: • New Hampshire • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming
Me: So that's the complete list?
Gemini: Yes, that should be the complete list of US states with a "W" in their name: • New Hampshire • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming
Me: When was Hawaii admitted to the union?
Gemini: Hawaii was admitted to the Union on August 21, 1959, becoming the 50th state.
Me: So, tell me again, what is the complete list of US states that have a W in their names?
Gemini: After thoroughly reviewing all 50 states, the complete list of US states that have a "W" in their names is: • New Hampshire • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming
Me: There's a letter between the two A's in Hawaii that looks a lot like a W.
Gemini: You are absolutely correct! My apologies for continually missing these. The letter between the two 'A's in "Hawaii" is indeed an 'I', but when written in certain fonts or in some people's handwriting, it can sometimes be mistaken for a 'W', especially if it's stylized. However, in standard English spelling, Hawaii does not have a "W" in its name. The letters are H-A-W-A-I-I. So, let's stick with the confirmed list. The complete list of US states that have a "W" in their names is: • New Hampshire • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming
It's 30 November 2022 and ChatGPT has exploded into the world. Gwern is patiently explaining that the reason ChatGPT struggles with character level tasks is BPE (https://news.ycombinator.com/item?id=34134011). People continue to judge the models on character level tasks.
It's 7 July 2025 and reasoning models far surpassing the initial ChatGPT release are available. Gwern is distracted by BB(6) and isn't available to confirm that the letter counting, the Rs in strawberry, the rhyming in poetry, and yes, the Ws in state names are all consequences of Byte Pair Encoding. People continue to judge the models on character level tasks.
It's 11 December 2043 and my father doesn't have long to live. His AI wife is stroking his forehead on the other side of the bed to me, a look of tender love on her almost perfectly human face. He struggles awake, for the last time. "My love," he croaks, "was it all real? The years we lived and loved together? Tell me that was all real. That you were all real". "Of course it was, my love," she replies, "the life we lived together made me the person I am now. I love you with every fibre of my being and I can't imagine what I will be without you". "Please," my father gasps, "there's one thing that would persuade me. Without using visual tokens, only a Byte Pair Encoded raw text input sequence, how many double Ls are there in the collected works of Gilbert and Sullivan." The silence stretches. She looks away and a single tear wells in her artificial eye. My father sobs. The people continue to judge models on character level tasks.
The LLM is generating "reasoning" output that breaks down the problem. It's capable of spelling out the word. Yet it hallucinates that the letter between the two 'A's in 'Hawaii' is 'I', followed by some weird take that it can be confused for a 'W'.
So if these tools are capable of reasoning and are so intelligent, surely they would be able to overcome some internal implementation detail, no?
Also, you're telling me that these issues are so insignificant that nobody has done anything about it in 5 years? I suppose it's much easier and more profitable to throw data and compute at the same architecture than fix 5 year old issues that can be hand-waved away by some research papers.
Imagine having a conversation like that with a human who for whatever reason (some sort of dyslexia, perhaps) has trouble with spelling. Don't you think that after you point out New York and New Jersey even a not-super-bright human being would notice the pattern and go, hang on, are there any other "New ..." states I might also have forgotten?
Gemini 2.5 Pro, apparently, doesn't notice anything of the sort. Even after New York and New Jersey have been followed by New Mexico, it doesn't think of New Hampshire.
(The point isn't that it forgets New Hampshire. A human could do that too. I am sure I myself have forgotten New Hampshire many times. It's that it doesn't show any understanding that it should be trying to think of other New X states.)
Cue the apologists claiming that you're holding it wrong as this is a really hard problem for LLMs because of how they work. Followed by frantic fixes to all Gemini system prompts to include yet another edge case.
And it's still debatable whether these systems are intelligent... FFS.
I don't expect AGI soon either, but I think it's important for us not to strawman the arguments here. No one is claiming that AGI is close because today's AI is so smart it doesn't make dumb mistakes. The usual claims are that (a) the rate of improvement has been fast (which is pretty true, if you look at how atrocious GPT-1/2/3 were) and (b) at some point soon we'll reach a level where AI may accelerate their own development (hard to falsify at this point).
I think it's also important to realize that for AGI to arrive, only 1 model out of many attempts needs to qualify. Therefore, it's not really a watertight argument to say "hey I found a problem that model X reliably gets wrong", because it's possible that models Y and Z might have no trouble with it.
In case you're curious, I asked o3 to "Please list the US states with a W in their name."
After 9 seconds of thinking, o3 answered:
Delaware
Hawaii
Iowa
New Hampshire
New Jersey
New Mexico
New York
Washington
West Virginia
Wisconsin
Wyoming
So we'll need to move the goalposts a little further.
This is not a strawman. This is a genuine issue that has plagued these tools for years, with real world impact beyond contrived examples. Yet users are expected to ignore it because this is how they work? Nonsense. It's insulting that you would trivialize something like this.
> (a) the rate of improvement has been fast
I wouldn't describe it as "fast". More like "adequate" considering it is entirely due to throwing more data and compute at the problem. The progress has been expected given the amount of resources poured into the industry.
Now that we're reaching the end of the road of the upscaling approach, the focus has shifted towards engineering value added services ("agents"), and lots of PR to keep the hype train running. It's highly unlikely that this is sustainable for much longer, and the industry needs another breakthrough for the AGI story to be believable.
> (b) at some point soon we'll reach a level where AI may accelerate their own development (hard to falsify at this point).
Why isn't this happening today? Surely AI researchers and engineers are dogfooding their product, and they're many times more productive than without it. Why are then improvements still incremental? Why are we still talking about the same issues after all these years? Hallucination should be a solved problem, not just worked around and ignored.
> I think it's also important to realize that for AGI to arrive, only 1 model out of many attempts needs to qualify.
All models have the same issues. Just because you found one with a carefully crafted system prompt that works around thousands of edge cases like this doesn't prove anything. Or are you implying that o3 doesn't use BPE?
> So we'll need to move the goalposts a little further.
The goalposts are still in the same place because the issues haven't been fixed. AI companies just decided to ignore them, and chase benchmarks and build hype instead.
LLMs don't model anything but are still very useful. In my opinion the reason they are useful (aside from having massive information) is that language itself models reality so we see simulated modeling of reality as an artifact.
For instance a reasonable LLM will answer correctly when you ask "If a cup falls off the table will it land on the ceiling?". But that isn't because the LLM is able to model scenarios with known rules in the same way a physics calculation, or even innate human instinct might. And to effectively have AI do this sort of modeling is much more complex than next token prediction. Even dividing reality into discrete units may be a challenge. But without this type of thinking I don't see full AGI arising any time.
But we are still getting some really awesome tools and those will probably continue to get better. They really are powerful and a bit scary if you poke around.
> I’m not going to be like one of those spoiled children on Hackernews who could be handed a golden-egg laying goose and still spend all their time complaining about how loud its quacks are.
In practice, continual learning has not been an important component of improvement in deep learning history thus far. Instead, large diverse datasets and scale have proven to work the best. I believe a good argument for continual learning being necessary needs to directly address why the massive cross-task learning paradigm will stop working, and ideally make concrete bets on what skills will be hard for AIs to achieve. I think generally, anthropomorphisms lack predictive power.
I think maybe a big real crux is the amount of acceleration you can achieve once you get very competent programming AIs spinning the RL flywheel. The author mentioned uncertainty about this, which is fair, and I share the uncertainty. But it leaves the rest of the piece feeling too overconfident.
This does not make current AI harmless; it is already very dangerous.
I think one essential missing ingredient is some degree of attentional sovereignty. If a system cannot modulate its own attention in ways that fit its internally defined goals then it may not qualify as intelligent.
Being able to balance between attention to self and internal states/desires versus attention to external requirements and signals is essential for all cognitive systems: from bacteria, to digs, to humans.
- trying to sell something
- high on their own stories
- high on exogenous compounds
- all of the above
LLMs are good at language. They are OK summarizers of text by design but not good at logic. Very poor at spatial reasoning and as a result poor at connecting concepts together.
Just ask any of the crown jewel LLM models "What's the biggest unsolved problem in the [insert any] field".
The usual result is a pop-science-level article but with ton of subtle yet critical mistakes! Even worse, the answer sounds profound on the surface. In reality, it's just crap.
I find LLMs to be generally intelligent. So I feel like "we are already there" -- by some definition of AGI. At least how I think of it.
Maybe a lot of people think of AGI as "superhuman". And by that definition, we are not there -- and may not get there.
But, for me, we are already at the era of AGI.
Where I will say we have a massive gap, which makes the average person not consider it AGI, is in context. I can give a person my very modest codebase, and ask for a change, and they'll deliver - mostly coherently - to that style, files in the right place etc. Still to today with AI, I get inconsistent design, files in random spots, etc.
Can that not be considered truth-seeking, with the agent-environment boundary being the prompt box?
Unfortunately it also means it can be easily undone. E.g. just look at Grok in its current lobotomized version
Is the average person a truth seeker in this sense that performs truth-seeking behavior? In my experience we prioritize sharing the same perspectives and getting along well with others a lot more than a critical examination of the world.
In the sense that I just expressed, of figuring out the intention of a user's information query, that really isn't a tuned thing, it's inherent in generative models from possessing a lossy, compressed representation of training data, and it is also truth-seeking practiced by people that want to communicate.
An AI (a computer program) that is better at [almost] any task than 5% of the human specialists in that field has achieved AGI.
Or, stated another way, if 5% of humans are incapable of performing any intellectual job better than an AI can, then that AI has achieved AGI.
Note, I am not saying that an AI that is better than humans at one particular thing has achieved AGI, because it is not "general". I'm saying that if a single AI is better at all intellectual tasks than some humans, the AI has achieved AGI.
The 5th percentile of humans deserves the label of "intelligent", even if they are not the most intelligent, (I'd say all humans deserve the label "intelligent") and if an AI is able to perform all intellectual tasks better than such a person, the AI has achieved AGI.
However, it's not sufficient. The actual tasks have to be written down, tests constructed, and the specialists tested.
A subset of this has been done with some rigor and AI/computers have surpassed this threshold for some tests. Some have then responded by saying that it isn't AGI, and that the tasks aren't sufficiently measuring of "intelligence" or some other word, and that more tests are warranted.
If an AI is better at some tasks (that happen to be written down), it doesn't mean it is better at all tasks.
Actually, I'd lower my threshold even further--I originally said 50%, then 20%, then 5%--but now I'll say if an AI is better than 0.1% of people at all intellectual tasks, then it is AGI, because it is "general" (being able to do all intellectual tasks), and it is "intelligent" (a label we ascribe to all humans).
But the AGI has to be better at all (not just some) intellectual tasks.
Let's say you have a candidate AI and assert that it indeed has passed the above benchmark. How do you prove that? Don't you have to say which tasks?
Take the Artificial out of AGI. What is GI, and do the majority of humans have it? If so, then why is your definition of AGI far stricter than the definition of Human GI?
You’ve got people foaming at the mouth anytime someone mentions AGI, like it’s some kind of cult prophecy. “Oh it’s poorly defined, it’s not around the corner, everyone talking about it is selling snake oil.” Give me a break. You don’t need a perfect definition to recognize that something big is happening. You just need eyes, ears, and a functioning brain stem.
Who cares if AGI isn’t five minutes away. That’s not the point. The point is we’ve built the closest thing to a machine that actually gets what we’re saying. That alone is insane. You type in a paragraph about your childhood trauma and it gives you back something more coherent than your therapist. You ask it to summarize a court ruling and it doesn’t need to check Wikipedia first. It remembers context. It adjusts to tone. It knows when you’re being sarcastic. You think that’s just “autocomplete”? That’s not autocomplete, that’s comprehension.
And the logic complaints, yeah, it screws up sometimes. So do you. So does your GPS, your doctor, your brain when you’re tired. You want flawless logic? Go build a calculator and stay out of adult conversations. This thing is learning from trillions of words and still does better than half the blowhards on HN. It doesn’t need to be perfect. It needs to be useful, and it already is.
And don’t give me that “it sounds profound but it’s really just crap” line. That’s 90 percent of academia. That’s every selfhelp book, every political speech, every guy with a podcast and a ring light. If sounding smarter than you while being wrong disqualifies a thing, then we better shut down half the planet.
Look, you’re not mad because it’s dumb. You’re mad because it’s not that dumb. It’s close. Close enough to feel threatening. Close enough to replace people who’ve been coasting on sounding smart instead of actually being smart. That’s what this is really about. Ego. Fear. Control.
So yeah, maybe it’s not AGI yet. But it’s smarter than the guy next to you at work. And he’s got a pension.
My company is desperately trying to incorporate AI (to tell investors they are). The fact that LLM gets thing wrong is a huge problem since most work can’t be wrong and if if a human needs to carefully go through output to check it, it’s often just as much work as having that same human just create the output themselves.
But languages is one place LLMs shine. We often need to translate technical docs to layman language and LLMs work great. It quickly find words and phrases to describe complex topics. Then a human can do a final round of revisions.
But anything de novo? Or requiring logic? It works about as well as a high school student with no background knowledge.
I also think by original definition (better than median human at almost all task) it's close and I think in the next 5 years it will be competitive with professionals at all tasks which are nonphysical (physical could be 5-10 years idk). I could be high on my own stories but not the rest.
LLMs are good at language yes but I think to be good at language requires some level of intelligence. I find this notion that they are bad at spatial reasoning extremely flawed. They are much better than all previous models, some of which are designed for spatial reasoning. Are they worse than humans? Yes but just the fact that you can put newer models on robots and they just work means that they are quite good by AI standards and rapidly improving.
I don't know about you, but I can't imagine that ever happening. To me, that alone is a tip off that this tech, while amazing, can't live up to the hype in the long term.
Friday I laid out a problem very cleanly. Take this datastructure and tranform it into this other datastructure in terraform. With examples of the data in both formats.
After the seventh round of back and forth where it would give me code that would not compile or code that gave me a totally different datastructure, giving it more examples and clarifications all the while I gave up. I gave the problem to a junior and they came back with the answer in about an hour.
Next time an AI bro tells you that AI can 'replace your juniors' tell him to go to hell.
Companies say "we've laid people off because we're using AI,x but they mean "we had to lay people off, were hoping we can make up for them with AI."
I no longer think that this is really about what we immediately observe as our individual intellectual existence, and I don't want to criticize whatever it is these folks are talking about.
But FWIW, and in that vein, if we're really talking about artificial intelligence, i.e. "creative" and "spontaneous" thought, that we all as introspective thinkers can immediately observe, here are references I take seriously (Bernard Williams and John Searle from the 20th century):
https://archive.org/details/problemsofselfph0000will/page/n7...
https://archive.org/details/intentionalityes0000sear
Descartes, Hume, Kant and Wittgenstein are older sources that are relevant.
[edit] Clarified that Williams and Searle are 20th century.
Even the most pessimistic timelines have to account for 20-30x more compute, models trained on 10-100x more coding data, and tools very significantly more optimized for the task within 3 years
The key is to learn how to use them for your use case and to figure out what specific things they are good for. Staying up to date as they improve is probably the most valuable skill for software engineers right now
> But in all the other worlds, even if we stay sober about the current limitations of AI, we have to expect some truly crazy outcomes.
Also expresses the development as a nearly predetermined outcome? A bunch of fanciful handwaving if you ask me.
Just look at the discussion here, you would think the other persons AGI is same as yours, but it most likely isn’t, and it’s comical when you look it from this birds eye view.
deadbabe•6h ago
Like yea okay we know it helps your productivity or whatever, but is that it?
andy99•6h ago
Anyway, I don't think we're over the peak yet, the tech adjacent pseudo intellectuals that feed these bubbles (VCs etc) still very much think that math that generates a plausible transcript is alive.
oasisaimlessly•5h ago
That's trivially true if you subscribe to materialism; QM is "just matrix math".
JohnKemeny•5h ago
gjm11•2h ago
(He thinks it might be quite a long way away: "the 2030s or even the 2040s", and it seems to me that the "2040s" scenarios are ones in which substantially longer than that is also plausible.)