(-here) damn that one hurts

You're absolutely right that AGI and ASI are not well-defined, but we also need to recognize that they definitely are actual things even though we lack a definition.

For any AI concept, I think it might be instructive to consider the human equivalent.

For example, what is the definition of a genius? We don't have one, but genius certainly is a thing that exists. We might argue about who is/isn't a genius because of the missing definition, but we probably agree that exceptional people are a thing.

One of the few things I’m sure about the current AI scene is that the marketing strategies of OpenAI, Anthropic, etc will be studied in business schools.

> I've never heard a decent definition

I would say AGI can do every intellectual task a human can do. Maybe the raw cognitive power is already there but scale matters. If it can't work on a task for months on end, it's not AGI. Because humans can do it.

> AI already blows most people out of an IQ test at a fraction of the computational power of a brain

Does it? I thought the brain is much more energy efficient.

    reply = AI.ask(task)
    while true:
        reply= AI.ask(f"""
            Improve the reply below:
            Task: {task}
            Reply: {reply}
        """")
        print(reply)

The key word in AI is "artificial"

Part of the difficulty here is that new age AI / LLMs in discussion has a lot of similarities with crypto, in the sense that there is a lot of nonsense out there. Unlike crypto there is obviously some value, as opposed to none, so it's not all grift. But like you I find it hard to sort the difference between the two.

Fundamentally for me I can't get over the idea that extraordinary claims require extraordinary evidence, and so far I haven't really seen any evidence. Certainly not any that I would consider extraordinary.

It's like saying that if a magician worked _really really hard_ on improving, evolving and revolutionising their "cut the assistant in half and put them back together" trick, they'll eventually be able to actually cut them in half and put them back together.

I have not seen a convincing reason to think that the path that is being walked down ends up at actual intelligence, in the same way that there is no convincing reason to think the path magicians walk down ends up in actual magic.

> AI already blows most people out of an IQ test at a fraction of the computational power of a brain

AFAIK, IQ tests used in psychological evaluations do not contain any randomness so exact answers are almost always in distribution. I haven't seen someone compare AI to an IQ test that is not in distribution.

On ARC-AGI, which is mildly similar to a randomly generated IQ test, humans still are much better than LLMs. https://arcprize.org/ (scroll down for chart)

> Firstly, what is AGI?

AGI is the biggest succesful scam in human history Sam Altman came up with to get the insane investment and hype they are getting. They are intentionally not defined what it is and when will be achieved, making it a never-reachable goal to keep the flow of money going. "we will be there in a couple of years", "this feels like AGI" was told every fucking GPT release.

It's the best interest for every AI lab to keep this lie going. They are not stupid, they know it can't be reached with the current state-of-the-art techniques, transformers, and even with the recent groundbreaking techniques like reasoning, and I think we are not even close.

> To make AI more general, you just make the input tokenization more granular, change the training method, and perhaps add some kind of iterative framework on top to make it do more stuff.

It's probably a bigger step than that. For example humans learn from experience, current AIs are trained offline and then frozen and can only "learn" by stuffing their short term memory with notes, like someone with anterograde amnesia.

AI technology is changing each year, and AGI will probably be more different from today's transformer-based systems than they were from convolutional nets.

But it will probably incorporate ideas and components from today's LLMs, and just as importantly, it's development will probably be paid for from the pockets of investors who have been tantalized by today's AI and think AGI must soon follow.

I personally hope it's a long way off but it might only be a few key insights away at this point.

> Firstly, what is AGI? I've never heard a decent definition. Some say it's an AI that is as smart or as general as humans; some say it's an AI that's conscious.

Correct, but many have noticed this including Sam Altman in some interviews.

Everyone disagrees about all three initials of the initialism, plus if the whole even means something implied by those initials at all, plus treating each as a boolean rather than a number.

Also some people loudly reject this observation.

The definition I was using for "AGI" befor ChatGPT came out was met by ChatGPT-3.5: it's an AI that's general-purpose, it doesn't just e.g. play chess. But there's people who reject that it even counts as "AI" at all, despite, well, the Turing Test.

Anyway.

> I don't see how it could be a "step" by these definitions because nothing actually changes between a regular AI and an AGI. To make AI more general, you just make the input tokenization more granular, change the training method, and perhaps add some kind of iterative framework on top to make it do more stuff.

That's a lot of things.

> Also, AI is already more capable than humans in almost every way except scale.

No, not really. There's specific metrics where some AI can beat a lot of humans, like playing chess, or how many languages it speaks to the level of a competent adult learner, and they can pay attention to a lot more than we can, and they can process data faster than we can, but…

…but LLMs* are only giving the illusion of intelligence by using superhuman speed and superhuman attention to make up for having a mind with the complexity of a squirrel's that's had half a million years of experience reading everything it can lay its metaphorical hands on.

* not VLMs, they're not fast enough or smart-looking enough yet

> AI already blows most people out of an IQ test at a fraction of the computational power of a brain, but that's because IQ tests compare competence to get relative intelligence; they don't test computation.

IQ tests are indeed bad, can be learned. This is demonstrable because the graph you may have seen for AI IQ ratings has two variants from the same people, one with a public IQ test and one with a private IQ test, and most of the AI do much much worse with the private test: https://trackingai.org/home

The problem you're pointing at is also why the ARC-AGI test exists, and indeed this shows that current AI aren't anything close to human performance.

> With that assumption, if AI can't be computationally stronger than humans, it's safe to say we won't have conscious computers for a while, but instead computers that act consciously instead.

If you think "AGI" is poorly defined, you'll be horrified to learn that "consciousness" has something like 40 definitions. Nobody knows where it comes from in our brain structures or how much structure we need to have it. Is a mouse conscious? If so, there's enough compute going on for many AI models to be also. But the quantity of compute is likely a red herring, and the compute exists even if it's not running an AI, just as our brain chemistry still goes on while we're sleeping.

> Does that mean that AI from hear on out is a waste that does nothing but take control from people while benefiting us the same?

Yes, but such people will take any opportunity or tech to do so, we've seen that since "automation" meant "steam engine".

> What is ChatGPT going to look like in 5 years? Am I just going to type in "do my taxes," and it's just going to do whatever it wants on my pc until my taxes are done?

If you're lucky. I think probably not, but 5 years is too long to rule out an architectural breakthrough that makes them much less error-prone; the Transformer models in 2020 were unimpressive for anything except translation.

> Why would I ever want that over a system designed to do my taxes correctly EVERY TIME by accountants?

$$$; plus accountants aren't perfect, they're only human.

> One thing I know about AI is it is slow as a mf, AI is great but you really have to think, we really just built a giant dictionary guy who's going to have the same problems human employees have.

If we're lucky.

Most likely it will continue to have new and exciting problems.

> Also, AI is already more capable than humans in almost every way except scale.

Not at all, AI still ridiculously fails at tasks that are quite simple for humans/children.

I define AGI as the only real AI that can exist. Anything less is just mimicry. To give a stupid/simplified example: a truck driving specialized "AI" would not be able to decide when to stop or not when somebody steps in front of it, making it trivial to rob "AI" driven trucks. To decide it needs to understand the kinds of people that exist, their motivations, the laws, etc. So it needs to be an AGI. Otherwise it will make horrible mistakes we don't even think are possible, or are so uncommon that when they happen to a human they make the news.