“The same way humans think” is the only kind of “think” that matters, for all intents and purposes. If we cannot define what it specifically is—because it loops us immediately back to the definition of consciousness et al.—the most precise definition of it will have to be along the lines of “the sort of thing that goes on in human minds”.

Would you invest so much in a bunch of if-then-else lists?

but what about compared to an average, or below average human would achieve?

>examples where quality far exceeds our existing best in class work

I'm no expert, but what percentage of the economy needs/is best in class work?

I agree running up against the limits of these models after reading the marketing material can give the impression they're "a kludge for the mediocre" and they certainly are in part that, but they're also so much more.[0]

[0] https://crawshaw.io/blog/programming-with-llms

but i agree with the others - AI currently shines not at being better than best-in-class humans, it's - sometimes - useful at doing intellectual chores.

You would be seeing far more than hype if we were at this point. This level of AI would be world changing.

I feel your expectations are too high. Realise what LLM’s are, what they can do and to what standard, and leverage them accordingly. It takes time to determine the scopes in which they increase your productivity. Trying it a few times and then disregarding it seems extremely foolish when so many developers are telling you it has enhanced their productivity to some degree.

A simple, naive solution from these LLMs runs 1000-2000x slower than the winning solution depending on the algorithm the LLM uses.

Asking the LLM to "make it faster" can knock the runtime by 10x. Close but no cigar.

Further prodding for even faster solutions sometimes shows glimmers of bridging the mental gap - but inevitably, the LLM doesn't know what to do with the newfound knowledge and wallows in its local minimum. Typically ending with "this is the fastest code to solve the problem".

Oh, grasshopper. You fail.

It wouldn't be so bad if this problem solving session was consistently reproducible. But every time I create a new session and give the problem, often times the naive solution appears. Sometimes the LLM can't even produce a working naive solution - even though the comments in the proposed source code shows the LLM knows what the correct answer is - the actual source code can't produce that answer.

Once, I decided to give some hints to the LLM to see if I could get it to jump the chasm. The LLM decided to limit its search space to a certain range - which I knew was wrong, and resulted in incorrect results.

I prompted the LLM to try and expand the search range, which it cheerfully tried and deduced an expanded search range - which was still too limited. Further prodding resulted in the LLM reverting back to the original search range. No amount of prodding would get the LLM to expand its search range. In a fit of rage, I told the LLM to use an arbitrary large search range which would cover the necessary search space. The LLM refused!

Dang - I was chatting with HAL from 2001 - A Space Odyssey.

I have been unable to repro that particular chat situation.

I guess we should be happy that the LLM can even produce a working solution - probably 5-10 years ago, we'd be gobsmacked. Even call it magic... But it's not remotely consistent.

Like, there are actual companies who have stopped hiring junior developers or even laid off junior developers in favour of our robot overlords. This is, obviously, a terrible idea, and will hurt those companies, because the output of these things is pretty much crap, but meanwhile some people _have_ lost their jobs.