I had a conversation with Michael Church about this. Yeah, that MChurch, the guy Dan Gackle and his ilk nonconsensually deplatformed here in 2015. He had an interesting observation. He’s been studying whether AIs can write fiction (they can’t do it well) and he said that while they can mimic style, well enough to fool all but the most attentive readers, they have no intentionality. In other words, the reason we’ll never see AI write a great literary novel is not that it’s impossible, but that no AI ever would. Intentionality is something AIs can’t fake, and the amount of handholding it would take to make an LLM write something worth reading exceeds what it would take to just write it.

I think Gemini 2.5 pro exp, openai o3, and claude 4 opus are already sort of publicly there.

I think there are others who are privately there.

>Rather, I think it’s genuinely hard to get normal humanlike labor out of LLMs. And this has to do with some fundamental capabilities these models lack.

There are a number of 'labours' which depend on things like scent sense that llms cant deal with.

Even with the latest models with millions of bytes of context, it wont remember deep stuff from weeks ago, but then again i dont either.

They'll also likely be bad at emotional manipulation. LLMs are going to be quite autistic in this sense.

>How do you teach a kid to play a saxophone? You have her try to blow into one, listen to how it sounds, and adjust. Now imagine teaching saxophone this way instead:

But you absolutely could do this. It's literally how we have starcraft, go, and chess engines that are better than humans.

Eventually someone will do this training.