I have to constantly make sure it’s building things in a way that will fit neatly together, in a way doesn’t over (or under) -generalize. In a way that doesn’t waste time and energy on idiotic tests that aren’t really testing anything at all.

LLMs are powerful autocomplete with vast and broad memory, they do not think. And thinking is just a part of the human condition.

Our challenge ahead is making a living at all. Who is going to be having hobbies if the economy is 100% automated? We're not going to sleepwalk our way into a post-scarcity automated economy where everyone's needs are provided for and they can just spend all their time on hobbies. If we believe AI can replace all cognitive labor, then it follows that it'll be able to replace all physical labor with robotics fairly quickly, and we're left with no labor left for humans. We need to plan for that future or else billions will suffer from the results.

By the way, there is another problem. For humans we have the saying "garbage in, garbage out." If you read a lot of trash, you will only come up with trash. If you give powerful tools to morons, they will not become geniuses. That's not just true for humans, it's also the case for AI. If you teach your AI trash, it's going to be some trashy AI. Since people do that all the time for various reasons, we're going to have a lot of artificial stupidity, too.

  "I hate this damn machine"
  "I wish they would just sell"
  "It never does what I want"
  "Only what I tell it."

I think we'll see a shift towards arts, creativity, research, etc. where true novel creation and an understanding of the human experience exists at a premium.

Except those don’t really fit, here, because the author is making a far more critical error in ascribing far too much to current human cognition. On the daily I see so-called “knowledge workers” failing to read basic signs, refusing to learn basic skills, and all too eager to cast off the slightest bit of critical thinking to a black box they can subscribe to - be it replacing curating a content library with streaming services and algorithms, or swapping a life of hard-won expertise for a swarm of chatbots and an API bucket. As someone with the uniquely privileged POV to be able to ingest vast sums of technical data but struggle with social data, I look at my fellow Americans and genuinely fret for what the next few decades hold for a cohort that seemingly cannot effectively communicate which direction they’re going but is all too content to hitch their accountability to an LLM they pay for.

In that sense, thinking has already become a hobby to the masses, in that it’s entirely optional to exist in current society at large. Autopay apps manage your bills, finance apps keep you on budget, fitness apps tell you to keep running, social media apps tell you who your friends (and enemies) are, video apps tell you what to watch, music apps tell you what to listen to, and cars increasingly tell you not just where to go but also to let them get there for you. I would argue that in that context, thinking is a critically endangered hobby we sorely need to sow new seeds for, lest we risk losing it entirely; AI does not seem keen to do that.

Now to be fair, so much of this is hyperspecialization knowledge overflowing into daily life and making existence a sort of cognitohazard in and of itself. We skip through hundreds of pages of legal boilerplate because we’re not lawyers, and then complain about all the rights we willingly signed away because of the volume of said agreements. Non-technical people engaging with modern computing are either addicts or miserable, unaware that this was both entirely optional and also inevitable in a populace that didn’t think for itself. Finance has become so hyperspecialized that it’s impossible for the layman to tell of something is good or bad, or even why the number went up or down on their portfolio in a given time period. LLM boosters say their tools will solve this complexity nightmare, yet the tools themselves grow increasingly complex just to eke out single-point gains on hyperspecific problems.

Which brings me to the OP again, their highly uncritical claims that AI will bring about a new revolution of thought itself, and that somehow this tool will both restore thought to an elevated pedestal while also making it nearly impossible to meter (at least on American currency; let’s not think too hard about how developing countries might be able to afford it). I don’t see any of that, but for the same reasons the OP feels it’s somehow inevitable. The only difference I can readily figure is our opinions of our fellow humans: I view us as in dire straits and able to do far better than we give ourselves credit for, while they view us as replaceable with servile digital labor for the better.

I felt stuck. I didn’t know what the AI had done so far or how to continue. There was nobody to message for a budget increase (it was night).

Luckily I managed to get more budget.

But that left me scared for the future. One day something is gonna fail, and nobody can do anything because the AI providers are also down. Or what if I can no longer afford paying for the frontier?

I have made some modifications to my vibe coding practices since then. I don’t want to live in this future that the author describes.

[1]: https://lecturia.org/en/short-stories/isaac-asimov-the-feeli...

We made a new tool. Using the tool for some things makes life better. Using the tool too much would make life worse. So let’s not do that!

Somewhere I encountered a quote like "the finest art is a life well-lived". This threw me, because, apparently, I had been thinking of art as something different in kind from life, and unrelated to the normal work or act of living. Seen in terms of this quote, art becomes once again of-a-kind with life, and may be seen as only a part of living well, instrumental to living well, or perhaps just as practice for living well.

Something similar seems to be happening with "thinking" and "intelligence". Somehow these things had become ends-in-themselves, separate from the regular act of living. We might wrap them in a similar quote to place them in relation to life, along the lines of "thinking well is for living well", or perhaps "the highest good of the intellect is to be well-fit to the problems of living well".

I am reminded also of a line which stuck with me from Augustine's Confessions, approximately: "The mind is not the supreme good"—as opposed to the body and perhaps the "spirit". Somehow the world I was born into had collectively come to think of the mind as the supreme good, and intelligence as the greatest virtue. But this is, in some way, in contradiction the basic fact of existence and mortality: no amount of cleverness can circumvent death, and all the glory attainable by feats of the intellect are only futile vanities compared to the privilege of being alive itself.

That kind of thing.

The example of AIs solving mathematics is all fun and games, but it assumes that mastery of mathematics implies that everything else will also be mastered eventually. But instead we're seeing that LLMs exhibit this thing called jagged intelligence, which means that they can be extremely good in one area, but completely fail in another. In theory the LLMs could be improved if data would be available for other areas and if there are good verifiability criteria. Fact is that many fields just don't have those and would not be willing to give up their data.

This could not be more decoupled from the truth.

Yes, industrialization automated a lot of physical activity but I think the author severely underestimates the value that blue collar workers produce to this day. At least this is how I perceive it.

I would not place a lot of emphasis on economic insights from software engineering academics tbh.

This is the we-don’t-need-BluRays-because-all-movies-are-on-Netflix phase.

Let’s wait until the market is settled

But a the conclusion is a nice one for people that will be able to pull it off.

The author is not talking about random thoughts more like "deep thoughts" or "Reflections.

Reflection must be more like a trained "habit" than a "hobby".