But now you've got me thinking. Has anyone studied whether the programmers who are more enamored of AI are also into RPGs?

All ten outputs might be valid. All ten will almost certainly be different -- though even that is not guaranteed.

The OP referred to the notion of there being no manual; we have to figure out how to use the tool ourselves.

A traditional programming tool manual would explain that you can provide input X and expect output Y. Do this, and that will happen. It is not so clear-cut with AI tools, because they are -- by default, in popular configurations -- nondeterministic.

To use an analogy, it would be like spending all your time before a battle making sure your knife is sharp when your opponent has a tank.

I also like to think that Einstein would be smart enough to explain things from a common point of understanding if you did drop him 2000 years in the past (assuming he also possesses the scientific knowledge humanity accrued in that 2000 year gap). So, your analogy doesn't really make a lot of sense here. I also doubt he'd be able to prove his theories with the technology of the past but that's a different matter.

If we did have AGI models, they would be able to solve our hardest problems (assuming a generous definition of AGI) even if we didn't immediately understand exactly how they got there. We already have a lot of complex systems that most people don't fully understand but can certainly verify the quality of. The whole "too smart for people to understand that they're too smart" is just a tired trope.

you are, for sure.

The mirage is alluring.

And since it's way, way less wrong than sonnet4, it might also improve my whole team velocity.

I won't lie, AI coding has been a net negative for the 'lazy devs' on my team who don't delves into their own generated code (by 'lazy devs' here I mean the subset of devs who do the work but often don't bother to truly understand the logic behind what they used/did. They are very good coworkers, add velue and are not really lazy, but I don't see another term for that).

Visual Studio Code is a different thing... and claims to be open source, but by intent and approach really is closer to source available.

Wild how much you can get for free now. Amazing free IDEs. Every LLM offers excellent free plans if you are on a zero budget.

$10/mo GitHub Copilot is an absurd deal that has to be a loss in terms of pure compute cost.

The framing allows the rest of us to get ourselves off the hook. "We didn't have a choice! It was INEVITABLE!"

And so, we have chosen.

It’s very easy to spend $100s per dev per day.

Lol sometimes I have to spend two turns convincing Claude to use its goddamn search and look up the damn doc instead of trying to shoot from the hip for the fifth time. ChatGPT at least has forced search mode.

Instead of stuffing the context with DDL I suggest:

1. Reorganize your data warehouse. It needs to be easy to find the correct data. Make sure you use ELT clear layers, meaningful schemas, and have per-model documentation. This is a ton of work, but if done right the payoff is massive.

2. I built a tool for myself to pull our warehouse into a graph for fuzzy search+dependency chain analysis. In the spring I made an MCP server for it and Claude uses that tool incredibly well for almost all queries. I haven't actually used the GUI or scripts since I built the MCP.

Claude and Devstral are the best models I've used for SQL. I cannot get Gemini to write decent modern sql -- even the Gemini data science/engineer agents in Google Cloud. I occasionally try the paid models through the API and still haven't been impressed.

[1] https://code.claude.com/docs/en/sandboxing#configure-sandbox...

[2] https://github.com/Piebald-AI/claude-code-system-prompts/blo...

"These things are more destructive than your average toddler, so you need to have a fence in place kind of like that one in Jurassic Park, except you need to make sure it absolutely positively cannot be shut off, but all this effort is worthwhile, because, kind of like civets, some of the artifacts they shit out while they are running amok appear to have some value."

For me, I'm planning to ride out this industry for another couple years building cash until I can't stand it, then pivot to driving a city bus.

I cannot count the times that I've had essentially this conversation:

"If x happens, then y, and z, it will crash here."

"What are the odds of that happening?"

"If you can even ask that question, the probability that it will occur at a customer site somewhere sometime approaches one."

It's completely crazy. I've had variants on the conversation from hardware designers, too. One time, I was asked to torture a UART, since we had shipped a broken one. (I normally build stuff, but I am your go-to whitebox tester, because I hone in on things that look suspicious rather than shying away from them.) When I was asked the inevitable "Could that really happen in a customer system?" after creating a synthetic scenario where the UART and DMA together failed, my response was:

"I don't know. You have two choices. Either fix it where the test passes, or prove that no customer could ever inadvertently recreate the test conditions."

He fixed it, but not without a lot of grumbling.

I think that for some people it is harder to reason about determinism because it is similar to correctness, and correctness can, in many scenarios be something you trade off - for example in relation to scaling and speed you will often trade off correctness.

If you do not think clearly about the difference with determinism and other similar properties like (real-time) correctness which you might be willing to trade off, you might think that trading off determinism is just more of the same.

Note: I'm against trading off determinism, but I am willing to think there might be a reason to trade it off, just I worry that people are not actually thinking through what it is they're trading when they do it.

It's wild that you think programmers is some kind of caste that makes any decisions.

The average programmer is already being pushed into doing a lot of things they're unhappy about in their day jobs.

Crappy designs, stupid products, tracking, privacy violation, security issues, slowness on customer machines, terrible tooling, crappy dependencies, horrible culture, pointless nitpicks in code reviews.

Half of HN is gonna defend one thing above or the other because $$$.

What's one more thing?

The ROI of doing this is weak because of how long it takes an expensive human. But if you could clean it up more cheaply, the ROI strengthens considerably- and there’s a lot of it.

I'd rather have LLMs build on top of proven, battle-tested production libraries than keep writing their own from scratch. You're going to fill up context with all of its re-invented wheels when it already knows how to use common options.

Not to mention that testing things like this is hard. And why waste time (and context and complexity) for humans and LLMs trying to do something hard like state syncing when you can focus on something else?

...is a loaded question, with a complex and nuanced answer. Especially when you continue:

> it's worth paying the React complexity/page-weight tax

All right; then why do we code in React when a smaller alternative, such as Preact, exists, which solves the same problem, but for a much lower page-weight tax?

Why do we code in React when a mechanism to synchronize data with tiny UI fragments through signals exists, as exemplified by Solid?

Why do people use React to code things where data doesn't even change, or changes so little that to sync it with the UI does not present any challenge whatsoever, such as blogs or landing pages?

I don't think the question 'why do we code with React?' has a simple and satisfactory answer anymore. I am sure marketing and educational practices play a large role in it.

I'm incentivised to use abstractions that are harder to learn, but execute faster or more safely once compiled. E.g. more Rust, Lean.

> If an LLM is typing that code - and it can maintain a test suite that shows everything works correctly - maybe we don't need that abstraction after all.

LLMs benefit from abstractions the same way as we do.

LLMs currently copy our approaches to solving problems and copy all the problems those approaches bring.

Letting LLMs skip all the abstractions is about as likely to succeed as genetic programming is efficient.

For example, writing more vanilla JS instead of React, you're just reinventing the necessary abstractions more verbosely and with a higher risk of duplicate code or mismatching abstractions.

In a recent interview with Bret Weinstein, a former professor of evolutionary biology, he proposed that one property of evolution that makes the story of one species evolving into another more likely is that it's not just random permutations of single genes; it's also permutations to counter variables encoded as telomeres and possibly microsatellites.

https://podcasts.happyscribe.com/the-joe-rogan-experience/24...

Bret compares this to flipping random bits in a program to make it work better vs. tweaking variables randomly in a high-level language. Mutating parameters at a high-level for something that already works is more likely to result in something else that works than mutating parameters at a low level.

So I believe LLMs benefit from high abstractions, like us.

We just need good ones; and good ones for us might not be the same as good ones for LLMs.

I've had plenty of junior devs justify massive code bases of random scripts and 100+ line functions with the same logic. There's a reason senior devs almost always push back on this when it's encountered.

Everything hinges on that "if". But you're baking a tautology into your reasoning: "if LLMs can do everything we need them to, we can use LLMs for everything we need".

The reason we stop junior devs from going down this path is because experience teaches us that things will break and when they do, it will incur a world of pain.

So "LLM as abstraction" might be a possible future, but it assumes LLMs are significantly more capable than a junior dev at managing a growing mess of complex code.

This is clearly not the case with simplistic LLM usage today. "Ah! But you need agents and memory and context management, etc!" But all of these are abstractions. This is what I believe the parent comment is really pointing out.

If AI could do what we originally hoped it could: follow simple instructions to solve complex tasks. We'd be great, and I would agree with your argument. But we are very clearly not in that world. Especially since Karpathy can't even keep up with the sophisticated machinery necessary to properly orchestrate these tools. All of the people decrying "you're not doing it right!" are emphatically proving that LLMs cannot perform these tasks at the level we need them to.

I'm instructing my agents to doing old school boring form POST, SSR templates, and vanilla JS / CSS.

I previously shifted away from this to abstractions because typing all the boilerplate was tedious.

But now that I'm not typing, the tedious but simple approach is great for the agent writing the code, and great for the the people doing code reviews.

The role of abstractions *IS* to prevent (eg "compress") the need for a test suite, because you have an easy model to understand and reason about

However, the quality of this code is fucking terrible, no one is reading what they push deeply, and these models don't have enough 'sense' to make really robust and effective test suites. Even if they did, a comprehensive test suite is not the solution to poorly designed code, it's a band aid -- and an expensive one at scale.

Most likely we will see some disasters happening in the next few years due to this mode of software development, and only then will people understand to use these agents as tools and not replacements.

...Or maybe we'll get AGI and it will fix/maintain the trash going out there today.

Are you able to efficiently verify that the test suite is testing what it should be testing? (I would not count "manually reviewing all the test code" as efficient if you have a similar amount of test code to actual code.)

Sometimes a change to the code under test means that a (perhaps unavoidably brittle) test needs to be changed. In this case, the LLM should change the test to match the behaviour of the code under test. Other times, a change to the code under test represents a bug that a failing test should catch -- in this case, the LLM should fix the code under test, and leave the test unchanged. How do you have confidence that the LLM chooses the right path in each case?

We seem to have created a large bureaucracy for software development, where telling a computer how to execute an app involves keeping a lot of cogs in a big complicated machine happy. But why use the automation to just roll the cogs? Why not just simplify/streamline? Does an LLM need to worry about using the latest and greatest abstractions? I have to assume this has been tried already...

If I just wanted the equivalent of Lodash's _.intersection() method, I get it. The requirements are pretty straightforward and I can verify the LLM code & tests myself. One less dependency is great. But with time, I know I don't know enough to verify the LLM's output.

Similar to encryption libraries, it's a common recommendation to leave time-based code to developers who live and breathe those black boxes. I trust the community verify the correctness of those concepts, something I can't do myself with LLM output.

Can you manually wade through thousands of functions and fix the issue?

for simple stuff, sure, React was ALWAYS inefficient. Even Javascript/client-side logic is still overkill a lot of the times except for that pesky "user expectations" thing.

for anything codebase that's long-lived and complex, combinatorics tells us how it'll near-impossible to have good+fast test coverage on all that.

part of the reason people don't roll their own is because being able to assume that the library won't have major bugs leads to an incredible reduction in necessary test service, and generally people have found it a safe-enough assumption.

throwing that out and trying to just cover the necessary stuff instead - because you're also throwing out your ability to quickly recognize risky changes since you aren't familiar with all the code - has a high chance of painting you into messy corners.

"just hire a thousand low-skilled people and force them to write tests" had more problems as a hiring plan then just "people are expensive."

  > I'm now incentivized to use less abstractions.

But this is a highly non-trivial problem. How do you even possibly manually verify that the test suite is complete and tests all possible corner cases (of which there are so many because synchronizing state is a hard problem)?

At least React solves this problem in a non-stochastic, deterministic manner. What can be a good reason to replace something like React that works determinstically with LLM-assisted code that is generated stochastically and there's no easy way to manually verify if the implementation or the test suite is correct and complete?

And if I have to closely supervise every single change, I don't believe my development process will be any better. If not worse.

Let alone new engineers who join the team and all of a sudden have to deal with a unique solution layer which doesn't exist anywhere else.

My code is constantly shrinking, becoming better quality, more performant, more best-practice on a daily basis. And I'm learning like crazy. I'm constantly looking up changes it recommends to see why and what the reasons are behind them.

It can be a big damned dummy too, though. Just today it was proposing a massive server-side script to workaround an issue with my app I was deploying, when the actual solution was to just make a simple one-line change to the app. ("You're absolutely right!")

I'm worried that there is a tendency in LLM-generated code to avoid even local abstractions, such as putting common code into separate (local functions), and even use records/structures. You end up with code that is best maintained with an LLM, which is good for the LLM provider and their future revenue. But we humans as reviewers and ultimate long-term maintainers benefit from those minor abstractions.

Jensen is someone I trust to understand the business side and some of those lower technical layers, so I'm not too concerned.

A couple weeks ago, under a freshly made account "llmslave", you said it's already replacing devs and the field is cooked, and anyone who doesn't see that lacks the skills to adopt AI [1]

I pointed out that given your name and low quality comments, you were likely an LLM run account. As SOON as I made that comment, you abandoned the account and have now made a duplicate llmslave2 account, with a different opinion

Are you doing an experiment or something?

[1] https://news.ycombinator.com/item?id=46291504#46292968

Cryptoboys did it first, please recognize their innovation ty

AI psychosis is getting lost in the sauce and becoming too intimate with your ChatGPT instance, or believing it's something it's not.

Skepticism, or a fear of being outside the core loop is the exact opposite, and that's what Karpathy is talking about here. If anything, this kind of post is an indicator that you're absolutely NOT in AI psychosis.

As a creator of an open-source platform myself, I find trusting a semi-random word generator in front of users unreliable.

Moreover, I believe it creates a bad habit. I've seen developers forget how to read documentation and instead trust AI, and of course, as a result AI makes mistakes that are hard to debug or provokes security issues that are easy to overlook.

I know this sounds like a luddite talking, but I'm still not convinced that AI in its current state can be reliable in any way. However, because of engineers like you, AI is learning to make better choices, and that might change in the future.

Answering your question, no matter how much I personally degrade or improve, I will not be able to produce anything even remotely comparable in terms of negative impact that AI brings to humanity these days.

You just spot checked it, so how can you be sure how accurate it is. Was it 80% accurate? 90%? 99%? And how does the domain influence the accuracy requirements?

If you truly believe AI is simply going to collapse and disappear, you are deep in some serious cope and are going to be unpleasantly surprised.

In terms of bubbles: Bubbles are economic concepts and they will burst but the underlying technology find its market. There are plenty of good open source models and open source projects like OpenCode/Toad that support them. We can use those without contributing (too much) to the bubble.

Unless you're in web dev because it seems like that's one of the few domains where AI actually works pretty well today.

???

And rest of my field. Automated tools do part of work. AI probably some, but not enough of actually verifying findings and then properly explaining the context and implications.

Earlier this year the ecosystem was still a mess I didn't have time to untangle. Now things are relatively streamlined and simple. Arguably stable, even.

I feel behind, sure, but I also don't think people on the bleeding edge are getting that much more utility that it's worth sinking dozens or hundreds of my very limited hours into understanding.

Besides, I'm a C programmer. I'll always be several decades behind the trend. I'm fine with that.

It's high dose copium. Please keep the good times rolling! Buy my books! Sub to my stack!

Meanwhile, with local models, local RAG, and shell scripts, I am wandering 3D immersive worlds via a GPU accelerated presentation layer I vibe coded with a single 24GB GPU. Natural language driven Unreal engines are viable outputs today given local only code gen.

Karpathy and the SV VC world thought this would be the next big thing to pump for a decade plus; like web pages and SaaS. But the world is smarter, more adept at catching up that it is just state management in a typical machine. The semantics are well known and do not need re-invention.

The hilarity at an entire industry unintentionally training their replacements.

What burden are you talking about? Using LLMs isn't that hard, we have done harder things before.

Sure, there will be people that refuses to "let go" and want to keep doing things the way the like them, but hey! I've been productive with vim (now neovim) for 25 years and I work with engineers that haven't mastered their IDEs at the same level. Not even close!

Sure, they have have never been "burdened" by knowing other editors before those IDEs existed, but claiming that I would have it harder to use any of those because I've mastered other tools before is ridiculous.

I'm very happy being decades behind the curve here. C's slowness is perfect for me.

https://github.com/left-pad

this is because SWEs love bloat and any good idea eventually needs to balloon into some ever-growing monstrosity :)

https://www.teslarati.com/tesla-ai-director-hiring-autopilot...

He gave a glowing recommendation for camera-only FSD in 2021:

https://thenextweb.com/news/tesla-ai-chief-explains-self-dri...

Then he left Tesla in 2022. So yes, you could argue that it was all Elon's fault and he just followed for 5 years. We won't know with 100% certainty, I'd find it odd to stay 5 years if you think it doesn't work.

Of course I wouldn't use an LLM to #yolo some Next.js monstrosity with a flavor-of-the-week ORM and random Tailwind. I have, however, had it build numerous parts of my apps after telling it all about the mise targets and tests and architecture of the code that I came up with up front. In a way it vindicates my approach to software engineering because it's able to use the tools available to it to (reasonably) ensure correctness before it says it's done.

Just the other day ChatGPT implemented something that would have taken me a week of research to figure out: in 10 minutes. What do you call that speedup? It's a lot more than 10x.

On other days I barely touch AI because I can write easy code faster than I can write prompts for easy code, though the autocomplete definitely helps me type faster.

The "10x" is just a placeholder for averaging over a series of stochastic exponents. It's a way of saying "somewhere between 1 and infinity"

I'm a half-decent developer with 40 years experience. AI regularly gives me somewhere in the range of 10-100X speed-up of development. I don't benefit from a meme, I do benefit from better code delivered faster.

Sometimes AI is a piece of crap and I work at 0.5X for an hour flogging a dead horse. But those are rarer these days.

(FYI: you can easily bypass the awful logged out view by replacing x.com with xcancel.com, I use a URL Autoredirector rule to do it automatically in Chromium browsers)

https://xcancel.com/karpathy/status/2004607146781278521

[1] https://github.com/zedeus/nitter/wiki/Instances

No it’s absolutely not. But I thought it’d be fun to offer Adams’ brilliant hyperbole for an affectionate ribbing of Karpathy. Both of them are great communicators of ideas.

Something lke the PDF's produced from sent(1) under Unix or MagicPont presentations are many times less fancier and they allow to produce effective no-bullshit ACTUAL product based presentations. But then half of the commercials and managers would actually useless (as they are) and they would be kicked out fast. And don't let me start on nepotism...

The end goal is to get rid of all frontends anyway, just have apps that you interact with through LLM prompts. A more advanced command line.

You can’t, in an honest argument, lump different strangers into a group you invented to accuse them of duplicity or hypocrisy.

Or maybe he didn't lie then but is lying now?

Coding agents are eating up programming from the lowest end, starting from pressing button on the keyboard to type the code in: completion was literally their first application. I don't think it will go all the way to the top, though, the essential part of the profession will remain until true AGI.

Metaphorically, think how integrated chips didn't replace electrical engineering, just changed which production tools and components engineers deal with and how.

Obviously we all are adapting to changes, but if he or someone are panicking about being behind, that can only be because they've never been in too deep.

If only more people understood what quadratic attention means in the real world.