I got into programming because the problems of programming were interesting to me. But if the problems go from "figure out why this calculator is off by one in France" to "Get this LLM to stop spamming cutsey emojis", then maybe it's time for a career change.
I'm not looking forward to using computers or technology over the next decade. There is a non-zero chance myself or a loved one is killed because of vibe coding.
This is what you want. You want comprehensive tests at every level, far more than is reasonable for a human to build or maintain, from unit, functional, to full end to end and beyond. Adversarial testing (both TDD-style "write tests to demonstrate this bug", and posthoc "prove this patch wrong with a new test") is the best way to keep AI on track and make those diffs you have to read clean and easy.
An even better way is to use a more strongly typed language and really lock it down, but you can use testing in any language. I feel like my background in TDD and "TATFT" has been secret sauce when working with AI
https://github.com/dprkh/eventfs
It has good test coverage, mostly unit tests but also a number of end-to-end tests. I also made the LLM build a benchmark, which you can find at the bottom of the readme. It is obviously slow, but I thought that it is good enough to work. When I tried to write a 1 GiB file, I found that it broke down, and after writing half the file, the speed went to under one megabyte per second. Implementation is 10k+ LoC, and I have no idea what is going on there.
Just because we work with computers doesn't mean we don't take, er, social-damage. Or perhaps parasocial damage, in this case.
Anyone else working on something like this or know of any projects attempting it?
Directionally if what you're doing is straightforward it's an amazing experience to be able to slap in an epic planning document and wake up the next day to it being "done", with a big asterisk that done-ness is directly proportional to how good of a spec and how good of a model you were using.
That being said, these days if you use Fable, slap in an epic planning document, and ask it to run a workflow (be sure to specify that subagents should use, say, Sonnet, or wave goodbye to your wallet), it's almost as good as gastown/gascity but far more predictable.
I am happy about all the little side-projects, and ideas it help my realize, and I enjoy exploring this new world, but I've noticed LLMs feed my unhealthy "don't want to take a break and waste time being idle" mindset, and I need to correct it.
W.r.t. article's main complain - I think the similar thing happened due to factory manufacturing automation. What used to be a varied skillful craft in a shop became standing in a single place of an assembly line doing the exact same thing whole day. LLM took away the more creative and variable part of the work, and left the repetitive QA rubber-stamping. Probably some of the mitigations used back then could be rediscovered today.
I confess that the above variant on the quotation is how I originally read it. And that's just about how I feel now with trying to sort through vibe-coded slop projects that are put forth by (well-meaning, probably good intentioned, not evil) people who represent them as if they're the handcrafted result of one dedicated developer.
"I did a Chat output, please fix and review it " is the kind of thing that empowers the people who used to have a minimal productivity, and now lets them to wreck things on an industrial scale.
Getting sent IM responses that are copy pasted LLM nonsense. Getting a massive PR to review that was generated overnight and the author didn't read it first.
My mind still can't function well without having knowledge about everything.
The productivity drive and the sheer feature set you can generate in record time makes it easy to forget proper sdlc hygiene.
Of course if you're supposed to achieve so much output that it's not possible to do anything but vibe it, fair enough.
I do not understand these complaints. Yes, those are the defaults and they're annoying, although the general public seems to like them. But you are not stuck with these. You can just tell the LLM how it should interact with you. If you're using any sort of harness beyond the chat window in a web browser, you can codify these instructions in a rules.md file or similar and have it automatically included in any new chat. It's not any harder than changing the default wallpaper or color scheme on your desktop operating system.
In reverse order, you can just tell the LLM to never use emojis. I don't like emphatic staccato fragments either, so I tell it to eschew the language of marketing and hype and stick to a factual and plain language, or to employ an academic tone. I explicitly instruct mine to ask clarifying questions whenever context is ambiguous and to push back on false assumptions or common misconceptions (by me). Hallucinationsa re the biggest problem of those you mention; it's not easy to totally eliminate them (for the same reason it's not easy to instruct people to not fall for scams or disinformation), but you can considerably reduce them by setting standards for citations.
I have ideas about reducing hallucinations over work material (ie a codebase) but am omitting them here as they are not fully thought out or tested.
Prior to the last 12mos AI companies were hell bent on squeezing out the best results from mediocre models.
But... now that the top models have progressed, those same AI companies have switched their efforts into reducing the computation (cost of a producing a result) as much as possible without being too obvious.
What was an exponential slope in the quality of results over the last 36 months has now nearly flat lined.
Have your fun and whimsy.
It'd be rather beautiful if all jobs were purely passion driven, but that is simply not the case. Nor could it be. And yeah, there are programmers with jobs that are mostly driven by passion, but most would pack it up and go home immediately if there was a sudden "we have stopped paying you" announcement.
And the majority of software is terrible so ya. Life is generally unfortunate.
My latest is, I'm really into fizzy/soda water and wanted my own continuous carbonator. My entire build from water source to tap with an ESP32 controlled pump, pressure, water level, cooling fans.
There were so many areas I made mistakes in my shopping cart and it found it - like Home Brewer likes 8mm lines but water filter systems like 9.5mm. Really optimized the versions from a simple on/off pump w/ float switch to effectively a full on PLC system. So many iterations gained by chatting with "someone more experienced". Once I get the parts I can build and have the software side running in less than an hour.
It doesn't make money, but man I really enjoy it.
The experience is much closer to working with an external API that you don't have control over and which simply doesn't do what the documentation says. Those have always been the most frustrating parts of programming, but at least previously you could reverse engineer the actual implementation to work around bugs. You can't even do that now because the "boundary" randomly change every day.
At least with agent-run tests I care about loop speed a lot, but I care about complete coverage more, so having the odd heavy weight full stack integration test is fine, I think.
Yes tests are conceptually isolated and that helps, but I've personally seen unit tests get generated that are semantically incorrect - that is, they test the structure of the code (e.g. they can check function output types and values), but they can't know _why_ the unit tests need to be there, so the really really helpful tests never get generated. Not to mention the obvious issues with generated tests only testing is x = x, or needless redundant tests for the same thing, or them essentially testing basic features of the language.
I actually have a public (AGPL) example here: https://github.com/pgdogdev/pgdog/tree/main/integration/sql - pgdog is particularly testable since it is trying for complete transparency, so you have a perfect oracle in hand via base postgresql, but it demonstrates the concept at least.
I've been burned by this in my honeymoon period with unit testing (pretty much the reason it ended). These days, I prefer broader scope of testing, especially user-facing part. The users may be other developers or end users. I only do unit testing for tricky algorithms or math formulae.
It is different though. Basically a lot of what I do has changed over the last 2 years. I totally get that a lot of people won't want to adapt though.
Or people don't want to be reverse centaur keeping the clankers happily running. Instead of helping to solve users/consumers problem.
I save myself by skimming things like tests, templates, some UI. Anything cosmetic. But I have to read the majority of code that ends up on my back end systems.
In my personal experience, the ones most enthusiastic about LLM magic are those that can't code, but can now walk away with something functional if not quite the best code. Now that they can produce workable code, it will make everyone better. Yet, they have no idea how maintainable the slop is or if it's slop at all.
When you see a perfectly clear function or object that just isn't your style, you have to accept it and move on. Where there are concrete concerns, or it's unreadable, demand excellence, but treat it like a coworker, not an IDE.
It's not. There is no one person that has universally good taste. Also, we're not in your head, no matter how much better of a coder or whatever. We're not in your head and it's all terribly painful to navigate.
This 1000 times. The number of times I now have to use my brain to do something engineer-y is now something like 5 times per week, usually eyeballing some architectural decision from (as you mentioned above) someone else's LLM.
I really loved writing code, and I loved that I could do it for a living. Obviously, nothing is stopping me from continuing to write code and solve problems with my own brain, but that's absolutely not the efficient way to work anymore.
The most high-leverage activity I've done in the last 6 months is a build-out of internal AI orchestration platforms and data access layers so every employee in the company can built with AI against our own data (particularly non-engineers). It has unlocked so much for everyone. Yet, my only real claim is a) coming up with the idea and general architecture and b) ensuring smooth roll-outs and hosting "office hours" for months to help new employees onboard to these new AI tools I built.
High-level design and architecture decisions, quality assurance, and balancing decision-making through subjective areas like UI/UX. That's mostly all I do. Now that I'm leading engineering, I just think even higher-level while the engineers use orchestrated AI do built vast swaths of code. Then I frantically bounce around approving sensitive PRs and architecture decisions -- most of which were written by Claude, described by Claude, and usually don't need a lot of correction.
hoppp•1h ago
https://github.com/JuliusBrussee/caveman
It's for getting it to output shorter answers, but also could help with your burnout.
ysavir•58m ago