Why do people think it means we can write enterprise applications without understanding the code/specifications?
The quip that “there’s nothing more permanent than a temporary solution” has been a truism of software engineering since long before AI arrived on the scene, vibe coding is just making the problem much worse.
I'm mildly optimistic that vibe coding won't make the problems that much worse and can actually lift the standards of quality in some cases. In my own personal careless / throwaway bash scripts, I've been using AI for them more and more, but I also notice that it puts more care into them than I otherwise would have with respect to things like error checking, friendlier help and other options, progress echos, and comments. AI tools still have a ways to go on larger projects though, and hallucinations seem particularly bad when it comes to foreign library bindings...
I've had a few AI generated PRs come my way and the code-review process is, shall we say, not fun. It takes me a lot more time to review these PRs, there is way more back-and-forth between me and the 'developer', and it takes much more time to get the PR merged. That's not saying anything about the increased difficulty in modifying this code in the future.
I have a feeling these claims of being more productive don't account for the entire development cycle.
Easy does not necessarily mean more productive when you're trading ease for something else. In the case of coding, you're trading ease for things like understanding and maintainability.
If you’re throwing the LLM at APIs you don’t know, how could you possibly verify it is using them properly?
Sure AI increases developer output, which is sometimes correlated with productivity -- but often times not. Insofar as AI is accelerating positive outcomes (we were able to write some tricky code that was blocking us), it's also accelerating the negative outcomes (we used the LLM to write 40k lines of code in an hour and no one know what any of it does). One of these things is "productive" the other is just performative work.
If "being more productive" is using an ai to write an email which is then summarized by AI on the receiving end, or students using AI to write papers which are graded by AI, or developers using AI to write code which is then reviewed by AI, then AI isn't actually making anything better.
I’d like to see the proof for TDD; last I heard it slowed development with only minor reliability improvements.
YMMV though.
My personal experience (and I think the experience of many who do it full time) is that it makes things faster.
How did I test and debug? Run my code and printf.
The former is desirable, not common. The latter is common, not desirable.
What it boils down to: - TDD in the hands of a junior is very good. Drastically reduces bugs, and teaches the junior how to write code that can be tested and is not just a big long single method of spaghetti with every data structure represented as another dimension on some array.
- TDD in the hands of a midlevel can be a mixed bag. They've learned how to do TDD well, but have not learned when and why TDD can go bad. This creates design damage, where everything is shoe-horned into TDD and the goal of 90% line coverage is a real consideration. This is maximum correctness but also potentially maximum design damage.
- TDD in the hands of a senior is a power tool. The "right" tests are written for the right reasons with the right level of coupling and the tests overall are useful. Every really complicated algorithm I've had to write, TDD was a life saver for getting it landed.
Feels a lot like asking someone if they prefer X or Y and they say "X" is the industry best practice. My response universally is now an eye brow raise "oh, is it? For which segments of the industry? Why? How do we know it's actually a best practice? Okay, given our context, why would it be a best practice for US". Juniors don't know the best practices, mid-levels apply them everywhere, seniors evaluate and consider when best practices are not best practices.
TDD slows development when tests are written in a blind way with an eye on code coverage and not correctness and design. TDD speeds up development in being a good way to catch errors and is one of the best ways to ensure correctness.
I’ll take your comment as testing is good and constraining your workflow to TDD is worthless.
That's just writing a regression test and making sure it catches the regression. What does that have to do with TDD? Does the philosophy of TDD lay claim to any test written before the bugfix, regardless of how much or little someone subscribes to TDD overall?
It can provide high level guardrails confirming implementation correctness that are as indifferent to software design as possible (giving freedom to refactor).
The worst actors find ways to make other people responsible for fixing their bugs.
In general, doing things work, planning to do things don't.
Most people prefer to play around and make several crappy attempts and combine them until the whole is somewhat solved, then go over and polish it a little, and maybe then add tests and fix the behavior in place.
For this last group, TDD it's jarring, unnatural and requires a lot of willpower to follow.
It's not bad in itself, it's just not for everyone.
I don't think that's the case. If they were really thinking up-front, they'd be doing proper req analysis and design work, rather than interactively growing a ball of mud that "does the minimal thing to pass a test". To me, it seems like TDD is sold as this "foolproof" design / dev approach, which is anything but.
The common struggle with TDD arises when people reduce it to writing glorified spell checkers. This usually goes hand in hand with the belief that unit tests must always check classes in isolation using excessive mocks—an approach that misses the real purpose of TDD.
It has less to do with thought patterns and more with simply misunderstanding the approach entirely because of clinging to the wrong dogmas they've heard somewhere.
Norvig starts with the theory building and creates a constraint solver in about 50 lines of code. Jeffries starts with TDD, assumes an implementation, has to change that implementation, therefore has to change the tests, and after a series of five blog posts kind of fizzles out on it.
To me it just highlights that defining the problem is with so much more than defining the tests, as you can’t write a test for a problem you haven’t defined yet. In this way the tests are an imposition. In short to me it shows that TDD only really works if you already knew how to buold the project to begin with.
[1] https://norvig.com/sudoku.html [2] https://ronjeffries.com/xprog/articles/oksudoku/
Once the business sees that the prototype more or less works, it's incredibly difficult to get them to spend money on a "sane" clean-sheet rewrite.
The paradox is that the better LLMs get, the more serious the bugs will be because the software will seem ok, only to blow up after people have developed a false sense of security.
VCs hope that with AI they can have a larger portfolio, shipping more things, so that by sheer luck, one is a success. That's why many employees are critical of the AI hype while VCs and C-level love it. The whole discussion about maintainability doesn't even register on the radar, employees vs. VCs and C-level are operating at a different definition of "failure".
You don’t. You either don’t get that, or you do but would rather people not know that you really just wanted to destroy a competitor and snake some of their customers.
If an AI-boosted startup makes it, the tech debt inside the company will be worse than in a "traditional" one. That seems like a net negative for society in the long run.
Dev: enables verbose/debug logging
App: encounters error, creating big log file
Dev: uploads entire logfile, containing secrets, to 3rd party LLM and asks "read this log and identify the problem"
meanwhile...
LLM: leaks prompt, logs, and secrets to hackers
LLM: uses prompt for training data, then provides secrets as responses to other users
When you’re trying to preserve features but fix bugs this information saves a lot of time and helps prevent regressions.
There are 2 instances of the word "understand" in the first paragraph, 3 if you count the beginning of the second.
In my book, "understanding" is a synonym for "intelligence" - the roots of the word are "read between the lines", where something else that just knowledge is, the ability to use and manipulate knowledge [1].
But the thing is, despite this tech being classified as "artificial intelligence", it does not understand a thing - or so little.
So, if we extrapolate Betteridge's law of headlines, no it is not temporary for this type of technology. But I think connecting it with formal computations - inference engines for formal logic, calculators [2], etc. could be amazing.
[1] https://en.wiktionary.org/wiki/intelligence - well, yes, that's a bit cherry-picked.
[2] https://arxiv.org/abs/2406.03445 - amazing result, and maybe will find out that it is exactly what we do "under the skull", but doing arithmetic with Fourier transforms is not the best use of a microprocessor.
* Optimized for task completion, with limited attention resources for global alignment (RL/RLAIF reward loops/hacking)
* These systems run outside of chat now. file systems, CLIs, DBs, browsers → real-world side‑effects that you cannot train for. Hallucination becomes a problem of contradiction in the real world, and alignment is something an agent will struggle with as it's optimized to complete tasks. Which is why you see things like databases being dropped today.
* These are baked-in problems, not even considering the the adversarial nuances of things like prompt injection.
Maybe it's cliche from an AI safety perspective. But I can never get over https://en.wikipedia.org/wiki/Instrumental_convergence as we see micro instances of it in our day-to-day with today's agents. Again, an issue that has existed from the dawn of these types of models. https://www.youtube.com/watch?v=s5qqjyGiBdc&t=1853s
I don't find that this requires discipline. AI code simply requires code review the same as anything else. I don't feel the need to let AI code in unchecked in the same way I don't feel the need to go to my pull request page one day and gleefully hit approve and merge on all of them without checking anything.
Imagine the 3 AM on-call alert. The engineer trying to fix it might be navigating a section of the codebase they've never seen before, generated entirely by an AI. In this scenario, you can't afford to vibe it out or gamble precious minutes while an AI agent attempts a fix. You need ground truth, and you need it fast.
Why automate the fun part and keep a human for the shitty part?
I don't really get the reasoning behind all the hype, or better said: I kinda do, but it's more of a knee jerk reaction or essentially FOMO.
What makes me think this is a bubble is the amount of emotion behind the decision making process (plus the fact that almost nobody is making a dime with this so far).
Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?
— The Elements of Programming Style, 2nd edition, chapter 2
So if vibe coding produces code that is as clever as -- or more clever than -- you, then you have no chance of debugging it.
The (occasionally) surprising part is that there are times where the generated clarifying questions actually spawn questions of my own. Making the process more interactive is sort of like a pseudo rubber duckie process: forcing yourself to specifically articulate ideas serves to solidify and improve them.
jihadjihad•5mo ago
Almost every day on this site for the past few months has been an instance of Mugatu's "I feel like I'm taking crazy pills!" moment.
rafterydj•5mo ago