what am I missing here?
edit: lol I "love" that I got downvoted for asking a simple question that might have an open answer. "be curious" says the rules. stay classy HN
Hype. There's nothing wrong with using, e.g., full-text search for RAG.
But realistically lots of RAG systems have LLM calls interleaved for various reasons, so what they probably mean it not doing the usual chunking + embeddings thing.
But I do think there is a qualitative different between getting candidates and adding them to context before generating (retrieval augmented generation) vs the LLM searching for context till it is satisfied.
I really like a lot of what Google produces, but they can't seem to keep a product that they don't shut down and they can be pretty ham-fisted, both with corporate control (Chrome and corrupt practices) and censorship
Nothing in the world is simply outright garbage. Even the seemingly worst products exist for a reason and is used for a variety of use cases.
So, take a step back and reevaluate whether your reply could have been better. Because, it simply "just sucks"
For the command line tool (claude code vs gemini code)? It isn't even close. Gemini code was useless. Claude code was mostly just slow.
I think Claude is much more predictable and follows instructions better- the todo list it manages seems very helpful in this respect.
My tactic is to work with Gemini to build a dense summary of the project and create a high level plan of action, then take that to gpt5 and have it try to improve the plan, and convert it to a hyper detailed workflow xml document laying out all the steps to implement the plan, which I then hand to claude.
This avoids pretty much all of Claude's unplanned bumbling.
I should mention I made that one for my research/stats workflow, so there's some specific stuff in there for that, but you can prompt chat gpt to generalize it.
That's been my experience, anyway. Maybe it hates me? I sure hate it.
It’s not consistent, though. I haven’t figured out what they are but it feels like there are circumstances where it’s more prone to doing ugly hacky things.
(I don’t use any clients that answer coding questions by using the context of my repos).
Either I'm worse than then at programming, to the point that I find an LLM useful and they don't, or they don't know how to use LLMs for coding.
I guess most people are not paying and cant therefore apply the project-space (which is one of the best features), which unleashes its full magic.
Even if I'm currently without a job, I'm still paying because it helps me.
(Not disagreeing, but most of these comments -- on both sides -- are pretty vague.)
And you start from the stratch all the time so you can generate all the documentation before you ever start to generate code. And when LLM slop become overwhelming you just drop it and go to check next idea.
If people are getting faster responses than this regularly, it could account for a large amount of the difference in experiences.
Despite the persistent memes here and elsewhere, it doesn't depend very much on the particular tool you use (with the exception of model choice), how you hold it, or your experience prompting (beyond a bare minimum of competence). People who jump into any conversation with "use tool X" or "you just don't understand how to prompt" are the noise floor of any conversation about AI-assisted coding. Folks might as well be talking about Santeria.
Even for projects that I initiate with LLM support, I find that the usefulness of the tool declines quickly as the codebase increases in size. The iron law of the context window rules everything.
Edit: one thing I'll add, which I only recently realized exists (perhaps stupidly) is that there is a population of people who are willing to prompt expensive LLMs dozens of times to get a single working output. This approach seems to me to be roughly equivalent to pulling the lever on a slot machine, or blindly copy-pasting from Stack Overflow, and is not what I am talking about. I am talking about the tradeoffs involved in using LLMs as an assistant for human-guided programming.
(Though now that I think of it, I might start interrupting people with “SUMMARIZING CONVERSATION HISTORY!” whenever they begin to bore me. Then I can change the subject.)
There are various hacks these tools take to cram more crap into a fixed-size bucket, but it’s still fundamentally different than how a person thinks.
Do you understand yourself what you just said? File is a way to organize data in memory of a computer by definition. When you write instructions to LLM, they persistently modify your prompts making LLM „remember“ certain stuff like coding conventions or explanations of your architectural choices.
> particularly if I have to do it
You have to communicate with LLM about the code. You either do it persistently (must remember) or contextually (should know only in context of a current session). So word „particularly“ is out of place here. You choose one way or another instead of bring able to just tell that some information is important or unimportant long-term. This communication would happen with humans too. LLMs have different interface for it, more explicit (giving the perception of more effort, when it is in fact the same; and let’s not forget that LLM is able to decide itself on whether to remember something or not).
> and in any case, it consumes context
So what? Generalization is an effective way to compress information. Because of it persistent instructions consume only a tiny fraction of context, but they reduce the need for LLM to go into full analysis of your code.
> but it’s still fundamentally different than how a person thinks.
Again, so what? Nobody can keep in short-term memory the entire code base. It should not be the expectation to have this ability neither it should not be considered a major disadvantage not to have it. Yes, we use our „context windows“ differently in a thinking process. What matters is what information we pack there and what we make of it.
I've yet had the "forgets everything" to be a limiting factor. In fact, when using Aider, I aggressively ensure it forgets everything several times per session.
To me, it's a feature, not a drawback.
I've certainly had coworkers who I've had to tell "Look, will you forget about X? That use case, while it look similar, is actually quite different in assumptions, etc. Stop invoking your experiences there!"
A lot of programmers work on maintaining huge monolith codebases, built on top of 10-years old tech using obscure proprietary dependencies. Usually they dont have most of the code to begin with and APIs are often not well documented.
I know it, because I recently learned jj, with a lot of struggling.
If a human struggles learning it, I wouldn't expect LLMs to be much better.
If I only ever wrote small Python scripts, did small to medium JavaScript front end or full stack websites, or a number of other generic tasks where LLMs are well trained I’d probably have a different opinion.
Drop into one of my non-generic Rust codebases that does something complex and I could spent hours trying to keep the LLM moving in the right direction and away from all of the dead ends and thought loops.
It really depends on what you’re using them for.
That said, there are a lot of commenters who haven’t spent more than a few hours playing with LLMs and see every LLM misstep as confirmation of their preconceived ideas that they’re entirely useless.
First of all, keep in mind that research has shown that people generally overestimate the productivity gains of LLM coding assistance. Even when using a coding assistant makes them less productive, they feel like they are more productive.
Second, yeah, experience matters, both with programming and LLM coding assistants. The better you are, the less helpful the coding assistant will be, it can take less work to just write what you want than convince an LLM to do it.
Third, some people are more sensitive to the kind of errors or style that LLMs tend to use. I frequently can't stand the output of LLMs, even if it technically works; it doesn't live to to my personal standards.
I've noticed the stronger my opinions are about how code should be written or structured, the less productive LLMs feel to me. Then I'm just fighting them at every step to do things "my way."
If I don't really have an opinion about what's going on, LLMs churning out hundreds of lines of mostly-working code is a huge boon. After all, I'd rather not spend the energy thinking through code I don't care about.
I don’t think this research is fully baked. I don’t see a story in these results that aligns with my experience and makes me think “yeah, that actually is what I’m doing”. I get that at this point I’m supposed to go “the effect is so subtle that even I don’t notice it!” But experience tells me that’s not normally how this kind of thing works.
Perhaps we’re still figuring out how to describe the positive effects of these tools or what axes we should really be measuring on, but the idea that there’s some sort of placebo effect going on here doesn’t pass muster.
Everytime I tried LLMs, I had the feeling of talking with a ignorant trying to sound VERY CLEVER: terrible mistakes at every line, surrounded with punchlines, rocket emojis and tons of bullshit. (I'm partly kidding).
Maybe there are situations where LLMs are useful e.g. if you can properly delimit and isolate your problem; but when you have to write code that is meant to mess up with the internal of some piece of software then it doesn't do well.
It would be nice to know from each part of the "happy users" and "mecontent usere" of LLMs in what context they experimented with it to be more informed on this question.
But the situation is very different if you’re coding slop in the first place (front end stuff, small repo simple code). The LLMs can churn that slop out at a rapid clip.
The thing is, a lot of the code that people write is cookie-cutter stuff. Possibly the entirety of frontend development. It's not copy-paste per se, but it is porting and adapting common patterns on differently-shaped data. It's pseudo-copy-paste, and of course AI's going to be good at it, this is its whole schtick. But it's not, like, interesting coding.
I get lost a bit at things like this, from the link. The lessons in the article match my experience with LLMs and tools around them (see also: RAG is a pain in the ass and vector embedding similarity is very far from a magic bullet), but the takeaway - write really good prompts instead of writing code - doesn't ring true.
If I need to write out all the decision points and steps of the change I'm going to make, why am I not just doing it myself?
Especially when I have an editor that can do a lot of automated changes faster/safer than grep-based text-first tooling? If I know the language the syntax isn't an issue; if I don't know the language it's harder to trust the output of the model. (And if I 90% know the language but have some questions, I use an LLM to plow through the lines I used to have to go to Google for - which is a speedup, but a single-digit-percentage one.)
My experience is that the tools fall down pretty quickly because I keep trying to make them to let me skip the details of every single task. That's how I work with real human coworkers. And then something goes sideways. When I try to pseudocode the full flow vs actually writing the code I lose the speed advantage, and often end up with a nasty 80%-there-but-I-don't-really-know-how-to-fix-the-other-20%-without-breaking-the-80% situation because I noticed a case I didn't explicitly talk about that it guessed wrong on. So then it's either slow and tedious or `git reset` and try again.
(99% of these issues go away when doing greenfield tooling or scripts for operations or prototyping, which is what the vast majority of compelling "wow" examples I've seen have been, but only applies to my day job sometimes.)
If you use GitHub Copilot - which has its own system level prompts - you can hotswap between models, and Claude outperforms OpenAI’s and Google’s models by such a large margin that the others are functionally useless in comparison.
With a subscription plan, Anthropic is highly incentivized to be efficient in their loops beyond just making it a better experience for users.
Try using opus with cline in vs code. Then use Claude code.
I don't know the best way to quantify the differences, but I know I get more done in CC.
Error: kill EPERM
at process.kill (node:internal/process/per_thread:226:13)
at Ba2 (file:///usr/local/lib/node_modules/@anthropic-ai/claude-code/cli.js:506:19791)
at file:///usr/local/lib/node_modules/@anthropic-ai/claude-code/cli.js:506:19664
at Array.forEach (<anonymous>)
at file:///usr/local/lib/node_modules/@anthropic-ai/claude-code/cli.js:506:19635
at Array.forEach (<anonymous>)
at Aa2 (file:///usr/local/lib/node_modules/@anthropic-ai/claude-code/cli.js:506:19607)
at file:///usr/local/lib/node_modules/@anthropic-ai/claude-code/cli.js:506:19538
at ChildProcess.W (file:///usr/local/lib/node_modules/@anthropic-ai/claude-code/cli.js:506:20023)
at ChildProcess.emit (node:events:519:28) {
errno: -1,
code: 'EPERM',
syscall: 'kill'
}
In any case, I wish it would stay alive and help me lol.
It makes me think that the language/platform/architecture that is "most known" by LLMs will soon be the preferred -- sort of a homogenization of technologies by LLM usage. Because if you can be 10x as successfully vibey in, say, nodejs versus elixir or go -- well, why would you opt for those in a greenfield project at all? Particularly if you aren't a tech shop and that choice allows you to use junior coders as if they were midlevel or senior.
I hear people say things like, “AI isn’t coming for my job because LLMs suck at [language or tech stack]!”
And I wonder, does that just mean that other stacks have an advantage? If a senior engineer with Claude Code can solve the problem in Python/TypeScript in significantly less time than you can solve it in [tech stack] then are you really safe? Maybe you still stack up well against your coworkers, but how well does your company stack up against the competition?
And then the even more distressing thought accompanies it: I don’t like the code that LLMs produce because it looks nothing like the code I write by hand. But how relevant is my handwritten code becoming in a world where I can move 5x faster with coding agents? Is this… shitty style of LLM generated code actually easier for code agents to understand?
Like I said, I don’t endorse either of these ideas. They’re just questions that make me uncomfortable because I can’t definitively answer them right now.
I use some ai tools and sometimes they're fine, but I won't in my lifetime anyway hand over everything to an AI, not out of some fear or anything, but even purely as a hobby. I like creating things from scratch, I like working out problems, why would I need to let that go?
It's really freeing to say "Well, if the linter and the formatter don't catch it, it doesn't matter". I always update lint settings (writing new rules if needed) based on nit PR feedback, so the codebase becomes easier to review over time.
It's the same principle as any other kind of development - let the machine do what the machine does well.
How do we get the LLM to gain knowledge on this new language that we have no example usage of?
So if you need to avoid GC issues, or have robust type safety, or whatever it is, to gain an edge in a certain industry or scenario, you can't just switch to the vibe tool of choice without (best case) giving up $$$ to pay to make up for the inefficiency or (worst case) having more failures that your customers won't tolerate.
But this means the gap between the "hard" work and the "easy" work may become larger - compensation included. Probably most notably in FAANG companies where people are brought in expected to be able to do "hard" work and then frequently given relatively-easy CRUD work in low-ROI ancillary projects but with higher $$$$ than that work would give anywhere else.
And the places currently happy to hire disaffected ex-FAANG engineers who realized they were being wasted on polishing widgets may start having more hiring difficulty as the pipeline dries up. Like trying to hire for assembly or COBOL today.
For now LLMs still suffers from hallucination and lack of generalizability, The large amount of code generated is sometimes not necessarily a benefit, but a technical debt.
LLMs are good for open and fast, prototype web applications, but if we need a stable, consistent, maintainable, secure framework, or scientific computing, pure LLMs are not enough, one can't vibe everything without checking details
For those who’ve built coding agents: do you think LLMs are better suited for generating structured config vs. raw code?
My theory is that agents producing valid YAML/JSON schemas could be more reliable than code generation. The output is constrained, easier to validate, and when it breaks, you can actually debug it.
I keep seeing people creating apps with vibe coder tools but then get stuck when they need to modify the generated code.
Curious if others think config-based approaches are more practical for AI-assisted development.
Then add a grader step to your agentic loop that is triggered after the files are modified. Give feedback to the model if there any errors and it will fix them.
Config files should be mature programming languages, not Yaml/Json files.
Might sound crazy but we built full web apps in just yaml.. Been doing this for about 5 years now and it helps us scale to build many web apps, fast, that are easy to maintain. We at Resonancy[1] have found many benefits in doing so. I should write more about this.
[1] - https://resonancy.io
This is essential to productivity for humans and LLMs alike. The more reliable your edit/test loop, the better your results will be. It doesn't matter if it's compiling code, validating yaml, or anything else.
To your broader question. People have been trying to crack the low-code nut for ages. I don't think it's solvable. Either you make something overly restrictive, or you are inventing a very bad programming language which is doomed to fail because professional coders will never use it.
It dumps out a JSON file as well as a very nicely formatted HTML file that shows you every single tool and all the prompts that were used for a session.
You can see the system prompts too.
It's all how the base model has been trained to break tasks into discrete steps and work through them patiently, with some robustness to failure cases.
That repository does not contain the code. It's just used for the issue tracker and some example hooks.
[1]: https://github.com/badlogic/lemmy/tree/main/apps/claude-brid...
I know, thus the :trollface:
> Happen to know where I can find a fork?
I don't know where you can find a fork, but even if there is a fork somewhere that's still alive, which is unlikely, it would be for a really old version of Claude Code. You would probably be better off reverse engineering the minified JavaScript or whatever that ships with the latest Claude Code.
Had a similar problems until I saw the advice "Dont say what it shouldn't but focus on what it should".
i.e. make sure when it reaches for the 'thing', it has the alternative in context.
Haven't had those problems since then.
I've now gone back to just using vanilla CC with a really really rich claude.md file.
I’m in the middle of some refactoring/bug fixing/optimization but it’s constantly running into issues, making half baked changes, not able to fix regressions etc. Still trying to figure out how to make do a better job. Might have to break it into smaller chunks or something. Been pretty frustrating couple of weeks.
If anyone has pointers, I’m all ears!!
Give programming a try, you might like it.
Next…
Another thing is that before, you were in a greenfield project, so Claude didn't need any context to do new things. Now, your codebase is larger, so you need to point out to Claude where it should find more information. You need to spoon-feed the relevant files with "@" where you want it to look up things and make changes.
If you feel Claude is lazy, force it to use more thinking budget "think" < "think hard" < "think harder" < "ultrathink.". Sometimes I like to throw "ultrathink" and do something else while it codes. [1]
[1]: https://www.anthropic.com/engineering/claude-code-best-pract...
In my case it was exactly the kind of situation where I would also run into trouble on my own - trying to change too many things at once.
It was doing superbly for smaller, more contained tasks.
I may have to revert and approach each task on its own.
I find I need to know better than Claude what is going on, and guide it every step. It will figure out the right code if I show it where it should go, that kind of thing.
I think people may be underestimating / underreporting how much they have to be in the loop, guiding it.
It’s not really autonomous or responsible. But it can still be very useful!
I have also a code reviewer agent in CC that writes all my unit and integration tests, which feeds into my CI/CD pipeline. I use the "/security" command that Claude recently released to review my code for security vulnerabilities while also leveraging a red team agent that tests my codebase for vulnerabilities to patch.
I'm starting to integrate Claude into Linear so I can assign Linear tickets to Claude to start working on while I tackle core stuff. Hope that helps!
Why????????????
Why do you want devs to lose cognaizance of their own "work" to the point that they have "existential worry"?
Why are people like you trying to drown us all in slop? I bet you could replace your slop pile with a tenth of the lines of clean code, and chances are it'd be less work than you think.
Is it because you're lazy?
Actually, no. When LLMs produce good, working code, it also tends to be efficient (in terms of lines, etc).
May vary with language and domain, though.
It may be the size of the changes you're asking for. I tend to micromanage it. I don't know your algorithm, but if it's complex enough, I may have done 4 separate prompts - one for each step.
Let the LLM do the boring stuff, and focus on writing the fun stuff.
Also, setting up logging in Python is never fun.
If it's a new, non-trivial algorithm, I enjoy writing it.
Oh, and the chatbot is cheap. I pay for API usage. On average I'm paying less than $5 per month.
> and I don't have to worry about random hallucinations.
For boilerplate code, I don't think I've ever had to fix anything. It's always worked the first time. If it didn't, my prompt was at fault.
import logging
Configuring it to produce useful stuff (e.g. timestamps, autologging exceptions, etc). Very boilerplate and tedious.
Honestly I don’t think customers care.
That it authored in the first place?
Would you mind linking to your startup? I’m genuinely curious to see it.
(I won’t reply back with opinions about it. I just want to know what people are actually building with these tools!)
Also if the task runs out of context it will get progressively worse rather than refresh its own context from time to time.
A few takeaways for me from this (1) Long prompts are good - and don't forget basic things like explaining in the prompt what the tool is, how to help the user, etc (2) Tool calling is basic af; you need more context (when to use, when not to use, etc) (3) Using messages as the state of the memory for the system is OK; i've thought about fancy ways (e.g., persisting dataframes, parsing variables between steps, etc, but seems like as context windows grow, messages should be ok)
for context, i want to build a claude code like agent in a WYSIWYG markdown app. that's how i stumbled on your blog post :)
Nice, do share a link, would love to check out your agent!
if true this seems like a bloated approach but tbh I wouldn't claim to know totally how to use Claude like the author here...
I find you can get a lot of mileage out of "regular" prompts, I'd call them?
Just asking for what you need one prompt at a time?
I still can't visualize how any of the complexity on top of that like discussed in the article adds anything to carefully crafted prompts one at a time
I also still can't really visualize how claude works compared to simple prompts one at a time.
Like, wouldn't it be more efficient to generate a prompt and then check it by looping through the appendix sections ("Main Claude Code System Prompt" and "All Claude Code Tools"), or is that basically what the LLM does somewhat mysteriously (it just works)? So like "give me while loop equivalent in [new language I'm learning]" is the entirety of the prompt... then if you need to you can loop through the appendix section? Otherwise isn't that a massive over-use of tokens, and the requests might even be ignored because they're too complex?
The control flow eludes me a bit here. I otherwise get the impression that the LLM does not use the appendix sections correctly by adding them to prompts (like, couldn't it just ignore them at times)? It would seem like you'd get more accurate responses by separating that from whatever you're prompting and then checking the prompt through looping over the appendix sections.
Does that make any sense?
I'm visualizing coding an entire program as prompting discrete pieces of it. I have not needed elaborate .md files to do that, you just ask for "how to do a while loop equivalent in [new language I'm learning]" for example. It's possible my prompts are much simpler for my uses, but I still haven't seen any write-ups on how people are constructing elaborate programs in some other way.
Like how are people stringing prompts together to create whole programs? (I guess is one question I have that comes to mind)
I guess maybe I need to find a prompt-by-prompt breakdown of some people building things to get a clearer picture of how LLMs are being used
Make sure you read it first though... I believe it expected Req to be present as a dependency when generating code that makes HTTP requests.
This server could not prove that it is minusx.com; its security certificate expired 553 days ago
Am I missing something here? Or is this just Anthropic shilling?
Claude can run commands to search code, test compilation, and perform various other operations.
Unix is great because its commands are well-documented, and the training data is abundant with examples.
LaGrange•12h ago
dang•12h ago
The idea here is: if you have a substantive point, make it thoughtfully. If not, please don't comment until you do.
https://news.ycombinator.com/newsguidelines.html
dingnuts•12h ago
Is this why HN is so dang pro-AI? the negative comments, even small ones, are moderated away? explains a lot TBH
h4ch1•12h ago
LaGrange•11h ago
Edit: bonus points if this gets me banned.
dang•6h ago
If it felt like a waste of time and energy to post something substantive, rather than the GP comment (https://news.ycombinator.com/item?id=44998577), then you should have just posted nothing. That comment was obviously neither substantive nor thoughtful. This is hardly a borderline call!
We want substantive, thoughtful comments from people who do have the time and energy to contribute them.
Btw, to avoid a misunderstanding that sometimes shows up: it's fine for comments to be critical; that is, it's possible to be substantive, thoughtful, and critical all at the same time. For example, I skimmed through your account's most recent comments and saw several of that kind, e.g. https://news.ycombinator.com/item?id=44299479 and https://news.ycombinator.com/item?id=42882357. If your GP comment had been like that, it would have been fine; you don't have to like Claude Code (or whatever the $thing is).
exe34•11h ago
danielbln•11h ago
dang•6h ago