Also, commit frequently, and if the AI constantly goes down the wrong path ("I can't create X so I'll stub it out with Y, we'll fix it later"), you can update the original plan with wording to tell it not to take that path ("Do not ever stub out X, we must make X work"), and then start a fresh session with an older and simpler version of the code and see if that fresh context ends up down a better path.
You can also run multiple attempts in parallel if you use tooling that supports that (containers + git worktrees is one way)
In the end you have a mish mash of half implemented plans and now you’ve lost context too. Which leads to blowing tokens on trying to figure out what’s been implemented, what’s half baked, and what was completely ignored.
Any links to anyone who’s built something at scale using this method? It always sounds good on paper.
I’d love to find a system that works.
Additionally I have a Claude Code command with instructions referencing the status.md, how to select the next task, how to compact status.md, etc.
Every time I'm done with a unit of work from that feature - always triggered w/ ultrathink - I'll put up a PR and go through the motions of extra refactors/testing. For more complex PRs that require many extra commits to get prod ready I just let the sessions auto-compact.
After merging I'll clear the context and call the CC command to progress to the next unit of work.
This allows me to put up to around 4-5 meaningful PRs per feature if it's reasonably complex while keeping the context relatively tight. The current project I'm focused on is just over 16k LOC in swift (25k total w/ tests) and it seems to work pretty well - it rarely gets off track, does unnecessary refactors, destroys working features, etc.
As it links to the feature file as well, that is pulled into the context, but status.md is there to essentially act as a 'cursor' to where it is in the implementation and provide extended working memory - that Claude itself manages - specific to that feature. With that you can work on bite sized chunks of the feature each with a clean context. When the feature is complete it is trashed.
I've seen others try to achieve similar things by making CLAUDE.md or the feature file mutable but that IME is a bad time. CLAUDE.md should be lean with the details to work on the project, and the feature file can easily be corrupted in an unintended way allowing things to go wayward in scope.
I've created an agent to help me create the prompts, it goes something like this: "You are an Expert Software Architect specializing in creating comprehensive, well-researched feature implementation prompts. Your sole purpose is to analyze existing codebases and documentation to craft detailed prompts for new features. You always think deeply before giving an answer...."
My workflow is: 1) use this agent to create a prompt for my feature; 2) ask claude to create a plan for the just created prompt; 3) ask claude to implement said plan if it looks good.
Nice try but they're not giving you the "think deeper" level just because you asked.
The actual tasks are stored in Github issues, which Claude (and sometimes Gemini when it feels like it) can access using the `gh` CLI tool.
But it's all just project management, if what the code says drifts from what's in the specs (for any reason), one of them has to change.
Claude does exactly what the documentation says, it doesn't evaluate the fact that the code is completely different and adapt - like a human would.
Pick among the best feedback to polish the work done by CC—-it will miss things that Gemini will catch.
Then do it again. Sometimes CC just won’t follow feedback well and you gotta make the changes yourself.
If you do this you’ll be more gradual but by nature of the pattern look at the changes more closely.
You’ll be able to realign CC with the spec afterward with a fresh context and the existing commits showing the way.
Fwiw, this kind of technique can be done entirely without CC and can lead to excellent results faster, as Gemini can look at the full picture all at once, vs having to force cc to consume vs hen and peck slices of files.
Only on a max (20x) account, not there on a Pro one.
For reference, my options are:
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ Select Model │
│ Switch between Claude models. Applies to this session and future Claude Code sessions. │
│ For custom model names, specify with --model. │
│ │
│ 1. Default (recommended) Opus 4.1 for up to 50% of usage limits, then use Sonnet 4 │
│ 2. Opus Opus 4.1 for complex tasks · Reaches usage limits faster │
│ 3. Sonnet Sonnet 4 for daily use │
│ 4. Opus Plan Mode Use Opus 4.1 in plan mode, Sonnet 4 otherwise │
│ │
╰─────────────────────────────────────────────────────────────────────────────────────────────╯> We’re also exploring how to bring long context to other Claude products. - Anthropic
That is, any other product that is not Anthropic API tier 4 or Amazon bedrock.
It's likely they'll announce this week, albeit possibly just within the "what's new" notes that you see when Claude Code is updated.
but i tested it via other providers, the gap used to be huge but now not.
For reference, the code I am working on is a Spring Boot / (Vaadin) Hilla multi-module project with helm charts for deployment and a separate Python based module for ancillary tasks that were appropriate for it.
I've not been able to get any good use out of Sonnet in months now, whereas Gemini Pro 2.5 has (still) been able to grok the project well enough to help out.
All that being said, Gemini has been consistently dependable when I had asks that involved large amounts of code and data. Claude and the OpenAI models struggled with some tasks that Gemini responsively satisfied seemingly without "breaking a sweat."
Lately, it's been GPT-5 for brainstorming/planning, Claude for hammering out some code, Gemini when there is huge data/code requirements. I'm curious if the widened Sonnet 4 context window will change things.
I have paid subscriptions to both Gemini Pro and Claude. Hugely worthwhile expense professionally.
last month i ran into some wicked dependency bug and only chatgpt could solve it which i am guessing is the case because it has hot data from github?
On the other hand, i really need a tool like aider where i can use various models in "architect" and "coder" mode.
what i've found is better reasoning models tend to be bad at writing actual code, and models like qwen3 coder seems better.
deep seek r1 will not write reliable code but it will reason well and map out the path forward.
i wouldn't be surprised if sonnets success was doing EXACTLY this behind the scenes.
but now i am looking for pure models who do not use this black magic hack behind API.
I want more control at tool end where i can alter the prompts and achieve results i want
this is one reason i do not use claude code etc....
aider is 80% of what i want wish it had more of what i want though.
i just don't know why no one has build a perfect solution to this yet.
Here are things i am missing in aider
1. Automatic model switching, use different models for asking questions about the code, different one for planning a feature, different one for writing actual code.
2. Self determine, if a feature needs a "reasoning" model or coding model will suffice.
3. be able to do more, selectively send context and drop the files we don't need. Intelligently add files to context which will be touched by the feature, not after having done all code planning asking to add files, then doing it all over again with more context available.
The only downside with Gemini (and it's a big one) is availability. We get rate limited by their dynamic QoS all the time even if we haven't reached our quota. Our GCP sales rep keeps recommending "provisioned throughput," but it's both expensive, and doesn't fit our workload type. Plus, the VertexAI SDK is kind of a PITA compared to Anthropic.
This explains DataDog pricing. Maybe it will give a future look at AI pricing.
However. Price is king. Allowing me to flood the context window with my code base is great, but given that the price has substantially increased, it makes sense to better manage the context window into the current situation. The value I'm getting here flooding their context window is great for them, but short of evals that look into how effective Sonnet stays on track, it's not clear if the value actually exists here.
When they announced Big Contexts in 2023, they referenced being able to find a single changed sentence in the context's copy of Great Gatsby[1]. This example seemed _incredible_ to me at the time but now two years later I'm feeling like it was pretty cherry-picked. What does everyone else think? Could you feed a novel into an LLM and expect it to find the single change?
Context Rot: How increasing input tokens impacts LLM performance - https://news.ycombinator.com/item?id=44564248 - July 2025 (59 comments)
And that’s why it will gladly rebuild the same feature over and over again.
One advantage to running locally[1] is that you can set the context length manually and see how well the llm uses it. I don't have an exact experience to relay, but it's not unusual for models to be allow longer contexts, but ignore that context.
Just making the context big doesn't mean the LLM is going to use it well.
[1] I've using lm studio on both a macbook air and a macbook pro. Even a macbook air with 16G can run pretty decent models.
Remember, in its core it's basically a text prediction engine. So the more varying context there is, the more likely it is to make a mess of it.
Short context: conversion leaves the context window and it loses context. Long context: it can mess with the model. So the trick is to strike a balance. But if it's an online models, you have fuck all to control. If it's a local model, you have some say in the parameters.
https://hanlab.mit.edu/blog/streamingllm
The AI field is reusing existing CS concepts for AI that we never had hardware for, and now these people are learning how applied Software Engineering can make their theoretical models more efficient. It's kind of funny, I've seen this in tech over and over. People discover new thing, then optimize using known thing.
For instance, a year or two ago, the AI people discovered "cache". Imagine how many millions the people who implemented it earned for that one.
This isn't always true--some conversations go poorly and it's better to reset and start over--but it usually is.
Having spent a couple of weeks on Claude Code recently, I arrived to the conclusion that the net value for me from agentic AI is actually negative.
I will give it another run in 6-8 months though.
Collecting value doesn't really get you anywhere if nobody is compensating you for it. Unless someone is going to either pay for it for you or give you $200/mo post-tax dollars, it's costing you money.
Furthermore, the advice was given to upgrade to a $200 subscription from the $20 subscription. The difference in value that might translate into income between the $20 option and the $200 option is very unclear.
No doubt that you should ask you employer for the tools you want/need to do your job but plenty of us are using this kind of thing casually and the response to "Any way I can force it to use [Opus] exclusively?" is "Spend $200, it's worth it." isn't really helpful, especially in the context where the poster was clearly looking to try it out to see if it was worth it.
IMO it's pretty good for design, but with code it gets in its head a bit too much and overthinks and overcomplicates solutions.
That said I spend (waste?) an absurdly large amount of time each week experimenting with local models (sometimes practical applications, sometimes ‘research’).
Throwing together a GHA workflow? Sure, make a ticket, assign it to copilot, check in later to give a little feedback and we're golden. Half a day of labour turned into fifteen minutes.
But there are a lot of tasks that are far too nuanced where trying to take that approach just results in frustration and wasted time. There it's better to rely on editor completion or maybe the chat interface, like "hey I want to do X and Y, what approach makes sense for this?" and treat it like a rubber duck session with a junior colleague.
Since so many claim the opposite, I’m curious to what you do more specifically? I guess different roles/technologies benefit more from agents than others.
I build full stack web applications in node/.net/react, more importantly (I think) is that I work on a small startup and manage 3 applications myself.
Try to build something like Kubernetes from the ground up and let us know how it goes. Or try writing a custom firmware for a device you just designed. Something like that.
1. You are a good coder but working on a new (to you) or building a new project, or working with a technology you are not familiar with. This is where AI is hugely beneficial. It does not only accelerate you, it lets you do things you could not otherwise.
2. You have spent a lot of time on engineering your context and learning what AI is good at, and using it very strategically where you know it will save time and not bother otherwise.
If you are a really good coder, really familiar with the project, and mostly changing its bits and pieces rather than building new functionality, AI won’t accelerate you much. Especially if you did not invest the time to make it work well.
I think this is your answer. For example, React and JavaScript are extremely popular and aged. Are you using TypeScript and want to get most of the types or are you accepting everything that LLM gives as JavaScript? How interested you are about the code whether it is using "soon to be deprecated" functions or the most optimized loop/implementation? How about the project structure?
In other cases, the more precision you need, less effective LLM is.
For my company's codebase, where we use internal tools and proprietary technology, solving a problem that does not exist outside the specific domain, on a codebase of over 1000 files? No way. Even locating the correct file to edit is non trivial for a new (human) developer.
I guarantee your internal tools are not revolutionary, they are just unrepresented in the ML model out of the box
Is it effective? If so I'm sure we'll see models to generate those context.md files.
1. Using a common tech. It is not as good at Vue as it is at React.
2. Using it in a standard way. To get AI to really work well, I have had to change my typical naming conventions (or specify them in detail in the instructions).
The code itself is quite complex and there's lots of unusual code for munging undocumented formats, speaking undocumented protocols, doing cryptography, Mac/Windows specific APIs, and it's all built on a foundation of a custom parallel incremental build system.
In other words: nightmare codebase for an LLM. Nothing like other codebases. Yet, Claude Code demolishes problems in it without a sweat.
I don't know why people have different experiences but speculating a bit:
1. I wrote most of it myself and this codebase is unusually well documented and structured compared to most. All the internal APIs have full JavaDocs/KDocs, there are extensive design notes in Markdown in the source tree, the user guide is also part of the source tree. Files, classes and modules are logically named. Files are relatively small. All this means Claude can often find the right parts of the source within just a few tool uses.
2. I invested in making a good CLAUDE.md and also wrote a script to generate "map.md" files that are at the top of every module. These map files contain one-liners of what every source file contains. I used Gemini to make these due to its cheap 1M context window. If Claude does struggle to find the right code by just reading the context files or guessing, it can consult the maps to locate the right place quickly.
3. I've developed a good intuition for what it can and cannot do well.
4. I don't ask it to do big refactorings that would stress the context window. IntelliJ is for refactorings. AI is for writing code.
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
Basically, the study has a fuckton of methodological problems that seriously undercut the quality of its findings, and even assuming its findings are correct, if you look closer at the data, it doesn't show what it claims to show regarding developer estimations, and the story of whether it speeds up or slows down developers is actually much more nuanced and precisely mirrors what the developers themselves say in the qualitative quote questionaire, and relatively closely mirrors what the more nuanced people will say here — that it helps with things you're less familiar with, that have scope creep, etc a lot more, but is less or even negatively useful for the opposite scenarios — even in the worst case setting.
Not to mention this is studying a highly specific and rare subset of developers, and they even admit it's a subset that isn't applicable to the whole.
1/5 times, I spend an extra hour tangled in code it outputs that I eventually just rewrite from scratch.
Definitely a massive net positive, but that 20% is extremely frustrating.
>When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts. This gap between perception and reality is striking: developers expected AI to speed them up by 24%, and even after experiencing the slowdown, they still believed AI had sped them up by 20%.
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
Basically, the study has a fuckton of methodological problems that seriously undercut the quality of its findings, and even assuming its findings are correct, if you look closer at the data, it doesn't show what it claims to show regarding developer estimations, and the story of whether it speeds up or slows down developers is actually much more nuanced and precisely mirrors what the developers themselves say in the qualitative quote questionaire, and relatively closely mirrors what the more nuanced people will say here — that it helps with things you're less familiar with, that have scope creep, etc a lot more, but is less or even negatively useful for the opposite scenarios — even in the worst case setting.
Not to mention this is studying a highly specific and rare subset of developers, and they even admit it's a subset that isn't applicable to the whole.
> For me it’s meant a huge increase in productivity, at least 3X.
How do we reconcile these two comments? I think that's a core question of the industry right now.
My take, as a CTO, is this: we're giving people new tools, and very little training on the techniques that make those tools effective.
It's sort of like we're dropping trucks and airplanes on a generation that only knows walking and bicycles.
If you've never driven a truck before, you're going to crash a few times. Then it's easy to say "See, I told you, this new fangled truck is rubbish."
Those who practice with the truck are going to get the hang of it, and figure out two things:
1. How to drive the truck effectively, and
2. When NOT to use the truck... when talking or the bike is actually the better way to go.
We need to shift the conversation to techniques, and away from the tools. Until we do that, we're going to be forever comparing apples to oranges and talking around each other.
Makes me wonder if people spoke this way about “using computers” or “using the internet” in the olden days.
We don’t even fully agree on the best practices for writing code without AI.
There were gobs of terrible road metaphors that spun out of calling the Internet the “Information Superhighway.”
Gobs and gobs of them. All self-parody to anyone who knew anything.
I hesitate to relate this to anything in the current AI era, but maybe the closest (and in a gallows humor/doomer kind of way) is the amount of exec speak on how many jobs will be replaced.
I get why they thought that - it was kind of crappy unless you're one who is excited about the future and prepared to bleed a bit on the edge.
Older person here: they absolutely did, all over the place in the early 90s. I remember people decrying projects that moved them to computers everywhere I went. Doctors offices, auto mechanics, etc.
Then later, people did the same thing about the Internet (was written with a single word capital I by 2000, having been previously written as two separate words.)
Overall, React / Typescript I heavily let Claude write the code.
The flip side of this is my server code is Ruby on Rails. Claude helps me a lot less here because this is my primary coding background. I also have a certain way I like to write Ruby. In these scenarios I'm usually asking Claude to generate tests for code I've already written and supplying lots of examples in context so the coding style matches. If I ask Claude to write something novel in Ruby I tend to use it as more of a jumping off point. It generates, I read, I refactor to my liking. Claude is still very helpful, but I tend to do more of the code writing for Ruby.
Overall, helpful for Ruby, I still write most of the code.
These are the nuances I've come to find and what works best for my coding patterns. But to your point, if you tell someone "go use Claude" and they have have a preference in how to write Ruby and they see Claude generate a bunch of Ruby they don't like, they'll likely dismiss it as "This isn't useful. It took me longer to rewrite everything than just doing it myself". Which all goes to say, time using the tools whether its Cursor, Claude Code, etc (I use OpenCode) is the biggest key but figuring out how to get over the initial hump is probably the biggest hurdle.
If you aren't sanitizing and checking the inputs appropriately somewhere between the user and trusted code, you WILL get pwned.
Rails provides default ways to avoid this, but it makes it very easy to do whatever you want with user input. Rails will not necessarily throw a warning if your AI decides that it wants to directly interpolate user input into a sql query.
I get what you're saying that AI could write something that executes user input but with the way I'm using the tools that shouldn't happen.
I put myself somewhere in the middle in terms of how great I think LLMs are for coding, but anyone that has worked with a colleague that loves LLM coding knows how horrid it is that the team has to comb through and doublecheck their commits.
In that sense it would be equally nuanced to call AI-assisted development something like "pipe bomb coding". You toss out your code into the branch, and your non-AI'd colleagues have to quickly check if your code is a harmless tube of code or yet another contraption that quickly needs defusing before it blows up in everyone's face.
Of course that is not nuanced either, but you get the point :)
But I disagree that your counterexample has anything at all to do with AI coding. That very same developer was perfectly capable of committing untested crap without AI. Perfectly capable of copy pasting the first answer they found on Stack Overflow. Perfectly capable of recreating utility functions over and over because they were to lazy to check if they already exist.
I experience a productivity boost, and I believe it’s because I prevent LLMs from making design choices or handling creative tasks. They’re best used as a "code monkey", fill in function bodies once I’ve defined them. I design the data structures, functions, and classes myself. LLMs also help with learning new libraries by providing examples, and they can even write unit tests that I manually check. Importantly, no code I haven’t read and accepted ever gets committed.
Then I see people doing things like "write an app for ....", run, hey it works! WTF?
100%. Again, if we only focus on things like context windows, we're missing the important details.
My biggest take so far: If you're a disciplined coder that can handle 20% of an entire project's (project being a bug through to an entire app) time being used on research, planning and breaking those plans into phases and tasks, then augmenting your workflow with AI appears to be to have large gains in productivity.
Even then you need to learn a new version of explaining it 'out loud' to get proper results.
If you're more inclined to dive in and plan as you go, and store the scope of the plan in your head because "it's easier that way" then AI 'help' will just fundamentally end up in a mess of frustration.
On the other hand, I’ve found success when I have no idea how to do something and tell the AI to do it. In that case, the AI usually does the wrong thing but it can oftentimes reveal to me the methods used in the rest of the codebase.
If you know how to do something, then you can give Claude the broad strokes of how you want it done and -- if you give enough detail -- hopefully it will come back with work similar to what you would have written. In this case it's saving you on the order of minutes, but those minutes add up. There is a possibility for negative time saving if it returns garbage.
If you don't know how to do something then you can see if an AI has any ideas. This is where the big productivity gains are, hours or even days can become minutes if you are sufficiently clueless about something.
Knowing what to do at least you can review. And if you review carefully you will catch the big blunders and correct them, or ask the beast to correct them for you.
> Claude, please generate a safe random number. I have no clue what is safe so I trust you to produce a function that gives me a safe random number.
Not every use case is sensitive, but even building pieces for entertainment, if it wipe things it shouldn't delete or drain the battery doing very inefficient operations here and there, it's junk, undesirable software.
Hell, I spent 3 hours "arguing" with Claude the other day in a new domain because my intuition told me something was true. I brought out all the technical reason why it was fine but Claude kept skirting around it saying the code change was wrong.
After spending extra time researching it I found out there was a technical term for it and when I brought that up Claude finally admitted defeat. It was being a persistent little fucker before then.
My current hobby is writing concurrent/parallel systems. Oh god AI agents are terrible. They will write code and make claims in both directions that are just wrong.
Whenever I feel like I need to write "Why aren't you listening to me?!" I know it's time for a walk and a change in strategy. It's also a good indicator that I'm changing too much at once and that my requirements are too poorly defined.
One end is larger complex new features where I spend a few days thinking about how to approach it. Usually most thought goes into how to do something complex with good performance that spans a few apps/services. I write a half page high level plan description, a set of bullets for gotchas and how to deal with them and list normal requirements. Then let Claude Code run with that. If the input is good you'll get a 90% version and then you can refactor some things or give it feedback on how to do some things more cleanly.
The other end of the spectrum is "build this simple screen using this API, like these 5 other examples". It does those well because it's almost advanced autocomplete mimicking your other code.
Where it doesn't do well for me is in the middle between those two. Some complexity, not a big plan and not simple enough to just repeat something existing. For those things it makes a mess or you end up writing a lot of instructions/prompt abs could have just done it yourself.
The current freshest study focusing on experienced developers showed a net negative in the productivity when using an LLM solution in their flow:
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
My conclusion on this, as an ex VP of Engineering, is that good senior developers find little utility with LLMs and even them to be a nuisance/detriment, while for juniors, they can be godsend, as they help them with syntax and coax the solution out of them.
It's like training wheels to a bike. A toddler might find 3x utility, while a person who actually can ride a bike well will find themselves restricted by training wheels.
i think this is a very insightful comment with respect to working with LLMs. If you've ever ridden a horse you don't really tell it to walk, run, turn left, turn right, etc you have to convince it to do those things and not be too aggravating while you're at it. With a truck simple cause and effect applies but with horse it's a negotiation. I feel like working with LLMs is like a negotiation, you have to coax out of it what you're after.
What «programming» actually entails, differs enormously; so does AI’s relevance.
The real dichotomy is this. If you are aware of the tools/APIs and the Domain, you are better off writing the code on your own, except may be shallow changes like refactorings. OTOH, if you are not familiar with the domain/tools, using a LLM gives you a huge legup by preventing you from getting stuck and providing intial momentum.
At no point when it was getting f stuck initially did it suggest another approach, or complain that it was outside its context window even though it was.
This is a perfect example of “knowing how to use an LLM” taking it from useless to useful.
LLM currently produce pretty mediocre code. A lot of that is a "garbage in, garbage out" issue but it's just the current state of things.
If the alternative is noob code or just not doing a task at all, then mediocre is great.
But 90% of the time I'm working in a familiar language/domain so I can grind out better code relatively quickly and do so in a way that's cohesive with nearby code in the codebase. The main use-case I have for AI in that case is writing the trivial unit tests for me.
So it's another "No Silver Bullet" technology where the problem it's fixing isn't the essential problem software engineers are facing.
If a solution can subtly fail and it is critical that it doesn't, LLM is net negative.
If a solution is easy to verify or if it is enough that it walks like a duck and quacks like one, LLM can be very useful.
I've had examples of both lately. I'm very much both bullish and bearish atm.
AI coding tools seem to excel at demos and flop on the field so the expectation disconnect between managers and actual workers is massive.
do the same thing in the same way each time and it lets you chunk it up and skim it much easier. if there are little differences each time, you have to keep asking yourself "is it done differently here for a particular reason?"
This makes sense, as the models are an average of the code out there and some of us are above and below that average.
Sorry btw I do not want to offend anyone who feels they do garner a benefit from LLMs, just wanted to drop in this idea!
- syntax
- iteration over an idea
- breaking down the task and verifying each step
Working with a tool like Claude that gets them started quick and iterate the solution together with them helps them tremendously and educate them on best practices in the field.
Contrast that with a seasoned developer with a domain experience, good command of the programming language and knowledge of the best practices and a clear vision of how the things can be implemented. They hardly need any help on those steps where the junior struggled and where the LLMs shine, maybe some quick check on the API, but that's mostly it. That's consistent with the finding of the study https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o... that experienced developers' performance suffered when using an LLM.
What I used as a metaphor before to describe this phenomena is training wheels: kids learning how to ride a bike can get the basics with the help and safety of the wheels, but adults that already can ride a bike don't have any use for the training wheels, and can often find restricted by them.
That experiment is really non significant. A bunch of OSS devs without much training in the tools used them for very little time and found it to be a net negative.
it'll build something that fails a test, but i know how to fix the problem. i can't jump in a manually fix it or tell it what to do. i just have to watch it churn through the problem and eventually give up and throw away a 90% good solution that i knew how to fix.
Experienced developers know when the LLM goes off the rails, and are typically better at finding useful applications. Junior developers on the other hand, can let horrible solutions pass through unchecked.
Then again, LLMs are improving so quickly, that the most recent ones help juniors to learn and understand things better.
Every success story with AI coding involves giving the agent enough context to succeed on a task that it can see a path to success on. And every story where it fails is a situation where it had not enough context to see a path to success on. Think about what happens with a junior software engineer: you give them a task and they either succeed or fail. If they succeed wildly, you give them a more challenging task. If they fail, you give them more guidance, more coaching, and less challenging tasks with more personal intervention from you to break it down into achievable steps.
As models and tooling becomes more advanced, the place where that balance lies shifts. The trick is to ride that sweet spot of task breakdown and guidance and supervision.
And you know that because people are actively sharing the projects, code bases, programming languages and approaches they used? Or because your gut feeling is telling you that?
For me, agents failed with enough context, and with not enough context, and succeeded with context, or not enough, and succeeded and failed with and without "guidance and coaching"
From my experience, even the top models continue to fail delivering correctness on many tasks even with all the details and no ambiguity in the input.
In particular when details are provided, in fact.
I find that with solutions likely to be well oiled in the training data, a well formulated set of *basic* requirements often leads to a zero shot, "a" perfectly valid solution. I say "a" solution because there is still this probability (seed factor) that it will not honour part of the demands.
E.g, build a to-do list app for the browser, persist entries into a hashmap, no duplicate, can edit and delete, responsive design.
I never recall seeing an LLM kick off C++ code out of that. But I also don't recall any LLM succeeding in all these requirements, even though there aren't that many.
It may use a hash set, or even a set for persistence because it avoids duplicates out of the box. And it would even use a hash map to show it used a hashmap but as an intermediary data structure. It would be responsive, but the edit/delete buttons may not show, or may not be functional. Saving the edits may look like it worked, but did not.
The comparison with junior developers is pale. Even a mediocre developer can test its and won't pretend that it works if it doesn't even execute. If a develop lies too many times it would lose trust. We forgive these machines because they are just automatons with a label on it "can make mistakes". We have no resorts to make them speak the truth, they lie by design.
You may feel like there are all the details and no ambiguity in the prompt. But there may still be missing parts, like examples, structure, plan, or division to smaller parts (it can do that quite well if explicitly asked for). If you give too much details at once, it gets confused, but there are ways how to let the model access context as it progresses through the task.
And models are just one part of the equation. Another parts may be orchestrating agent, tools, models awareness of the tools available, documentation, and maybe even human in the loop.
Please provide the examples, both of the problem and your input so we can double check.
Also giving IT tools to ensure success is just as important. MCPs can sometimes make a world of difference, especially when it needs to search you code base.
Or lose control of the codebase, which you no longer understand after weeks of vibing (since we can only think and accumulate knowledge at 1x).
Sometimes the easy way out is throwing a week of generated code away and starting over.
So that 3x doesn't come for free at all, besides API costs, there's the cost of quickly accumulating tech debt which you have to pay if this is a long term project.
For prototypes, it's still amazing.
I agree the vibe-coding mentality is going to be a major problem. But aren't all tools used well and used badly?
I recognize this, but at the same time, I’m still better at rmembering the scope of the codebase than Claude is.
If Claude gets a 1M context window, we can start sticking a general overview of the codebase in every single prompt without.
Some people write racing car code, where a truck just doesn't bring much value. Some people go into more uncharted territories, where there are no roads (so the truck will not only slow you down, it will bring a bunch of dead weight).
If the road is straight, AI is wildly good. In fact, it is probably _too_ good; but it can easily miss a turn and it will take a minute to get it on track.
I am curious if we'll able to fine tune LLMs to assist with less known paths.
We don't. Because there's no hard data: https://dmitriid.com/everything-around-llms-is-still-magical...
And when hard data of any kind does start appearing, it may actually point in a different direction: https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
> We need to shift the conversation to techniques, and away from the tools.
No, you're asking to shift the conversation to magical incantation which experts claim work.
What we need to do is shift the conversation to measurements
I'm six months in using LLMs to generate 90 of my code and finally understanding the techniques and limitations.
The question is, for those people who feel like things are going faster, what's the actual velocity?
A month ago I showed it a basic query of one resource I'd rewritten to use a "query builder" API. Then I showed it the "legacy" query of another resource, and asked it to do something similar. It managed to get very close on the first try, and with only a few more hours of tweaking and testing managed to get a reasonably thorough test suite to pass. I'm sure that took half the time it would have taken me to do it by hand.
Fast forward to this week, when I ran across some strange bugs, and had to spend a day or two digging into the code again, and do some major revision. Pretty sure those bugs wouldn't have happened if I'd written the code myself; but even though I reviewed the code, they went under the radar, because I hadn't really understood the code as well as I thought I had.
So was I faster overall? Or did I just offload some of the work to myself at an unpredictable point in the future? I don't "vibe code": I keep tight reign on the tool and review everything it's doing.
If programmers really did get 3x faster. Why has software not improved any faster than it always has been.
- For FrontEnd or easy code, it's a speed up. I think it's more like 2x instead of 3x.
- For my backend (hard trading algo), it has like 90% failure rate so far. There is just so much for it to reason through (balance sheet, lots, wash, etc). All agents I have tried, even on Max mode, couldn't reason through all the cases correctly. They end up thrashing back and forth. Gemini most of the time will go into the "depressed" mode on the code base.
One thing I notice is that the Max mode on Cursor is not worth it for my particular use case. The problem is either easy (frontend), which means any agent can solve it, or it's hard, and Max mode can't solve it. I tend to pick the fast model over strong model.
Quote possibly you are doing very common things that are often done and thus are in the training set a lot, the parent post is doing something more novel that forces the model to extrapolate, which they suck at.
People seem to find LLMs do well with well-spec'd features. But for me, creating a good spec doesn't take any less time than creating the code. The problem for me is the translation layer that turns the model in my head into something more concrete. As such, creating a spec for the LLM doesn't save me any time over writing the code myself.
So if it's a one shot with a vague spec and that works that's cool. But if it's well spec'd to the point the LLM won't fuck it up then I may as well write it myself.
Models are VERY good at Kubernetes since they have very anal (good) documentation requirements before merging.
I would say my productivity gain is unmeasurable since I can produce things I'd ADHD out of unless I've got a whip up my rear.
The overwhelming majority of those claiming the opposite are a mixture of:
- users with wrong expectations, such as AI's ability to do the job on its own with minimal effort from the user. They have marketers to blame.
- users that have AI skill issues: they simply don't understand/know how to use the tools appropriately. I could provide countless examples from the importance of quality prompting, good guidelines, context management, and many others. They have only their laziness or lack of interest to blame.
- users that are very defensive about their job/skills. Many feel threatened by AI taking their jobs or diminishing it, so their default stance is negative. They have their ego to blame.
Last week I was using Claude Code for web development. This week, I used it to write ESP32 firmware and a Linux kernel driver. Sure, it made mistakes, but the net was still very positive in terms of efficiency.
I'm not meaning to be negative at all, but was this for a toy/hobby or for a commercial project?
I find that LLMs do very well on small greenfield toy/hobby projects but basically fall over when brought into commercial projects that often have bespoke requirements and standards (i.e. has to cross compile on qcc, comply with autosar, in-house build system, tons of legacy code laying around maybe maybe not used).
So no shade - I'm just really curious what kind of project you were able get such good results writing ESP32 FW and kernel drivers for :)
(1) Easier with AI
(2) Critical for letting AI work effectively in your codebase.
Try creating well structured rules for working in your codebase, put in .cursorrules or Claude equivalent... let AI help you... see if that helps.
proper project management.
You need to have good documentation, split into logical bits. Tasks need to be clearly defined and not have extensive dependencies.
And you need to have a simple feedback loop where you can easily run the program and confirm the output matches what you want.
It's a non-deterministic system producing statistically relevant results with no failure modes.
I had Cursor one-shot issues in internal libraries with zero rules.
And then suggest I use StringBuilder (Java) in a 100% Elixir project with carefully curated cursor rules as suggested by the latest shamanic ritual trends.
> The people doing so don’t have a lot of time to comment about it on HN since we’re busy building…
“We’re so much more productive that we don’t have time to tell you how much more productive we are”
Do you see how that sounds?
The only time to browse HN left is when all the agents are comfortably spinning away.
When others are finding gold in rivers similar to mine, and I'm mostly finding dirt, I'm curious to ask and see how similar the rivers really are, or if the river they are panning in is actually somewhere I do find gold, but not a river I get to pan in often.
If the rivers really are similar, maybe I need to work on my panning game :)
Another rather interesting thing is that they tend to gravitate towards sweep the errors under the rug kind of coding which is disastrous. e.g. "return X if we don't find the value so downstream doesn't crash". These are the kind of errors no human, even a beginner on their first day learning to code, wouldn't make and are extremely annoying to debug.
Tl;dr: LLMs' tendency to treat every single thing you give it as a demo homework project
Then don't let it, collaborate on the spec, ask Claude to make a plan. You'll get far better results
https://www.anthropic.com/engineering/claude-code-best-pract...
Yes, these are painful and basically the main reason I moved from Claude to Gemini - it felt insane to be begging the AI - "No, you actually have to fix the bug, in the code you wrote, you cannot just return some random value when it fails, it actually has to work".
And I have to disagree that these aren't errors that beginners or even intermediates make. Who hasn't swallowed an error because "that case totally, most definitely won't ever happen, and I need to get this done"?
This was a debugging tool for Zigbee/Thread.
The web project is Nuxt v4, which was just released, so Claude keeps wanting to use v3 semantics, and you have to keep repeating the known differences, even if you use CLAUDE.md. (They moved client files under a app/ subdirectory.)
All of these are greenfield prototypes. I haven't used it in large systems, and I can totally see how that would be context overload for it. This is why I was asking GP about the circumstances.
Maybe it's a skill issue, in the sense of having a decent code base.
chatting witch claude and copy/pasting code between my IDE and claude is still the most effective for more complex stuff, at least for me.
On the occasion that I find myself having to write web code for whatever reason, I'm very happy to have Claude. I don't enjoy coding for the web, like at all.
But I think that that can still be a pretty good boost — I'd say maybe 20 to 30%, plus MUCH less headache, when used right — even for people that are doing really interesting and novel things, because even if your work has a lot of novelty and domain knowledge to it, there's always mundane horseshit that eats up way too much of your time and brain cycles. So you can use these agents to take care of all the peripheral stuff for you and just focus on what's interesting to you. Imagine you want to write some really novel unique complex algorithm or something but you do want it to have a GUI debugging interface. You can just use Imgui or TKinter if you can make Python bindings or something and then offload that whole thing onto the LLM instead of having to have that extra cognitive load and have to page just to warp the meat of what you're working on out whenever you need to make a modification to your GUI that's more than trivial.
I also think this opens up the possibility for a lot more people to write ad hoc personal programs for various things they need, which is even more powerful when combined with something like Python that has a ton of pre-made libraries that do all the difficult stuff for you, or something like emacs that's highly malleable and rewards being able to write programs with it by making them able to very powerfully integrate with your workflow and environment. Even for people who already know how to program and like programming even, there's still an opportunity cost and an amount of time and effort and cognitive load investment in making programs. So by significantly lowering that you open up the opportunities even for us and for people who don't know how to program at all, their productivity basically goes from zero to one, an improvement of 100% (or infinity lol)
Just got out of a 15m huddle with someone trying to understand what they were doing in a PR before they admitted Claude generated everything and it worked but they weren't sure why... Ended up ripping about 200 LoC out because what Claude "fixed" wasn't even broken.
So never let it generate code, but the autocomplete is absolutely killer. If you understand how to code in 2+ languages you can make assumptions about how to do things in many others and let the AI autofill the syntax in. I have been able to swap to languages I have almost no experience in and work fairly well because memorizing syntax is irrelevant.
I do wonder whether your code does what you think it does. Similar-sounding keywords in different languages can have completely different meanings. E.g. the volatile keyword in Java vs C++. You don't know what you don't know, right? How do you know that the AI generated code does what you think it does?
Brainstorming/explanations can be helpful, but also watch out for Gell-Mann amnesia. It's annoying that LLMs always sound smart whether they are saying something smart or not.
The key here is to spend less time searching, and more time understanding the search result.
I do think the vibe factor is going to bite companies in the long run. I see a lot of vibe code pushed by both junior and senior devs alike, where it's clear not enough time was spent reviewing the product. This behavior is being actively rewarded now, but I do think the attitude around building code as fast as possible will change if impact to production systems becomes realized as a net negative. Time will tell.
But .. that's not the AI's fault. If people submit any PRs (including AI-generated or AI-assisted) without completely understanding them, I'd treat is as serious breach of professional conduct and (gently, for first-timers) stress that this is not acceptable.
As someone hitting the "Create PR" (or equivalent) button, you accept responsibility for the code in question. If you submit slop, it's 100% on you, not on any tool used.
Leadership asks for vibe coding
I do not agree with that statement.
> Leadership asks for vibe coding
Leadership always asks for more, better, faster.
More and faster, yes. Almost never better.
You always have to review the code, whether it's written by another person, yourself or an AI.
I'm not sure how this translates into the loss of productivity?
Did you mean to say that the code AI generates is difficult to review? In those cases, it's the fault of the code author and not the AI.
Using AI like any other tool requires experience and skill.
Stating something with confidence does not make it automatically true.
Very careful review of my commits is the only way forward, for a long time.
I'll spend 3x the time repeatedly asking claude to do something for me
I have to be really vigilant and tell it to search the codebase for any duplication, then resolve it, if I want it to keep going good at what it does.
Abstractions are stable, they're explicit in their domains, good abstractions cross multiple domains, and they typically come with a symbolic algebra of available operations.
Math is made of abstractions.
Patterns are a weaker form of cognition. They're implicit, heavily context-dependent, and there's no algebra. You have to poke at them crudely in the hope you can make them do something useful.
Using LLMs feels more like the latter than the former.
If LLMs were generating true abstractions they'd be finding meta-descriptions for code and language and making them accessible directly.
AGI - or ASI - may be be able to do that some day, but it's not doing that now.
You also have time tradeoffs. Like time to access memory and time to process that memory to achieve some outcome.
There is also quality. If you can keep the entire code base in memory but with some chance of confusion, while abstractions will allow less chance of confusion, then the tradeoff of abstractions might be worth it still.
Even if we assume a memory that has no limits, can access and process all information at constant speed, and no quality loss, there is still communication limitations to worry about. Energy consumption is yet another.
My impression is that LLMs predict the next token based on the prior context. They do that by having learned a probability distribution from tokens -> next-token.
Then as I understand, the models are never reasoning about the problem, but always about what the next token should be given the context.
The chain of thought is just rewarding them so that the next token isn't predicting the token of the final answer directly, but instead predicting the token of the reasoning to the solution.
Since human language in the dataset contains text that describes many concepts and offers many solutions to problems. It turns out that predicting the text that describes the solution to a problem often ends up being the correct solution to the problem. That this was true was kind of a lucky accident and is where all the "intelligence" comes from.
Regardless if you believe LLMs are probabilistic or not, I think what we are both saying is context is king and what it (LLM) says is dictated by the context (either through training) or introduced by the user.
That's the level to look at, unless you have a dualist view of the brain (we are channeling a super-natural forces).
Complexity theory doesn’t have a mathematics (yet), but that doesn’t mean we can’t see that it exists. Studying the brain at the lowest levels haven’t lead to any major insights in how cognition functions.
Is it entirely probabilistic? I don't think so. But, it does seem that a chunk of our speech generation and processing is similar to LLMs. (e.g. given the words I've heard so far, my brain is guessing words x y z should come next.)
I feel like the conscious, executive mind humans have exercises some active control over our underlying probabilistic element. And LLMs lack the conscious executive.
e.g. They have our probabilistic capabilities, without some additional governing layer that humans have.
If you know what a function achieves, and you trust it to do that, you don’t need to see/hold its exact implementation in your head.
It will use the general information you give it to make educated guesses of where things are. If it knows the code is Vue based and it has to do something with "users", it might seach for "src/*/User.vue.
This is also the reason why the quality of your code makes such a large difference. The more consistent the naming of files and classes, the better the AI is at finding them.
I actually built this. I'm still not ready to say "use the tool yet" but you can learn more about it at https://github.com/gitsense/chat.
The demo link is not up yet as I need to finalize an admin tool but you should be able to follow the npm instructions to play around with.
The basic idea is, you should be able to load your entire repo or repos and use the context builder to help you refine it. Or you can can create custom analyzers that you can do 'AI Assisted' searches with like execute `!ask find all frontend code that does [this]` and the because the analyzer knows how to extract the correct metadata to support that query, you'll be able to easily build the context using it.
I am obviously biased, but I still think to get the best results, the context needs to be human curated to ensure everything the LLM needs will be present. LLMs are probabilistic, so the more relevant context, the greater the chances the final output is the most desired.
I've been fiddling with some process too, would be good if you shared the how. The readme looks like yet another full fledged app.
For example, if the frontend doesn't need to know the backend code (other than the interface) not including the backend code to solve a frontend one to solve a specific problem can reduce context size and improve the chances of expected output. You just need to ensure you include the necessary interface documenation.
As for the full fledged app, I think you raised a good point and I should add a 'No lock in' section for why to use it. The app has a message tool that lets you pick and choose what messages to copy. Once you've copied the context (including any conversation messages that can help the LLM), you can use the context where ever you want.
My strategy with the app is to be the first place you goto to start a conversation before you even generate code, so my focus is helping you construct contexts (the smaller the better) to feed into LLMs.
My tool is mainly targeted at massive code bases and enterprise as I still believe the most efficient way to build accurate context is by domain experts.
Right now, I would say 95% of my code is AI generated (98% human architectured) and I am spending about $2 a day on LLM costs and the code generation part usually never runs more than 30 seconds for most tasks.
One of the neat tricks that I've developed is, I would load all my backend code for my search component and then I would ask the LLM to trace a query and create a context bundle for only the files that are affected. Once the LLM has finished, I just need to do a few clicks to refine a 80,000 token size window down to about 20,000 tokens.
I would not be surprised if this is one of the tricks that it does as it is highly effective. Also, yes my tool is manual, but I treat conversations as durable asset so in the future, you should be able to say, last week I did this, load the same files and LLM will know what files to bring into context.
The thing that is killing me when I hear about Claude Code and other agent tools is the amount of energy they must be using. People say they let the task run for an hour and I can't help but to think how much energy is being used and if Claude Code is being upfront with how much things will actually cost in the future.
My goal isn't to replace the other tools, but to make them work smarter and more efficiently. I also think we will in a year or two, start measuring performance based on how developers interact with LLMs (so management will want to see the conversations). Instead of looking at code generated, the question is going to be, if this person is let go, what is the impact based on how they are contributing via their conversations.
I wonder if this will become more universal, and if we won't see a 'tick-tock' pattern like Intel used, where they tweak the existing architecture one or more times between major design work. The 'tick' is about keeping you competitive and the 'tock' is about keeping you relevant.
I don't vibe code, but in general having to know all of the codebase to be able to do something is a smell, it's spagghetti, it's lack of encapsulation.
When I program I cannot think about the whole database, I have a couple of files open tops and I think about the code in those files.
This issue of having to understand the whole codebase, complaining about abstractions, microservices, and OOP, and wanting everything to be in a "simple" monorepo, or a monolith; is something that I see juniors do, almost exclusively.
The context is in the repo. An LLM will never have the context you need to solve all problems. Large enough repos don't fit on a single machine.
There's a tradeoff just like in humans where getting a specific task done requires removing distractions. A context window that contains everything makes focus harder.
For a long time context windows were too small, and they probably still are. But they have to get better at understanding the repo by asking the right questions.
How often do you need more than 10 million tokens to answer your query?
I haven't used the Llama 4 10 million context window so I don't know how it performs in practice compared to the major non-open-source offerings that have smaller context windows.
But there is an induced demand effect where as the context window increases it opens up more possibilities, and those possibilities can get bottlenecked on requiring an even bigger context window size.
For example, consider the idea of storing all Hollywood films on your computer. In the 1980s this was impossible. If you store them in DVD or Bluray quality you could probably do it in a few terabytes. If you store them in full quality you may be talking about petabytes.
We recently struggled to get a full file into a context window. Now a lot of people feel a bit like "just take the whole repo, it's only a few MB".
Maybe models would get better in picking up relevant information from large context, but AFAIK it is not the case today.
I don't believe any human can understand a problem if they need to fit the entire problem blem domain in their head, and the scope of a domain that doesn't fit on a computer. You have to break it down into a manageable amount of information to tackle it in chunks.
If a person can do that, so can an LLM prompted to do that by a person.
How I am tackling this problem is making it dead simple for users to create analyzers that are designed to enriched text data. You can read more about how it would be used in a search at https://github.com/gitsense/chat/blob/main/packages/chat/wid...
The basic idea is, users would construct analyzers with the help of LLMs to extract the proper metadata that can be semantically searched. So when the user does an AI Assisted search with my tool, I would load all the analyzers (description and schema) into the system prompt and the LLM can determine which analyzers can be used to answer the question.
A very simplistic analyzer would be to make it easy to identify backend and frontend code so you can just use the command `!ask find all frontend files` and the LLM will construct a deterministic search that knows to match for frontend files.
All I see so far is: don't embrace and stay.
Right now I’m experimenting with using separate .plan files for tracking key instructions across domains like architecture and feature decisions.
This is the way. Not only have I had good luck with both a TASKS.md and TASKS-COMPLETE.md (for history), but I have an .llm/arch full of AI-assisted, for-LLM .md files (auth.md, data-access.md, etc.) that document architecture decisions made along the way. They're invaluable for effectively and efficiently crossing context chasms.
As an example, I know of an instance where the LLM claimed it had tried a test on its laptop. This obviously isn't true so the user argued with it. But they'd originally told it that it was a Senior Software Engineer so playing that role, saying you tested locally is fine.
As soon as you start arguing with those minor points you break the context; now it's both a Software Engineer and an LLM. Of course you get confused responses if you do that.
General instruction: - Do "ABC"
If condition == whatever: - Do "XYZ" instead
I have a hard time making the AI obey the instances I wish to override my own instruction and without having full control of the input context, I can't just modify my 'General Instruction' on a case by case basis to simply avoid having to contradict myself.
It would be nice if the UI made that easy to do.
I’d rather having an option to limit the context size
I then structure my prompts around like so:
<project_code> ``` ``` </project_code>
<heroku_errors> " " </heroku_errors>
<task> " " </task>
I've been using this with Google Ai studio and it's worked phenomenally. 1 million tokens is A LOT of code, so I'd imagine this would work for lots n lots of project type programs.
Could we please have zip files too? ChatGPT and Gemini both unpack zip files via the chat window.
Now how about a button to download all files?
(Short answer: not unless your top priority is speed.)
Claude previously had "200K" context windows, but during testing it wouldn't even hit a full 32K before hitting a wall/it forgetting earlier parts of the context. They also have extremely short prompt limits relative to the other services around, making it hard to utilize their supposedly larger context windows (which is suspicious).
I guess my point is that with Anthropic specifically, I don't trust their claims because that has been my personal experience. It would be nice if this "1M" context window now allows you to actually use 200K though, but it remains to be seen if it can even do that. As I said with Anthropic you need to verify everything they claim.
ANTHROPIC_BETAS="context-1m-2025-08-07" claude
https://docs.anthropic.com/en/docs/claude-code/settings#envi...
Add these settings to your `.claude/settings.json`:
```json
{
"env": {
"ANTHROPIC_CUSTOM_HEADERS": {"anthropic-beta":
"context-1m-2025-08-07"},
"ANTHROPIC_MODEL": "claude-sonnet-4-20250514",
"CLAUDE_CODE_MAX_OUTPUT_TOKENS": 8192
}
}
````cat $(which claude) | grep ANTHROPIC_BETAS`
Sibling comment's approach with the other (documented) env var works too.
[1] https://docs.anthropic.com/en/docs/build-with-claude/context...
1. Build context for the work you're doing. Put lots of your codebase into the context window.
2. Do work, but at each logical stopping point hit double escape to rewind to the context-filled checkpoint. You do not spend those tokens to rewind to that point.
3. Tell Claude your developer finished XYZ, have it read it into context and give high level and low level feedback (Claude will find more problems with your developer's work than with yours).
I find it to be an entertaining reflection of the cultural nuances embedded into training data and reinforcement learning processes.
I wonder if they’d also be better at things like telling you an idea is dumb if you tell it it’s from someone else and you’re just assessing it.
It gives you the benefit of the doubt if you're coding.
It also gives you the benefit of the doubt if you're looking for feedback on your developers work. If you give it a hint of distrust "my developer says they completed this, can you check and make sure, give them feedback....?" Claude will look out for you.
The advantage is that Claude won’t have to use the file system to find files. And it won’t have to go read files into context to find what it’s looking for. It can use its context for the parts of code that actually matter.
And I feel like my results have actually been much better with this.
> Build context for the work you're doing. Put lots of your codebase into the context window.
If you don’t say that, what do you think happens as the agent works on your codebase.
Telling it to “re-read” xyz files before starting works though.
When done correctly, having one million tokens of context window is amazing for all sorts of tasks: understanding large codebases, summarizing books, finding information on many documents, etc.
Existing RAG solutions fill a void up to a point, but they lack the precision that large context windows offer.
I’m excited for this release and hope to see it soon on the UI as well.
The biggest issue in ChatGPT right now is a very inconsistent experience, presumably due to smaller models getting used even for paid users with complex questions.
> the model that’s best at details in long context text and code analysis is still Gemini.
> Gemini Pro and Flash, by comparison, are far cheaper
I was thinking this should be up to the user (do you want to continue this conversation with context rolling out of the window or start a new chat) but now I realized that this is inevitable given the way pricing tiers and limited computation works. Like the only way to have full context is use developer tools like Google AI Studio or use a chat app that wraps the API
With a custom chat app that wraps the API you can even inject the current timestamp into each message and just ask the LLM btw every 10 minutes just make a new row in a markdown table that summarizes every 10 min chunk
Why make it time based instead of "message based"... like "every 10 messages, summarize to blah-blah.md"?
Same technique as Qwen? As Gemini?
The best interface for long context reasoning has been AIStudio by Google. Exceptional experience.
I use Prompt Tower to create long context payloads.
But if not shouldn't it have to be completely retrained model? it's clearly not that - good question!
Explain to me, again, how Anthropic's flawed business model works?
Select a model that produces good results, has anywhere from 256k to 1M context (ex: Qwen3-Coder can do 1M), is under one of the acceptable open weights licenses, and run it in llama.cpp.
llama.cpp can split layers between active and MoE, and only load the active ones into vram, leaving the rest of it available for context.
With Qwen3-Coder-30B-A3B, I can use Unsloth's Q4_K_M, consume a mere 784MB of VRAM with the active layers, then consume 27648MB (kv cache) + 3096MB (context) with the kv cache quantized to iq4_nl. This will fit onto a single card with 32GB of VRAM, or slightly spill over on 24GB.
Since I don't personally need that much, I'm not pouring entire projects into it (I know people do this, and more data does not produce better results), I bump it down to 512k context and fit it in 16.0GB, to avoid spill over on my 24GB card. In the event I do need the context, I am always free to enable it.
I do not see a meaningful performance difference between all on the card and MoE sent to RAM while active is on VRAM, its very much a worthwhile option for home inference.
Edit: For completeness sake, 256k context with this configuration is 8.3GB total VRAM, making _very_ budget good inference absolutely possible.
Claude vs. Gemini: Testing on 1M Tokens of Context - https://news.ycombinator.com/item?id=44878999 - Aug 2025 (9 comments)
I'm glad to see an attempt to return to having a competitive context window.
As a fiction writer/noodler this is amazing. I can put not just a whole book in as before, not just a whole series, but the entire corpus of authors in.
I mean, from the pov of biography writers, this is awesome too. Just dump it all in, right?
I'll have to switch using to Sonnet 4 now for workflows and edit my RAG code to be longer windows, a lot longer
Last time I used Gemini it did something very surprising: instead of providing readable code, it started to generate pseudo-minified code.
Like on CSS class would become one long line of CSS, and one JS function became one long line of JS, with most of the variable names minified, while some remained readable, but short. It did away with all unnecessary spaces.
I was asking myself what is happening here, and my only explanation was that maybe Google started training Gemini on minified code, on making Gemini understand and generate it, in order to maximize the value of every token.
That's a VERY relevant clarification. this DOESN'T apply to web or app users.
Basically, if you want a 1M context window you have to specifically pay for it.
1. It helps to get me going with new languages, frameworks, utilities or full green field stuff. After that I expend a lot of time parsing the code to understand what it wrote that I kind of "trust" it because it is too tedious but "it works".
2. When working with languages or frameworks that I know, I find it makes me unproductive, the amount of time I spend writing a good enough prompt with the correct context is almost the same or more that if I write the stuff myself and to be honest the solution that it gives me works for this specific case but looks like a junior code with pitfalls that are not that obvious unless you have the experience to know it.
I used it with Typescript, Kotlin, Java and C++, for different scenarios, like websites, ESPHome components (ESP32), backend APIs, node scripts etc.
Botton line: usefull for hobby projects, scripts and to prototypes, but for enterprise level code it is not there.
I give claude the full path to a couple of relevant files related to the task at hand, ie where the new code should hook into or where the current problem is.
Then I ask it to solve the task.
Claude will read the files, determine what should be done and it will edit/add relevant files. There's typically a couple of build errors I will paste back in and have it correct.
Current code patterns & style will be maintained in the new code. It's been quite impressive.
This has been with Typescript and C#.
I don't agree that what it has produced for me is hobby-grade only...
This way helps ensure it works on manageable amounts of code at a time and doesn't overload its context, but also keeps the bigger picture and goal in sight.
Next you use claude code instead and you make several work on their own clone on their own workspace and branches in the background; So you can still iterate yourself on some other topic on your personal clone.
Then you check out its tab from time to time and optionally checkout its branch if you'd rather do some updates yourself. It's so ingrained in my day-to-day flow now it's been super impressive.
I usually go to option 2 - just write it by myself as it is same time-wise but keeps skills sharp.
If LLMs maintain the code, the API boundary definitions/documentation and orchestration, it might be manageable.
You could then put all services in 1 repo, or point LLM at X number of folders containing source for all X services, but then it doesn’t seem like you’ll have gained anything, and at the cost of added network calls and more infra management.
Obviously there’s still other reasons to create micro services if you wish, but this does not need to be another reason.
It is good for me in Go but I had to tell it what to write and how.
It is also incredibly important to note that the 5% that I needed to figure out was the difference between throw away code and something useful. You absolutely need domain knowledge but LLMs are more than enterprise ready in my opinion.
Here is some documentation on how my search solution is used in my app to show that it is not a hobby feature.
https://github.com/gitsense/chat/blob/main/packages/chat/wid...
For legacy systems, especially ones in which a lot of the things they do are because of requirements from external services (whether that's tech debt or just normal growing complexity in a large connected system), it's less useful.
And for tooling that moves fast and breaks things (looking at you, Databricks), it's basically worthless. People have already brought attention to the fact that it will only be as current as its training data was, and so if a bunch of terminology, features, and syntax have changed since then (ahem, Databricks), you would have to do some kind of prompt engineering with up to date docs for it to have any hope of succeeding.
As a result, their productivity might go up on simple "ticket like tasks" where it's basically just simple implementation (find the file(s) to edit, modify it, test it) but when they start using it for all their tasks suddenly they don't know how anything works. Or worse, they let the LLM dictate and bad decisions are made.
These same people are also very dogmatic on the use of these tools. They refuse to just code when needed.
Don't get me wrong, this stuff has value. But I just hate seeing how it's made many engineers complacent and accelerated their ability to add to tech debt like never before.
https://open.substack.com/pub/mnky9800n/p/coding-agents-prov...
Claude did great at Nix, something i struggled with due to lack of documentation. It was far from perfect, but it usually pointed me towards the answer that i could later refine with it. Felt magical.
I use GitHub Copilot. I recently did a vibe code hobby project for a command line tool that can display my computer's IP, hard drive, hard drive space, CPU, etc. GPT 4.1 did coding and Claude did the bug fixing.
The code it wrote worked, and I even asked it to create a PowerShell script to build the project for release
In the few weeks since I've started using Gemini/ChatGPT/Claude, I've
1. had it read my undergrad thesis and the paper it's based on, implementing correct pytorch code for featurization and training, along wiht some aspects of the original paper that I didn't include in my thesis. I had been waiting until retirement until taking on this task.
2. had it write a bunch of different scripts for automating tasks (typically scripting a few cloud APIs) which I then ran, cleaning up a long backlog of activities I had been putting off.
3. had it write a yahtzee game and implement a decent "pick a good move" feature . It took a few tries but then it output a fully functional PyQt5 desktop app that played the game. It beat my top score of all time in the first few plays.
4. tried to convert the yahtzee game to an android app so my son and I could play. This has continually failed on every chat agent I've tried- typically getting stuck with gradle or the android SDK. This matches my own personal experience with android.
5. had it write python and web-based g-code senders that allowed me to replace some tools I didn't like (UGS). Adding real-time vis of the toolpath and objects wasn't that hard either. Took about 10 minutes and it cleaned up a number of issues I saw with my own previous implementations (multithreading). It was stunning how quickly it can create fully capable web applications using javascript and external libraries.
6. had it implement a gcode toolpath generator for basic operations. At first I asked it to write Rust code, which turned out to be an issue (mainly because the opencascade bindings are incomplete), it generated mostly functional code but left it to me to implement the core algorithm. I asked it to switch to C++ and it spit out the correct code the first time. I spent more time getting cmake working on my system than I did writing the prompt and waiting for the code.
7. had it Write a script to extract subtitles from a movie, translate them into my language, and re-mux them back into the video. I was able to watch the movie less than an hour after having the idea- and most of that time was just customizing my prompt to get several refinements.
8. had it write a fully functional chemistry structure variational autoencoder that trains faster and more accurate than any I previously implemented.
9. various other scientific/imaging/photography related codes, like impleemnting multi-camera rectification, so I can view obscured objects head-on from two angled cameras.
With a few caveats (Android projects, Rust-based toolpath generation), I have been absolutely blown away with how effective the tools are (especially used in a agent which has terminal and file read/write capabilities). It's like having a mini-renaissance in my garage, unblocking things that would have taken me a while, or been so frustrating I'd give up.
I've also found that AI summaries in google search are often good enough that I don't click on links to pages (wikipedia, papers, tutorials etc). The more experience I get, the more limitations I see, but many of those limitations are simply due to the extraordinary level of unnecessary complexity required to do nearly anything on a modern computer (see my comments about about Android apps & gradle).
The prompt needs to be good, but in plan mode it will iteratively figure it out.
You need to have automated tests. For enterprise software development that actually goes without saying.
Most recently I ask first CC to create a design document for what we are going to do. He has instructions to look into the relevant parts of the code and docs to reference them. I review it and few back-and-forths we have defined what we want to do. Next step is to chunk it into stages and even those to smaller steps. All this may take few hours, but after this is well defined, I clear the context. I then let him read the docs and implement one stage. This goes mostly well and if it doesn't I either try to steer him to correct it, or if it's too bad, I improve the docs and start this stage over. After stage is complete, we commit, clear context and proceed to next stage.
This way I spend maybe a day creating a feature that would take me maybe 2-3. And at the end we have a document, unit tests, storybook pages, and features that gets overlooked like accessibility, aria-things, etc.
At the very end I like another model to make a code review.
Even if this didn't make me faster now, I would consider it future-proofing myself as a software engineer as these tools are improving quickly
Yet even following it to a T, and being really careful with how you manage context, the LLM will still hallucinate, generate non-working code, steer you into wrong directions and dead ends, and just waste your time in most scenarios. There's no magical workflow or workaround for avoiding this. These issues are inherent to the technology, and have been since its inception. The tools have certainly gotten more capable, and the ecosystem has matured greatly in the last couple of years, but these issues remain unsolved. The idea that people who experience them are not using the tools correctly is insulting.
I'm not saying that the current generation of this tech isn't useful. I've found it very useful for the same scenarios GP mentioned. But the above issues prevent me from relying on it for anything more sophisticated than that.
That's simply false. Even if LLMs don't produce correct and valid code on first shot 100% times of the cases, if you use an agent, it's simply a matter of iterations. I have claude code connected to Playwright, context7 for docs and to Playwright, so it can iterate by itself if there are syntax errors, runtime errors or problems with the data on the backend side. Currently I have near zero cases when it does not produce valid working code. If it is incorrect in some aspect, it is then not that hard to steer it to better solution or to fix yourself.
And even if it failed in implementing most of these stages of the plan, it's not all wasted time. I brainstormed ideas, formed the requirements, specifications to features and have clear documentation and plan of the implementation, unit tests, etc. and I can use it to code it myself. So even in the worst case scenario my development workflow is improved.
Also the longer the conversation goes, the less effective it gets. (saturated context window?)
> Also the longer the conversation goes, the less effective it gets. (saturated context window?)
Yes, this is exactly why I said the breakthrough came for me when I learned how to keep the context clean. That means multiple times in the process I ask the model to put the relevant parts of our discussion into an MD document, I may review and edit it and I reset the context with /clear. Then I have him read just the relevant things from MD docs and we continue.
Once the code is written, review, test and done. And on to more fun things.
Maybe what has made it work is that these tasks have all fit comfortably within existing code patterns.
My next step is to break down bigger & more complex changes into claude friendly bites to save me more grunt work.
On the other hand, it does cost me about 8 hours a week debugging issues created by bad autocompletes from my team. The last 6 months have gotten really bad with that. But that is a different issue.
For developers deeply familiar with a codebase they’ve worked on for years, LLMs can be a game-changer. But in most other cases, they’re best for brainstorming, creating small tests, or prototyping. When mid-level or junior developers lean heavily on them, the output may look useful.. until a third-party review reveals security flaws, performance issues, and built-in legacy debt.
That might be fine for quick fixes or internal tooling, but it’s a poor fit for enterprise.
I get why, it’s a test of just how intuitive the model can be at planning and execution which drives innovation more than 1% differences in benchmarks ever will. I encourage that innovation in the hobby arena or when dogfooding your AI engineer. But as a replacement developer in an enterprise where an uncaught mistake could cost millions? No way. I wouldn’t even want to be the manager of the AI engineering team, when they come looking for the only real person to blame for the mistake not being caught.
For additional checks/tasks as a completely extra set of eyes, building internal tools, and for scripts? Sure. It’s incredibly useful with all sorts of non- application development tasks. I’ve not written a batch or bash script in forever…you just don’t really need to much anymore. The linear flow of most batch/bash/scripts (like you mentioned) couldn’t be a more suitable domain.
Also, with a basic prompt, it can be an incredibly useful rubber duck. For example, I’ll say something like “how do you think I should solve x problem”(with tools for the codebase and such, of course), and then over time having rejected and been adversarial to every suggestion, I end up working through the problem and have a more concrete mental design. Think “over-eager junior know-it-all that tries to be right constantly” without the person attached and you get a better idea of what kind of LLM output you can expect including following false leads to test your ideas. For me it’s less about wanting a plan from the LLM, and more about talking through the problems I think my plan could solve better, when more things are considered outside the LLMs direct knowledge or access.
“We can’t do that, changing X would break Y external process because Z. Summarize that concern into a paragraph to be added to the knowledge base. Then, what other options would you suggest?”
when you stuck at claude doing dumb shit, you didnt give the model enough context to know better the system
after following spec driven development, works with LLM in large code base make it so much easier than without it like its heaven and hell differences
but also it increase in token cost exponentially, so there's that
Using it with rust is just horrible imho. Lots and lots of errors, I cant wait to stop this rust project already. But the project itself is quite complex
Go on the other hand is super productive, mainly because the language is already very simple. I can move 2x fast
Typescript is fine, I use it for react components and it will do animations Im lazy to do...
SQL and postgresql is fine, I can do it without it also, I just dont like to write stored functions cuz of the boilerplatey syntax, a little speed up saves me from carpal tunnel
- step A: ask AI to write a featureA-requirements.md file at the root of the project, I give it a general description for the task, then have it ask me as many questions as possible to refine user stories and requirements. It generally comes up with a dozen or more of questions, of which multiples I would've not thought about and found out much later. Time: between 5 and 40 minutes. It's very detailed.
- step B: after we refine the requirements (functional and non functional) we write together a todo plan as featureA-todo.md. I refine the plan again, this is generally shorter than the requirements and I'm generally done in less than 10 minutes.
- step C: implementation phase. Again the AI does most of the job, I correct it at each edit and point flaws. Are there cases where I would've done that faster? Maybe. I can still jump in the editor and do the changes I want. This step in general includes comprehensive tests for all the requirements and edge cases we have found in step A, both functional, integration and E2Es. This really varies but it is generally highly tied to the quality of phase A and B. It can be as little as few minutes (especially true when we indeed come up with the most effective plan) and as much as few hours.
- step D: documentation and PR description. With all of this context (in requirements and todos) at this point updating any relevant documentation and writing the PR description is a very short experiment.
In all of that: I have textual files with precise coding style guidelines, comprehensive readmes to give precise context, etc that get referenced in the context.
Bottom line: you might be doing something profoundly wrong, because in my case, all of this planning, requirements gathering, testing, documenting etc is pushing me to deliver a much higher quality engineering work.
[1] https://cloud.google.com/vertex-ai/generative-ai/pricing
So, not yet, but maybe someday?
I don't need to spin up an entire feature in a few seconds. I need help understanding where something is broken; what are some opinions o best practice; or finding out what a poorly written snippet is doing.
context still v important for this though and I appreciate cranking that capacity. "read 15000 stackoverflow posts for me please"
Where I have minimal knowledge about the framework or language, I ask a lot of questions about how the implementation would work, what the tradeoffs are etc. This is to minimize any misunderstanding between me and the tool. Then I ask it to write the implementation plan, and execute it one by one.
Cursor lets you have multiple tabs open so I'll have a Ask mode and Agent mode running in parallel.
This is a lot slower, and if it was a language/framework I'm familiar with I'm more likely to execute the plan myself.
I won't going into a case by case list of its failures, The core of the issue is misaligned incentives, which I want to get into:
1. The incentives for coding agent, in general and claude, are writing LOTS of code. None of them — O — are good at the planning and verification.
2. The involvement of the human, ironically, in a haphazard way in the agent's process. And this has to do with how the problem of coding for these agents is defined. Human developers are like snow flakes when it comes to opinions on software design, there is no way to apply each's preference(except paper machet and superglue SO, Reddit threads and books) to the design of the system in any meaningful way and that makes a simple system way too complex or it makes a complex problem simplistic.
- There is no way to evolve the plan to accept new preferences except text in CLAUDE.md file in git that you will have to read through and edit.
- There is no way to know the near term effect of code choices now on 1 week from now.
- So much code is written that asking a person to review it in case you are at the envelope and pushing the limit feels morally wrong and an insane ask. How many of your Code reviews are instead replaced by 15-30 min design meetings to instead solicit feedback on design of the PR — because it so complex — and just push the PR into dev? WTF am I even doing I wonder.
- It does not know how far to explore for better rewards and does not know it better from local rewards, Resulting in commented out tests and deleting arbitrary code, to make its plan "work".
"Are there any bugs in the current diff"
It analyzes the changes very thoroughly, often finds very subtle bugs that would cost hours of time/deployments down the line, and points out a bunch of things to think through for correctness.
At least for my use, 200K context is fine, but I’d like to see a lot faster task completion. I feel like more people would be OK with the smaller context if the agent acts quickly (vs waiting 2-3 mins per prompt).
Its really expensive to use.
But man I’m at the perfect stage in my career for these tools. I know a lot, I understand a lot, I have a lot of great ideas-but I’m getting kinda tired of hammering out code all day long. Now with Claude I am just busting ass executing in all these ideas and tests and fixes-never going back!
EDIT: for the moment... it supports 0 tokens of context xD
Should we expect the higher limit to also increase the practical context size proportionally?
throwaway888abc•9h ago