I have some complex large and strict compliance projects that the AI is a pair programmer but I make most of the decisions, and I have smaller projects that, despite great impact on the bottom line, can be entirely done unsupervised due to the low risk factor of "mistakes" and the easiness of correcting them after the fact they are caught by the AI as well.
I wrote beads-skills for Claude that I'll release soon to enforce this process.
2026 will be the year of agent orchestration for those of us who are frustrated having 10 different agents to check on constantly.
gastown is cool but too opinionated.
I'm excited about this promising new project: https://github.com/jzila/canopy
We're writing an internal tool to help with planning, which most people don't think is a problem but I think is a serious problem. Most plans are either too long and/or you end up repeating yourself.
I mainly work in Python, and I've been ensuring that all of my projects have a test suite which runs cleanly with "uv run pytest" - using a dev dependency group to ensure the right dependencies are installed.
This means I can run Claude Code against any of my repos and tell it "run 'uv run pytest', then implement ..." - which is a shortcut for having it use TDD and write tests for the code it's building, which is essential for having coding agents produce working code that they've tested before they commit.
Once this is working well I can drop ideas directly into the Claude app on my iPhone and get 80% of the implementation of the idea done by the time I get back to a laptop to finish it off.
I wrote a bit about "uv run pytest" and dependency groups here: https://til.simonwillison.net/uv/dependency-groups
MESA drivers are a godsend in Claude Code for web if working on non-web GUIs. It can take screenshots and otherwise interact with them.
#!/usr/bin/env -S uv run --script
# /// script
# dependencies = [
# "requests<3",
# "rich",
# ]
# ///
import requests, rich
# ... script goes here`
I'm curious, for those of you who work like this, what level of subscription do you have?
I often have Claude Code build little HTML+JavaScript tools which are simple enough that I don't need a template ("Vanilla JavaScript, no dependencies or build script, test it with Playwright").
I've built a couple of Go apps which works great because Go has such strong default patterns and a full featured standard library, so again no need for a template.
Huh, this is...unexpected. TIL, indeed!
For side projects? It's been a 10x+ multiplier.
1. The context window size to output accuracy problem seems like a genuinely hard problem
2. The vast majority of resources are being poured into driving down costs, not increasing output accuracy (and definitely not on gigantic context windows)
I will also add:
* I'm astounded at how terrible LLMs are at writing assembly in my experience. This seems like something they should excel at. So I'm genuinely confused on that piece.
/gh-issue [issue number]
/gh-pr [pr number]
Edit: replaced links to private github repo to pastebin.
Where I've automated more aggressively is everywhere around the code. My main challenge was running experiments repeatedly across different systems and keeping track of the various models I ran and their metrics, etc. I started using Skyportal.ai as an ops-side agent. For me, it's mostly: take the training code I just iterated on, automatically install and configure the system with the right ML stack, run experiments via prompt, and see my model metrics from there.
The first saves me days of work/month by sparing me endless paper pages of notes trying to figure out why things work in a certain way in legacy work codebases. The second spares me from having to dig too much in partially outdated or lacking documentation or having to melt my brain understanding the architecture of every different dependency.
So I just put major deps in my projects in a `_vendor` directory that contains the source code of the dependencies and if I have doubts LLMs dig into it and their test to shed light.
What I haven't seen anybody yet accomplish is produce quality software by having AI write them. I'm not saying they can't help here, but the bottleneck is still reviewing and as soon as you get sloppy, codebase quality goes south, and the product quality follows soon after.
I’ll sometimes have it help read really long error messages as well.
I got it to help me fix a reported security vulnerability, but it was a long road and I had to constantly work to keep it from going off the rails and adding insane amounts of complexity and extra code. It likely would have been faster for me to read up on the specific vulnerability, take a walk, and come back to my desk to write something up.
When I don't know what I want to do, I read existing code, think about it, and figure it out. Sometimes I'll sketch out ideas by writing code, then when I have something I like I'll get Claude to take my sketch as an example and having it go forward.
The big mistake I see people make is not knowing when to quit. Even with Opus 4.5 it still does weird things, and I've seen people end up arguing with Claude or trying to prompt engineer their way out of things when it would have been maybe 30 seconds of work to fix things manually. It's like people at shopping malls who spend 15 minutes driving in the parking lot to find a spot close to the door when they could have parked in the first spot they saw and walked to the door in less than a minute.
And as always, every line of code was written by me even if it wasn't written by me. I'm responsible for it, so I review all of it. If I wouldn't have written it on my own without AI assistance I don't commit it.
This is sage advice. I spent the better part of a day trying to steer Gemini into correcting an inconsistency when I likely could have solved it in under an hour. I think persevering with Gemini was due to a number of factors, including novelty, stubbornness, and (unfortunately) not knowing in detail what Gemini had written up to that point.
I eventually studied the resulting code, which ended up having a number of nested 'hacks' and required refactoring - more time wasted, but still much faster overall.
I think these calculations are very different now that we have these very good LLMs, but they aren't irrelevant.
For instance, this skill of web development: https://raw.githubusercontent.com/vercel-labs/web-interface-...
That’s too much for a model to carry in its context while it’s trying to do actual work.
Far better is to give that skill.md to a model and have it produce several hundred lines of code with a shebang at the top. Now you haven’t got a skill, you’ve got a script. And it’s a script the model can run any time to check its work, without knowing what the script does, how, or why - it just sees the errors. Now all your principles of web dev can be checked across your codebase in a few hundred milliseconds while burning zero tokens.
TDD is codification too: codifying in executable form the precise way you want your logic to work. Enforce a 10ms timeout on every unit test and as a side effect your model won’t be able to introduce I/O or anything else that prevents parallel, randomized execution of your test suite. It’s awesome to be able to run ALL the tests hundreds of times per day.
Constantly checking your UI matches your design system? Have the model write a script that looks at your frontend codebase and refuses to let the model commit anything that doesn’t match the design system.
Codification is an insanely powerful thing to build into your mindset.
Maybe one should just search for advice from the last 20 years on how to make a human development team more effective, and do that stuff.
It’s funny how this advice has always been around, but we needed to invent this new kind of idiot savant developer to get the human developers to want to do it…
Am I the only one who worries about agents creating malicious/unsafe code to execute?
You can think of it literally as turning prompts that are non-deterministic into code. But it is also the idea of documenting all the things you learned while doing something. It’s what companies do when they invent process for humans (eg. “expense reports must be approved by your manager and finance before you get reimbursed”)
AI is a great to for exploring and experimenting with ways to do things- but once you know how you want it done, AI is just a liability because it might randomly decide to do things some other way, and it might do the thing differently tomorrow even though that’s not what you want.
AI coding tools are burning massive token budgets on boilerplate thousands of tokens just to render simple interfaces.
Consider the token cost of "Hello World":
- Tkinter: `import tkinter as tk; tk.Button(text="Hello").pack()`
- React: 500MB of node_modules, and dependencies
Right now context windows token limits are finite and costly. What do you think?
My prediction is that tooling that manage token and context efficiency will become essential.
function Hello() { return <button>Hello</buttton> }
The more code, the more surface area the LLM needs to cover before understanding or implementing correctly.
Right now the solution to expensive token limits is the most token-efficient technology. let's reframe it better. Was react made to help humans organize code better or machines?
Is the High Code-to-Functionality Ratio 3 lines that do real work > 50 lines of setup really necessary?
At current prices you can pretty much get away with murder even for the most expensive models out there. You know, $14/million output tokens. 10k output tokens is 14 cents. Which is ~40k words, or whatever.
The way to use LLM's for development is to use the API.
But my side projects which I kinda abandoned a long time ago are getting a second life and it is really fun just to direct the agent instead of slowly re-aquire all of the knowledge and waste time typing in all the stuff into the computer.
Instead of having it write the code, I try to use it like a pair reviewer, critiquing as I go.
I ask it questions like "is it safe to pass null here", "can this function panic?", etc.
Or I'll ask it for opinions when I second guess my design choices. Sometimes I just want an authoritative answer to tell me my instincts are right.
So it becomes more like an extra smart IDE.
Actually writing code shouldn't be that mechanical. If it is, that may signify a lack of good abstractions. And some mechanical code is actually quite satisfying to write anyway.
> Sometimes I just want an authoritative answer to tell me my instincts are right.
You realize that LLM answers highly depend on how you frame your question?
Of course I do. I'm not a moron.
In these cases, I already know the most likely answer to my question. The LLM just helps me reduce my self doubt
It doesn't write the code for me, but I talk to it like it is a personal technical consultant on this product and it has been very helpful.
The more time you spend making guidelines and guardrails, the more success the LLM has at acing your prompt. There I created a wizard to get it right from the beginning, simplifying and "guiding" you into thinking what you want to achieve.
AI excels at finding the "seams," those spots where a feature connects to the underlying tech stack, and figuring out how the feature is really implemented. You might think just asking Claude or Cursor to grab a feature from a repo works, but in practice they often miss pieces because key code can be scattered in unexpected places. Our skills fix that by giving structured, complete guides so the AI ports it accurately. For example, if an e-commerce platform has payments built in and you need payments in your software, you can reference the exact implementation and adapt it reliably.
The generators typically generates about 90% of the code I need to write a biz app. Leaving the most important code to me: the biz logic.
No AI. Just code that takes a (simple) declarative spec file and generates Typescript/C++/Java/... code.
I am also using AI's daily. However the code generators are still generating more productivity for me than AI's ever have.
What would you make if you could make anything? Does it all just lose meaning?
True I spent a year making a platform for using the API.. but the results... are stupendous!! Very cheap and unlimited access and custom tooling, etc... to get large amounts done of anything you want to do with an LLM!
Looks like this: https://mastodon.social/@rcarmo/115937685095982965
SyncThing syncs their workspaces to my desktop/laptop, I make small adjustments (I don’t do stupid wasteful things like the Ralph approach, I prefer clear SPEC documents and TODO checklists, plus extensive testing and switching models for doing code audits on each other), my changes sync back, etc.
I’m considering calling it “million monkeys”, really.
The real issue is that I can't trust the AI generated code, or trust the AI to code-review for me. Some repeated issues I see:
- In my experience the AI doesn't integrate well with the code that there is already there: it often rewrites functionality and tend not to adhere to the project's conventions, but rather use what it is trained on.
- The AI often lacks depth into more complex issues. And because it doesn't see the broader implication of changes, it often doesn't write the tests that would cover them. Developers that wrote the PRs accept the AI tests without much investigation into the code-base. Since the changes passes the (also insufficient) tests, they send the PR to code-review.
- With AI I think (?) I'm more often the one careful deep diving into the project and re-designing the generated code in the code-review. In a way it's an indirect re-prompting.
I'm very happy with the increased PRs: they push the project forward, with great ideas of what to implement, and I'm very happy about AI increased productivity. Also, with AI developers are bolder in their contributions.
But this doesn't scale -- or I'll spend all my time code-reviewing :) I hope the AIs get better quickly.
Forgive me because this is a bit of a tangential rant on the second issue, but Gemini Pro 3 was absolutely heinous about this so I cancelled my sub. I'm completely puzzled what it's supposed to be good for.
To your third issue, you should maybe consider building a dataset from those interactions... you might be able to train a LoRA on them and use it as a first pass before you lift a finger to scroll through a PR.
I think a really big issue is that there is a lack of consistency in the use of AI for SWE. There are a lot of models and poorly designed agents/assistants with really unforgivable performance and people just blindly using them without caring about the outputs amounts to something that is kind of Denial-of-Service-y and I keep seeing this issue be raised over and over again.
At the risk of sounding elitist, the world might be a better place for project maintainers when the free money stops rolling into the frontier labs to offer anyone and everyone free use of the models...never give a baby powertools and so on.
Testing is no yet the main focus, so we haven't looked into automating that. But we will in the future. We have automated most of our documentation updates though. After releases or big merges we use askmanu to automatically update/create the docs. These are internal docs, but super useful for us and Claude.
Disclaimer: I'm the founder of askmanu
jackfranklyn•2w ago
The pattern that works best for me: I describe what I want at a high level, let it scaffold, then I read through and course-correct. The reading step is crucial. Blindly accepting generates technical debt faster than you can imagine.
Where it really shines is the tedious stuff - writing tests for edge cases, refactoring patterns across multiple files, generating boilerplate that follows existing conventions. Things that would take me 45 minutes of context-switching I can knock out in 5.
The automation piece I've landed on: I let it handle file operations and running commands, but I stay in the loop on architecture decisions. The moment you start rubber-stamping those, you end up with a codebase you don't understand.
mandeepj•2w ago
Yeah, treat it like an intern or junior engineer who needs constant feedback and reviews.
lucabraccani•2w ago
langarus•2w ago
catigula•2w ago
Tests are where the moat still exists because prior to creating tests the outcomes are unverifiable.
Yodel0914•2w ago
lordnacho•2w ago
I rarely let it run for over 10 minutes unattended, but the benefits are not just pure time.
Being able to change the code without getting bogged down allows you to try more things. If I have to wait half an hour between iterations, I'm going to run into bedtime quite fast.
On top of this, I'm finding that the thing that takes the deepest attention is often, amazingly, trivial things. Fiddling with a regex takes attention, but it doesn't often decide the success of the project.
By contrast, the actual work, which is making technical decisions, is something I can do without concentrating in the same way. It's strange that the higher value thing feels less stressful.
Put these together and I'm more productive. I can string together a bunch of little things, and not have to be at my sharpest. Work can be scheduled for whenever, which means days are more flexible. More gets done, with less attention.
sjm-lbm•2w ago
When you send Claude Code something and already have an idea for what an acceptable solution looks like, you're a massive step ahead when it's time to read the diff and decide what you think of it. This does mean that every so often my productivity drops to basically zero as I try to understand what is actually happening before I just put the AI on the job, but so far it seems to be a good rule to keep in mind that allows me to use AI effectively while generating a code base that I still understand.