However, I can't read this, not because I refuse to -- I want to -- but because my the cogs in my mind just stop whirring at all of the LLM-isms, my eyes glaze over, and I stop being able to follow what the article is trying to communicate to me. I would love for the OP to call me out and correct me (that they haven't used LLMs to write this).
Seems like a disconnect there.
I can get good code out of LLMs. I can rarely get good writing out of them.
Maybe it's a skill issue, but it's not inherently a contradiction that they're better at some things than others. It's much easier to RL coding than writing.
Now I just assume that it's AI-slop and there's no other "way of thinking" that I can adopt to make it sensible. I'm surely missing out on learning through some human-written articles, but the volume of slop means that I can't spend all my time digging through nonsense in the hope that there's a gem of insight.
110% ai
I think I may be developing defensive scanning strategies, that are probably useful for avoiding slop, but not good for my focus or general reading comprehension.
What people are describing is more like a "software kitchen" or "software machine shop".
like to torture the metaphors, i think we already have the factory (coding agents you can ask to manufacture software to a spec) but it's the whole process including QA, shipping, listening to users and iterating that has to happen over time (unless you have an accurate world simulation so you know exactly how your product will be received... and I know there are people working on that too but I'm skeptical)
Writing software is like building a factory. You try to find the optimal process to do something. This is a creative process and can never be straight forward. Using the software is running the factory.
This won't change with AI, since you never now all requirements beforehand and you always will have to iterate.
The only reason people are selling software factories is that customers like to imagine that this was possible. That a creative, iterative process can be perfectly planable and predictable.
This is just wrong.
And international distributuon and scaling is simple too, compared to physical goods.
A factory in the traditional sense is a process and infrastructure tuned to run efficiently that produces something of value that does not require highly skilled/expensive labor.
Agent swarms, self learning, Ralph loops, execution DAGs, spending hours trying to convey my preferences into skills, yada yada — the reality is that no matter how long I spend trying to close the amortization gap between my expertise and (insert autonomous agent loop) … I always come back to see shit that I have opinions about and need to polish.
If you’re keeping on top of your craft, I just don’t think general purpose pretraining / RL is going to give you something which perfectly copies you. It can get closer and closer! But I’ve never seen agent work where I didn’t immediately start thinking of refinements …
I guess many things are okay if you’re okay with averages
In the article they talk of a swarm of agents speccing out the software and I can't help but wonder -- at that point, what are you creating, and for who??
Imagine a large multi dimensional grid. Each axis of the grid represents a spectrum of how a specific feature could be implemented. You thus have one axis per feature.
Now you plot a cloud of random points all over the grid and at each point have an agent build a solution where the features are expressed according to the corresponding intensity values at the point.
Now you go through and review the solutions you like, create more points around clusters where you kind of like what is going on, until the perfect product emerges.
I'm spending a relatively large amount of time waiting for CI to finish running, then clicking a button for an agent to fix failures, then waiting for CI to finish again. Might as well push that down into the core development loop.
Do folks have advice for doing this in practice?
Hooks in the harness? Pre-commit hooks? Different GitHub configurations?
2. Split out "syntactically correct" fast checks like linters into a standalone script and call it along with the slower checks in a full-check script
3. Set the fast-check script as a pre-commit hook and the full-check script as a pre-push hook.
4. Give the model instructions that it needs to run the fast-check script after every change and the full-check script when it thinks it's done.
5. Run full-check in CI.
Then you're good as long as the model doesn't bypass the pre-push hook, and even then CI will catch it.
Then push straight to main? Or do we need GitHub to help manage the queue https://docs.github.com/en/repositories/configuring-branches...
I DO see small instances where this is overcome, albeit, it occurs when risk is minimal, the work is non-critical, and the required environment is already available. For example, exploratory research of new features. Before, this would be quite costly and particularly cumbersome since the results may indicate that the new feature isn't viable and the work is basically "wasted." Now, you can send some agents to explore it for you and give you a "first pass" on the validity of the concept. You still end up with humans taking over immediately after (often to verify that first pass), but this has allowed a bit more efficiency in terms of understanding when investing that human time in exploratory investigation is appropriate and when it isn't. People are willing to give agents this level of control because the alternative is usually just a default "no."
Regardless, it seems obvious we're in a painful transition period where everyone in software can see that there's a ton of value here, but where everyone is simultaneously constrained by existing infrastructure, processes, expectations, etc., so as to not be willing or able to tap into that value. I suspect it won't be until there's just more new companies coming in without these existing constraints will the actual new version of these processes start to emerge and MAYBE some of the existing companies will be more willing to shake things up as needed to take advantage of it.
That's because it's still painfully clear that genAI has no taste. It's a median content generator. And the median kinda sucks. Of course you still need people to review the output.
We'll see if in the future, as people begin software projects this way, if there is more standardization. I suspect that there's just too much going on too fast at the moment to do it any other way, a decent factory for Opus 4.8 looks very different than one good for Astra, I'd guess. And models are just one axis.
Long term, can we? If not, what do we do once we’re addicted to the new process and it’s impossible to get out and from?
So to say that no one has built a software factory is a strong statement...and factually incorrect.
i'm not at the point where I'd go totally hands-off with code review / QA, so this setup is the right balance of automation for my current read on agent capability.
Like:
> A merge queue is a script you run to push to origin/main (instead of using a PR UI like we did back in the GitHub, all-human days).
This really doesn’t capture what a merge queue is. It’s a queue of change sets to merge, and CI has to pass on each change set before it can merge. There’s still a CI process. You can’t really enforce that just by having a bash script agents run locally, you need something at a higher level to enforce it, which is… CI.
CI doesn’t have to be slow, and the author isn’t even arguing for removing the tests.
asdev•53m ago