Unless its an easy memory/parallel/algorithmic win, its not worth it.
The great thing about LLM is that it seems to have the checklist for everything. If I rattle off a few things like "don't allocate on the hot path" and "remember to pin the cores" it will come up with a few items of its own that I might have forgotten.
Eventually, it will have gone through the whole list with me, while having documented all the measurements along the way.
But it's still guided by experience. If I see unusual numbers, I might say "hey did you forget to compile it in release mode?" and it will apologize and fix that. If I don't, it may just continue exploring without realising everything is wrong.
hombre_fatal•31m ago
Once I had repo commands that could dump `sample` results and a cpu profiler/trace and then a benchmark tool that let me A/A + ABBA/BAAB-test the current modified git workspace against HEAD or any commit, the LLMs could just do their thing.
And that's how my homemade terminal uses much less memory than ghostty/kitty/iterm yet has more throughput.
AI is going to increasingly unmask people and companies who don't care about correct and performant software now that it's become so trivial to guarantee both. It used to at least be expensive and time-consuming and expertise-demanding to do those things.
Capricorn2481•17m ago
Then they can start attempting to optimize it. They can also spin round and round making the numbers worse because they don't actually know what to do.
hombre_fatal•15m ago
You need a measurement that can falsify hypotheses and reject branches that won't work.
Also, if all you have left in your project are performance issues that are hard to identify without flailing around (even with Fable/Astra) despite sampler/profiler reports, then you're doing really well.
minimaxir•13m ago
In one case I used a made-up metric (since I didn't know the exact name or if it existed) and it somehow optimized that too.
dasil003•14m ago
This isn't a new problem by any means, but now that code is cheap, it means instead of getting frustrated with engineering and their pesky unimportant details, people will get frustrated with the AI and it's pesky unimportant details.
hombre_fatal•6m ago
I think it's one reason why ADRs are an important of a software project, especially with LLMs. You need a place were you can document invariants, why you have them + the rejected ideas and acceptable risks.
It helps smart agents like Fable help you decide on trade-offs and it's kind of incredible to witness that happening.