https://github.com/usestrix/strix/blob/main/strix/prompts/vu...
... and this is great, I'm not dunking, but pretty basic?
We just had the DARPA AIxCC results come in, and those systems are (1) open source and (2) presumably simpler/less polished than Xbow (some of the authors will be quick to tell you that they're doing PoC work, not product development), and (3) they're more complicated than this.
Again, to be super clear: I think there's a huge amount of potential in building something like this up. Nessus was much simpler than ISS when it first shipped, but you'd rather be Nessus than an ISS scanner developer! I'm just: why set this bar for your project?
Best of luck with this!
Good luck!
[0]: Why I say this --- a 10kLOC piece of software that was mostly human-written would require a large amount of testing, even manual, to ensure that it works, reliably, at all. All this testing and experimentation would naturally force a certain depth of exploration for the approach, the LLM prompts, etc across a variety of usecases. A mostly AI-written codebase of this size would've required much less testing to get it to "doesn't crash and runs reliably", and so this depth is not a given anymore.
waihtis•2h ago
_pdp_•2h ago
ericmcer•1h ago
ai-christianson•1h ago
0cf8612b2e1e•1h ago
codys•1h ago
Still could be worth doing a bit of manual work like this, but it's worth being cautious about drawing conclusions from it.
tptacek•1h ago
There's nothing fundamentally bad about having Oompa Loompa's behind the scenes, as long as you're honest about the outcomes you can provide.
I agree, though: also a very sensible way to prioritize development work.
Steeeve•1h ago
tptacek•1h ago
guhcampos•1h ago
At least they're not lying right? It's just people using computers.