After taking a deep look at the new prompting guides from for the 5.5 series of Opus, Sonnet, and Haiku we realized we need to change our approach to our software factory tool.
What is now apparent is we can not run a "generic" agent.md role file. While prompting has gotten "Lighter" is now more flavored if you will.
So when we have a gauntlet round to see which model is the better security reviewer, comment auditor, coder, plan writer we can no longer have:
I do not know if that was obvious to everyone else but it caught use flatfooted. So we went back to the drawing board restructured everything and replayed the trascripts the tests the evaluation anything judgement related back through the models and fine tuned the role definition to be by the model and saw an immediate drop in loops; near missing and redo; ceremony in the factory became much less of an issue immediately. The cool part was being able to use any of the models opus 5.5, sonnet 5.5, haiku 5.5, sol 6.1, terra 6.1 on a surprisingly wide range of tasks. Haiku turning out to be a relatively good security reviewer was not something I expected at all (still skeptical but pushing it hard to see if it hold up).
So with all that I did go back and update fleet-generator as a first public item. It now creates files derived from this new work that anyone can grab and use; compare them to what you are doing; or compare them to prior repo commits and see if this can help you.
survivor0xb0b•13m ago
What is now apparent is we can not run a "generic" agent.md role file. While prompting has gotten "Lighter" is now more flavored if you will.
So when we have a gauntlet round to see which model is the better security reviewer, comment auditor, coder, plan writer we can no longer have:
sec-review.md com-audit.md planner.md
That works but is now sub optimal we now need
sec-review-fable.md sec-review-opus.md com-audit-hiaku.md com-audit-sonnet.md
I do not know if that was obvious to everyone else but it caught use flatfooted. So we went back to the drawing board restructured everything and replayed the trascripts the tests the evaluation anything judgement related back through the models and fine tuned the role definition to be by the model and saw an immediate drop in loops; near missing and redo; ceremony in the factory became much less of an issue immediately. The cool part was being able to use any of the models opus 5.5, sonnet 5.5, haiku 5.5, sol 6.1, terra 6.1 on a surprisingly wide range of tasks. Haiku turning out to be a relatively good security reviewer was not something I expected at all (still skeptical but pushing it hard to see if it hold up).
So with all that I did go back and update fleet-generator as a first public item. It now creates files derived from this new work that anyone can grab and use; compare them to what you are doing; or compare them to prior repo commits and see if this can help you.