With the age of agents CI costs are generally going up, how is your build setup handling things? Have you been able to optimize, and how did you? Most importantly, how fast is it?
Comments
verdverm•1h ago
Trigger a review agent for any commit which has an open PR and has passed build. Use open weights, or at minimum a different model family from the one which wrote the code. I will be publishing a write up soon(tm), inspired by but different from, cloudflare's post.
I would napkin math the averaged cost per-review to be less than $1, which is less than our most rigorous validation pathway in CI (needs GPU for up to an hour). Time-wise around ~10-15m, with variability - and a bug in Argo with pod scheduling delays. This is fine because developers check in on their own PRs when they do and then make the changes that are needed. A key point here is that we are keeping humans in the loop in the PR.
This is a large monorepo, I would expect much lower numbers for polyrepo setups.
easterncalculus•24m ago
That's impressive! Would certainly read it when it comes out.
verdverm•1h ago
I would napkin math the averaged cost per-review to be less than $1, which is less than our most rigorous validation pathway in CI (needs GPU for up to an hour). Time-wise around ~10-15m, with variability - and a bug in Argo with pod scheduling delays. This is fine because developers check in on their own PRs when they do and then make the changes that are needed. A key point here is that we are keeping humans in the loop in the PR.
This is a large monorepo, I would expect much lower numbers for polyrepo setups.
easterncalculus•24m ago