Building an open-source repo with deterministic validators backed by agent evaluation loops to turn agent instructions (.e.g. .cursorrules, claude-guidelines.md, copilot-prompts.md) into hard guarantees.
Comments
dkargatzis•1h ago
Hi HN!
Maintainers are increasingly dealing with large volumes of low-context PRs, especially as AI coding tools make it easier to generate changes without the surrounding intent.
Many repos already define contribution expectations in places like .cursorrules, CONTRIBUTING.md, or internal guidelines but those rules rarely translate into something enforceable during review.
We built Watchflow to bridge that gap: an open-source GitHub check that translates repository guidelines into enforceable validation logic that runs on every PR.
The model is intentionally hybrid:
- ~95% deterministic validators (fast, predictable, pure Python)
- Optional LLM checks for semantic cases like description / diff alignment
- LLM layer fails open so merges never block on model issues
Example rules:
- Require a linked issue
- Require CODEOWNERS review
- Block PRs above a LOC threshold
- Ensure PR descriptions align with code changes
- Validate contribution patterns defined in .cursorrules or agent guidelines
The goal is to close the loop between repository intent -> code diff -> enforceable governance.
dkargatzis•1h ago
Maintainers are increasingly dealing with large volumes of low-context PRs, especially as AI coding tools make it easier to generate changes without the surrounding intent.
Many repos already define contribution expectations in places like .cursorrules, CONTRIBUTING.md, or internal guidelines but those rules rarely translate into something enforceable during review.
We built Watchflow to bridge that gap: an open-source GitHub check that translates repository guidelines into enforceable validation logic that runs on every PR.
The model is intentionally hybrid:
- ~95% deterministic validators (fast, predictable, pure Python)
- Optional LLM checks for semantic cases like description / diff alignment
- LLM layer fails open so merges never block on model issues
Example rules:
- Require a linked issue
- Require CODEOWNERS review
- Block PRs above a LOC threshold
- Ensure PR descriptions align with code changes
- Validate contribution patterns defined in .cursorrules or agent guidelines
The goal is to close the loop between repository intent -> code diff -> enforceable governance.
Repo: https://github.com/warestack/watchflow
If there are review patterns you’re seeing that aren’t covered yet, happy to turn them into new validators!