I created a github app that allows people to label suspicious LLM slop prs based on heuristic driven scoring of the submitter's github profile, activity, and activity on the repo itself.

Profiles and PRS that fail heuristics have the PR labelled appropriately.

I might also add LLM based scoring in the future, but for now this project uses only heuristic based scoring.

People who have already merged to the repo that the app is installed on should be exempted from the scan.

The project is a FastAPI service that processes requests using background tasks (https://fastapi.tiangolo.com/tutorial/background-tasks/#depe...), to allow for the github webhooks to return in a timely manner.

Might have to upgrade to a proper celery worker if this takes off.

Github app link: https://github.com/apps/pr-slop-stopper

The project uses UV for dependency management, ty for type checking and ruff for linting.

AI DISCLOSURE: I heavily used Claude Code to generate this, but I did read the code during development to ensure it aligned with my personal tastes and habits.

NOTE: I have not been able to test this myself, I tried making a fake github account and test it via forking the source repo (I have the app installed on source repo). Unfortunately, github does not allow forking repos to new accounts under a week old. I verified that the webhook urls are configured correctly with secrets properly being accepted by the server, but have not been able to guarantee functionality. If someone would like, feel free to open a low quality PR and lets see if it gets flagged! (I can almost guarantee it will break since I haven't been able to test this live yet. But a broken thing is the first step to a working thing)