For me, Github often feels like a serious (and sometimes soulless) place. I built this website to showcase the human, silly side of developers. Repo Roast is built on top of LLM-powered GitHub analytics tooling we’re developing internally at Merit Systems [0].

One of the historically cited weak points for LLMs has been their sense of humor. I found that with the right prompt, at scale, they mostly can _identify_ humor pretty well. We experimented with a couple of different models to tune the cost/effectiveness and eventually landed on a strategy where we have gpt-4o-mini do a very light filter pass to reduce the number of calls to GPT-4o to do the actual humor scoring. Interestingly, our first prompt attempts ended up surfacing wayyy more aggressively NSFW content, which is funny that this happens despite all the censoring that OpenAI is doing.

So far we have processed the last 90 days of historical comments and continuously process new ones every hour. We theoretically can run this against a wider historical window given there is sufficient interest to justify the costs!

Repo Roast is in no way intended to make you a better or more productive dev. You can use it as a leaderboard or hiring tool (if you’re looking for personality), or just a way to distract yourself!

If you've had any funny Github interactions, I'd love if you shared them below.

Cheers!

--Mason

[0] https://www.merit.systems/