You get ten simple "what happened first" questions and give a confidence level (for each answer). The game tracks your success rates per confidence level and gives you a calibration graph/table (e.g., "you consistently over-estimate your success rate"). You can play multiple times to see aggregated statistics.
After 60 games (480 questions), I found out that I am relatively well-calibrated, though I should ~never bet under 60% (for this specific type of binary questions): https://alejo.ch/3mk
The game is open source (GPLv3 at https://github.com/alefore/estimate) and it's just static files. There's no server-side logic: the game log/stats are stored in your browser (local storage API). It's pure TypeScript, no framework.
You can read more about it on https://alejo.ch/3m9
Just figured I'd share in case others find it useful and/or have suggestions to improve it. I'd also be curious to see the calibration stats of other people.