frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Show HN: I built a 2-min quiz that shows you how bad you are at estimating

https://www.convexly.app/
17•convexly•9h ago
I've gotten to the point in my career where I now make strategic decisions often (hiring, firing, choosing what equipment to go with, etc.), as well as in my personal life where I need to strongly weigh my options for a big purchase or investment. I found a not-so-surprising parallel between the two as these decisions "resolved." Am I making good decisions or am I getting lucky?

Did some research, read some books, and realized I should get in the habit of tracking my decision process. That quickly turned into the idea that formed Convexly.

The landing page is a 10-question calibration quiz where you assign a confidence level to statements drawn from a rotating pool of 100 (working on making the pool larger) and you get a Brier score back instantly. No signup required, and you can share your scores right away.

If you find it interesting, you can create a free account where you can track your decisions with probability estimates, resolve them over time, and get calibration curves that show if you are over/underconfident. From what I've seen so far, users are overconfident when they say they're between 70-90% sure about something.

For the math: Beta-PERT distributions for the payoff modeling, Kelly criterion for the position sizing, signal detection theory for separating skill from randomness.

On the coding side: FastAPI with NumPy/SciPy, frontend in Next.js and Supabase.

So far this has been a solo project of mine. If you want to see all the features use code SHOWHN for 30 days of full access, no credit card required.

Curious if anything about your score surprised you after taking the quiz.

Comments

Evgeniuz•1h ago
There’s a bias, I think. When I saw the title that is about how bad I’m at estimating, I’ve leaned towards counterintuitive answers. This got me quite a high score. I think test set should also include intuitive facts (or maybe I was just lucky).
convexly•1h ago
As much as it is counterintuitive, that is actually a valid calibration strategy. If you notice the questions lean slightly towards counterintuitive and adjust for it, that IS better calibration! But you raise a fair point about framing bias from the title.
convexly•1h ago
Interesting data from the quiz so far: 160+ quiz takers! The average is 0.239 (barely better than a coin flip at 0.25), but almost everyone indicates they are confident in their answers.
convexly•1h ago
Update: 400+ quiz takers now... insane. Best Brier score so far is 0.007 (nearly perfect calibration). The worst came in at 0.600. Average is 0.230, still just better than a coin flip. Where did you land?
tommica•1h ago
Worst came in 0.600? Fuck, I got 0.550...
convexly•1h ago
Just need practice! People have no idea how overconfident they actually are.
bovermyer•1h ago
I hit 0.012.

As a test of general knowledge it was interesting. The confidence angle was the most interesting part, though.

convexly•1h ago
That's the second best score I've seen today out of 700+ quiz takers! Exceptional calibration. The confidence angle is the whole point, people don't know how far off they actually are until they see the hard data!
iamtedd•1h ago
Why do I need to sign up to get the results? Why couldn't it just be on the page?
convexly•1h ago
The Brier score and "diagnosis" are shown immediately, no signup needed. The email is optional and only if you want to see the calibration curve and the question breakdown sent to you. I'll make that clearer!
reltnek•1h ago
I think this might be conflating confidence with accuracy. I tried leaving the slider the the middle (nominally the least confident position) and it gave a score of 0.25 and diagnosed it as 'overconfident'.
convexly•1h ago
That is definitely a bug, thank you for pointing that out. Should have been neutral! I'll push a fix for this.
macleginn•1h ago
The Brier score is pathological when the guess is 0.5: regardless of the outcome, it will be equal to 0.25, so if you define "better than random" as having a score < 0.25, actually acting randomly makes you "overconfident".
testycool•1h ago
I thought it was interesting, but don't appreciate having to give you my email to see full results.

I unsubscribe from mails that aren't useful to me day-to-day because they're distracting.

Other than that it seems like a cool idea. I'd recommend slightly bigger fonts. I often have this issue with Gemini.

  Brier Score: 0.216 (lower is better)
  Diagnosis: Overconfident
convexly•1h ago
Just pushed a fix for that! You should be able to see everything without inputting your email now. I've made a note about font size, thank you for the feedback.
suralind•1h ago
Did it twice: once had 0.177, 2nd time got 0.280. Note sure what to make of this, I guess I should always leave it on 50/50?
convexly•1h ago
The variance is normal, the questions pull from a pool of 138 questions so far. 0.177 is strong. Setting everything to 50% would just get you 0.25, so you did way better on the first attempt. The goal isn't 50/50 on everything, only on the occasions where you are not confident that you are right.
convolvatron•1h ago
I didn't find the questions very representative about estimation. that is maybe if happen to know many of random root facts about the world under which they were based, then their application might be a revenant question about ability to estimate. I really felt more like I was making uneducated guesses (0.155). I suppose I was expecting more ping pong balls in airplanes
convexly•1h ago
The point I was going for was more so how people handle questions they don't know the answer to. Someone that is "well-calibrated" would set things they are uncertain about at closer to 50% instead of guessing one way or the other (overconfident). That score is excellent, so it suggests you did exactly that!
zupa-hu•1h ago
It is very disappointing that you can't see what you got right or wrong without giving out your email. I'm not even sure if one would learn from the email or whatever the calibration result is.

I'm happy for you if it works but I sure feel cheated. I hope others also feel it's against the spirit of a Show HN. But maybe it's just me.

convexly•1h ago
That's a good point, I might have gated it too hard. I'll open up the full results now. Appreciate the feedback.
fred_is_fred•1h ago
Is it down? The start and skip button both dont work and I see this error in my console.

Manifest fetch from https://www.convexly.app/manifest.json failed, code 403

convexly•1h ago
Just checked and everything is up. That might just be a console warning, but shouldn't affect the quiz. Can you try a hard refresh (ctrl+shift+R)? If that still doesn't work, what browser are you on?
fred_is_fred•1h ago
I tried Chrome and Safari. It's working great on my phone, so probably zscalar.
convexly•1h ago
For sure. Zscalar can block certain requests. Glad it works on your phone!
lorenzohess•1h ago
Maybe I don't know enough about "calibration" in a technical sense, but it seems like this quiz cant really distinguish between factual knowledge and calibration skill?

Is this type of quiz reproducible for individuals and across various cross-sections of the population?

Are there studies on this? Is the quiz based on these studies?

convexly•1h ago
Great question. Calibration specifically is about whether your confidence in an answer matches your accuracy, not whether you know the answer. Someone who knows a lot but is always 90% confident would score poorly even if they're wrong 20% of the time, as an example.

In terms of research, Tetlock's Expert Political Judgement and Superforecasting were the foundation. He did a 20 year study that showed domain experts were barely better than chance at long-range predictions. The Brier score was the standard metric for that research.

lorenzohess•1h ago
I see, that makes a lot of sense. Maybe the UI should reflect this? Have one button for True or False or Uncertain, and then the slider for confidence in the answer?
convexly•1h ago
That's a really good UX idea. I can see how it's not the most intuitive now. Separating the direction from the confidence level would make it much clearer. Adding that to my list.
addisonl•1h ago
> Question: A fair die rolling a 6 twice in a row is more likely than rolling 1-2-3-4-5-6 in sequence

Two 6s in a row is 1/36 chance (1/6)^2

1-2-3-4-5-6 is a 1/46656 chance (1/6)^6

Website is claiming they are the same probability:

> Same probability: 1/46,656 — Both outcomes have exactly the same probability: (1/6)^6 = 1/46,656. This illustrates the representativeness heuristic — random-looking sequences feel more probable than ordered ones.

Website's "answer" is wrong: was the question supposed to be rolling a 6 six times in a row?

1qaboutecs•1h ago
came with the same complaint. the website then had the nerve to tell me i am overconfident.
convexly•1h ago
Fair point! Bad question on my end. The overconfidence was based on all 10 questions though, not just that one!
convexly•1h ago
You're right, that's a mistake in how I phrased the question. It should say "six times in a row" not "twice in a row". Fixing it now! Thanks for pointing that out!
cyanydeez•1h ago
Yeah, most likely it was try to identify a bias of human perception, that 1,2,3,4,5,6 would be more probably than 6x6.

A better way to illustrate this bias is with coin flips. People will tell you that odds of 6 heads is more rare than the odds 3 tails then 3 heads. The difficulty is understanding whether they mean "in order" or "as a group".

If it's in order, the odds are the same. Every order of H/T has the same probability, but humans will see "all heads" and think that's more rare. But the important bit is whether there's a clear understanding ordering.

convexly•1h ago
That's definitely better framing for this question. Much cleaner way to illustrate that point!
snarf21•1h ago
If anyone is interested in why we are bad at estimating, please check out the amazing book Thinking, Fast and Slow: Daniel Kahneman.
convexly•50m ago
Great recommendation. That was one of the biggest influences for starting to write my decisions down and then building this.
Hnus•1h ago
Why is it asking for email?
convexly•1h ago
I just removed that, full results should be fully visible without email! A hard refresh should show the update.
gcanyon•1h ago
Wait, so roughly is it rewarding being confident when correct, and penalizing being confident when wrong? Meaning that the highest score is only achievable if you answer fully confident true or false, and get all 10 correct?

If so, isn't that conflating knowledge with over/under confidence?

convexly•59m ago
Your point on scoring is correct, if you're 100% confident and right on everything you would score a perfect 0. The calibration insight is in how you handle the questions where you don't know the answer. Say you're highly knowledgeable and 95% confident on everything, but get 2 wrong scores compared to someone that says they are 70% confident on those same two questions. That would indicate that you are overconfident compared to the other person!
macleginn•54m ago
How are they different? If you "know" something, you are 100% confident in it, which gives you an easy 0 for this question (or a surprising 1). Philosophically, the problem is more that there is no difference between confidently and modestly wrong in terms of consequences of binary decisions.
loloquwowndueo•1h ago
“You averaged 97% confidence but were right 80% of the time.”

Heck yeah.

convexly•1h ago
Actually means you have really strong knowledge. That 17% gap though is what gets people in trouble in high-stakes settings!
loloquwowndueo•39m ago
Yeah! I’m confident when I give an answer. In a real life scenario I would actually research the ones I’m not so sure about - but having a confident first take narrows down that research a lot.
convexly•33m ago
Can't fault that, actually a great workflow. Calibration helps you know which ones to research!
rahimnathwani•1h ago
This reminds me of:

https://taketest.xyz/confidence-calibration

The same site also has something with a fixed confidence level: https://taketest.xyz/ci-calibration

convexly•54m ago
That's awesome, hadn't seen this one! I like the confidence interval approach.
sonofhans•1h ago
I’ve taken the quiz but not been compelled to sign up. The site feels manipulative, e.g., the “show me all the questions” link is tiny and hidden between two larger boxes, and even then it only shows 2 questions with a signup CTA. Maybe that’s best practice growth hacking these days, but to me it’s a manipulative turnoff. If you’d given me all the questions and answers simply then I would signed up for more, especially with the discount code. Otherwise, how am I supposed to even know what I’m signup up for? Every interaction I’ve had with the site so far is a sales attempt, so mostly I expect more of those.
convexly•57m ago
That's honest feedback, I appreciate it. The post quiz shouldn't feel like a sales funnel. I'll clean that up. Working on it!
Havoc•53m ago
I'd consider removing some questions that are bound to be country specific. e.g. The one about time spent in front of a red light.

>0.188

Slightly above avg - yay

convexly•52m ago
That's fair, I'll flag those or maybe even add regional context. Nice score, well above average!
convexly•46m ago
Update at 2 hours: 1350+ quiz takers! 50% overconfident, 40% well-calibrated, and 10% underconfident. The average score is around 0.228, with the best score still at 0.007 (nearly perfect). The pattern so far is people are most overconfident in the 70-90% range, but are right closer to ~55% of the time.
convexly•27m ago
Made a few changes based on feedback from this thread: full results now shown immediately with no email gate, changed the UX to include true/false/uncertain buttons + a confidence slider, I cleaned up the quiz result page, and fixed the die probability question. Thanks for all the honest feedback!

Show HN: Ghost Pepper – Local hold-to-talk speech-to-text for macOS

https://github.com/matthartman/ghost-pepper
112•MattHart88•2h ago•55 comments

Ah, peptides. where to begin?

https://www.science.org/content/blog-post/ah-peptides-where-begin
32•A_D_E_P_T•50m ago•21 comments

Launch HN: Freestyle – Sandboxes for Coding Agents

https://www.freestyle.sh/
162•benswerd•5h ago•87 comments

A cryptography engineer's perspective on quantum computing timelines

https://words.filippo.io/crqc-timeline/
247•thadt•6h ago•105 comments

Show HN: GovAuctions lets you browse government auctions at once

https://www.govauctions.app/
144•player_piano•5h ago•54 comments

German police name alleged leaders of GandCrab and REvil ransomware groups

https://krebsonsecurity.com/2026/04/germany-doxes-unkn-head-of-ru-ransomware-gangs-revil-gandcrab/
235•Bender•8h ago•124 comments

Issue: Claude Code is unusable for complex engineering tasks with Feb updates

https://github.com/anthropics/claude-code/issues/42796
631•StanAngeloff•8h ago•403 comments

HackerRank (YC S11) Is Hiring

1•rvivek•1h ago

Sam Altman may control our future – can he be trusted?

https://www.newyorker.com/magazine/2026/04/13/sam-altman-may-control-our-future-can-he-be-trusted
346•adrianhon•11h ago•100 comments

What being ripped off taught me

https://belief.horse/notes/what-being-ripped-off-taught-me/
285•doctorhandshake•9h ago•164 comments

Book review: There Is No Antimemetics Division

https://www.stephendiehl.com/posts/no_antimimetics/
176•ibobev•8h ago•125 comments

Battle for Wesnoth: open-source, turn-based strategy game

https://www.wesnoth.org
334•akyuu•4h ago•84 comments

Show HN: Tusk for macOS and Gnome

https://shapemachine.xyz/tusk/
17•factorialboy•2d ago•2 comments

The cult of vibe coding is dogfooding run amok

https://bramcohen.com/p/the-cult-of-vibe-coding-is-insane
417•drob518•3h ago•342 comments

Sky – an Elm-inspired language that compiles to Go

https://github.com/anzellai/sky
109•whalesalad•6h ago•37 comments

Agent Reading Test

https://agentreadingtest.com
33•kaycebasques•3h ago•8 comments

Show HN: Hippo, biologically inspired memory for AI agents

https://github.com/kitfunso/hippo-memory
3•kitfunso•23m ago•0 comments

Show HN: TTF-DOOM – A raycaster running inside TrueType font hinting

https://github.com/4RH1T3CT0R7/ttf-doom
6•4RH1T3CT0R•2h ago•2 comments

The Last Quiet Thing

https://www.terrygodier.com/the-last-quiet-thing
115•coinfused•2d ago•76 comments

A macOS bug that causes TCP networking to stop working after 49.7 days

https://photon.codes/blog/we-found-a-ticking-time-bomb-in-macos-tcp-networking
93•RyanZhuuuu•1h ago•59 comments

Eighteen Years of Greytrapping – Is the Weirdness Finally Paying Off?

https://nxdomain.no/~peter/eighteen_years_of_greytrapping.html
40•jruohonen•2d ago•3 comments

Show HN: Docking – extensible Linux dock in Python

https://docking.cc
13•edumucelli•2d ago•3 comments

AI singer now occupies eleven spots on iTunes singles chart

https://www.showbiz411.com/2026/04/05/itunes-takeover-by-fake-ai-singer-eddie-dalton-now-occupies...
54•flinner•6h ago•70 comments

Root Persistence via macOS Recovery Mode Safari

https://yaseenghanem.com/recovery-unrestricted-write-access/
15•yaseeng•1h ago•10 comments

NYC Families Need over $125,000 in Income to Live in Any Borough

https://www.bloomberg.com/news/articles/2026-04-06/nyc-families-need-over-125-000-in-income-to-li...
15•boh•39m ago•5 comments

Adobe modifies hosts file to detect whether Creative Cloud is installed

https://www.osnews.com/story/144737/adobe-secretly-modifies-your-hosts-file-for-the-stupidest-rea...
186•rglullis•4h ago•87 comments

SOM: A minimal Smalltalk for teaching of and research on Virtual Machines

http://som-st.github.io/
11•tosh•3h ago•0 comments

Zooming UIs in 2026: Prezi, impress.js, and why I built something different

58•tinchox6•3h ago•32 comments

Reducto releases Deep Extract

https://reducto.ai/blog/reducto-deep-extract-agent
40•raunakchowdhuri•5h ago•5 comments

I won't download your app. The web version is a-ok

https://www.0xsid.com/blog/wont-download-your-app
776•ssiddharth•7h ago•457 comments