The game consists of 6 geography-related questions (countries, capitals, flags, languages, borders, etc) every day, where the rarer your answer, the rarer your score (tiers are [common, uncommon, rare, ultra rare, legendary]). No account needed or anything.
Something I didn't expect to cause so many issues were definitions. What counts as a country? A territory? When does a country 'speak' a language? I would set a definition, and players would always (justifiably so!) break it -> "Well OK then if this is the definition, how come you don't accept [very obscure super borderline answer]". Eventually I converged on a happy-medium with enough inclusion (I think), without the definition being a whole paragraph long:
(i) Country: a sovereign state. Niue and the Cook Islands still cause problems: the UN lists them as "non-member states", but they exhibit quite a bit of sovereignty, so I count them as countries.
(ii) Territory: a place with an ISO 3166-1 alpha-2 code, given or reserved, that isn't a sovereign state (e.g., Heard Island and McDonald Islands, Sint Maarten, Sark,...)
(iii) Language spoken in {country}: working or official language, as said so by the government (or the entity's governing body)
Two components I think would be interesting for Show HN, and would like feedback on, are how I determine an answer's rarity and the day's difficulty.
Difficulty: I trained a linear regression model on past play data (30 days, currently) to predict a number between 0 and 1. For each of the first 5 questions (the 6th is different in nature), I gather 3 features for the regression model: ln(num. possible answers), ln(1 + num. 'common' answers), and whether answers contain a 'legandary' (0 or 1), then take the weighted average. Fixed cut-offs snap that into a 1-5 difficulty level (e.g., 4/5).
Rarity (e.g, "Name a country in the EU" -> Belgium):
(i) Fame. Using Google Books Ngram (how often the name appears in English), QRank (pageviews of the name's wikidata item, summed over every wikipedia language), and Wikipedia page views (median yearly views), I calculate a "Fame Score" for every country, territory, and capital (e.g., USA: 7.18, France: 6.85, Tuvalu: 5.76)
(ii) Weight. Sometimes rarities still don't feel quite right, so every question template (e.g., "Name a country in Europe") gets a weight based on how much information it gives the player. Luxembourg is valid for both "Name a country" and "Name a country in Europe", but knowing it's in Europe narrows your possibilities, making Luxembourg an "easier" answer for the latter.
(iii) Multiply Fame times the Weight to get a rarity score from 0-100, snapped to one of the rarity tiers.
It's a side project, and I'm continuously improving things based on feedback from players emailing me. Perhaps some of you could also provide some feedback?
1. How did the game 'feel' to you? 2. Were any of your answers rejected? 3. What is something you like about the game? 4. What is something you dislike about the game? 5. Any other miscellaneous observations/suggestions/bugs
Anything is appreciated. Thank you for your read, and have a good day/night!