> This can lead to the paradoxical phenomenon of a party becoming increasingly successful while simultaneously inflicting negative outcomes to the majority of their supporters; or of "players" deliberately abandoning accurate sources of objective information about their current and future "game state", in favor of sources controlled by other players that are designed to offer narratives about those states that are factually inaccurate to the point of being detrimental as guidance, but nevertheless provide emotional payoffs such as a sense of identity, comfort, or being on a "winning" side.

Tao undervalues the importance of identity. As he says, it's not a single-player game, and as we all know, players will form teams. If you can convince your opponents to instead play solo (by, say, deconstructing their sense of identity), while you keep yours, you've basically won.

This is mostly just a really high-level overview and the exciting stuff is only teased in the conclusion:

> (There is a nascent field of epistemic game theory, as well as some models of social media manipulation, but these fields are still in their infancy.) A more systematic study of such games would help provide a basic conceptual framework to understand these very real dynamics, and develop strategies to counter or mitigate them.

Time for a renaissance! Honestly game theory feels more practically relevant now than earlier with MAD, and it also seems obvious that the "rational actor" posited by classical behavioural economics is a pretty limited abstraction if you're interested in modeling the world. Besides politics/misinformation and wild stuff that happens in aggregate at the highest levels of "rational" economics policy.. it also feels like "management science" never really succeeded in actually saying much about the difference between healthy vs unhealthy bureaucracies, and the varieties and lifecycles of these kinds of systems. Plus epistemic/nonmonotonic logics capable of explicit belief modeling seems very well positioned for analyzing and architecting with AI systems, like checking theoretical properties of agentic interaction protocols, or answering what good mixtures of (credulousness for creativity) vs (skeptics for grounding beliefs) look like, etc.

Here's a really interesting thing, basically TLA+ style model-checking engine that supports agents, environments, protocols etc and explicitly takes into account epistemics: https://sail.doc.ic.ac.uk/software/mcmas/ Anyone else know of similar things? Software suites that are useful for game-theoretical analysis and modeling are kind of hard to find unless it's yet another toy for prisoners dillema.

Belief-and-knowledge stuff seems to be consulted and adopted in robotics/autonomous vehicles research sometimes, a place where wrong answers actually matter. But I sort of expect modeling/specs/invariants/determinism to continue to be kind of neglected almost everywhere else, because resolving ambiguity in advance is kind of threatening for groups that benefit from a zero-theory "just try it!" and "you're doing it wrong, buy more tokens and use this framework" kind of approach with AI and ML. Hope this changes.