ELO is presumably more accurate for over the board games at tournaments where players bring their A game than low stakes online games where someone may be less engaged.
It is like asking someone to pick a random number between 1 and 1 million and then saying, “oh my god, it must not actually be random… the chances of choosing the exact number 729,619 is 1 in a million! That is too rare to be random!”
-Feynman, from Six Easy Pieces
You could look at a bunch of other metrics to identify cheating: how many errors/perfect moves^ and whether that's within the usual range. How well were the opponents playing? Etc
If you consider that Nakamura might have been having a good day/week, was already stronger than his opponents, and some of them may have had bad games/days, you can change something from "extremely unlikely" to "about a dice roll"
^ according to stockfish
The sound conclusion is that this is not evidence of cheating, but it is not evidence of the contrary either.
and then in the publication itself:
>>The likelihood principle [Edwards et al., 1963] is a fundamental concept in Bayesian statistics that states that the evidence from an experiment is contained in the likelihood function. It implies that the rules governing when data collection stops are irrelevant to data interpretation. It is entirely appropriate to collect data until a point has been proven or disproven, or until the data collector runs out of time, money, or patience
Surely there is a difference when you look at someone who played 46 games online in his life and scored 45.5 and when you look at someone who played 46000 games and scored 45.5/46 once.
The difference is that Kramnik wasn't "collecting the data" but looked at the whole Nakamura's playing history and found a streak.
Another example would be looking at coinflips and discarding everything before and after you encounter 10 heads in a row to claim you have solid evidence that the coin is biased.
They are misapplying the principle here. If what they wrote was correct then someone claiming: "Look, Nakamure won 100 out of 100 if you just look at games 3, 17, 21, 117...." would be proving Nakamura cheated if they applied methodology from the paper even assuming one in 10000 guilty players. Just because you can choose sampling strategy and stopping rules (what the likelyhood principle states) doesn't mean you can discard data you collected or cherry pick parts that support your hypothesis.
How the data is collected is absolutely relevant and Nakamura is right to point it out.
I don't quite understand this objection? If I won the lottery at odds of 10 million to 1, you'd say that was a very lucky purchase. But if it turned out I bought 10 million tickets, then that context would surely be important for interpreting what happened, even if the odds of that specific ticket winning would be unchanged?
5tk18•1h ago
cortesoft•1h ago
bleuarff•1h ago
adw•55m ago
mrala•1h ago
bediger4000•31m ago
RegnisGnaw•29m ago
joshuat•27m ago
thieving_magpie•10m ago
512•57m ago
rendall•30m ago
giancarlostoro•53m ago
ceejayoz•51m ago
Or, more mundanely, bathroom breaks. https://www.nytimes.com/2024/10/16/sports/kirill-shevchenko-...
Your iPhone can reliably beat the best chess players in the world.
AdamN•49m ago
ceejayoz•48m ago
recursive•24m ago
amdsn•42m ago
fwip•40m ago
michaelt•33m ago
You use a chess engine to tell you the best move - you can run a chess engine on a modern phone that will easily best the world's top human chess players.
The simplest forms of this are things like: "play online, chess engine open in another window", "use your phone hiding in a bathroom cubicle" and "member of the audience follows your game with a chess engine and signals you somehow"
There are also rumoured to be very subtle ways of doing this - like playing unassisted for most of the game, but an engine providing 'flashes of genius' at one or two crucial moves of the game.
Major competitions have things like metal detectors and time-delay video feeds hoping to make cheating harder.
kevin_thibedeau•21m ago
tomku•27m ago
The way people cheat online is by running a chess engine that analyzes the state of the board in their web browser/app and suggests moves and/or gives a +/- rating reflecting the balance of the game. Sometimes people run it on another device like their phone to evade detection, but the low-effort ways are a browser extension or background app that monitors the screen. The major online chess platforms are constantly/daily banning significant amounts of people trying to cheat in this way.
Chess.com and Lichess catch these cheaters using a variety of methods, some of which are kept secret to make it harder for cheaters to circumvent them. One obvious way is to automatically compare people's moves to the top few engine moves and look for correlations, which is quite effective for, say, catching people who are low-rated but pull out the engine to help them win games occasionally. It's not that good for top-level chess because a Magnus or Hikaru or basically anyone in the top few hundred players can bang out a series of extremely accurate moves in a critical spot - that's why they're top chess players, they're extremely good. Engine analysis can still catch high-level cheaters, but it often takes manual effort to isolate moves that even a world-champion-class human would not have come up with, and offers grounds for suspicion and further investigation rather than certainty.
For titled events and tournaments, Chess.com has what's effectively a custom browser (Proctor) that surveils players during their games, capturing their screen and recording the mics and cameras that Chess.com requires high-level players to make available to show their environment while they play. This is obviously extremely onerous for players, but there's often money on the line and players do not want to play against cheaters either so they largely put up with the inconvenience and privacy loss.
Despite all of the above, high-level online cheating still happens and some of it is likely not caught.
Edit: More information on Proctor here: https://www.chess.com/proctor
mft_•19m ago
Interesting; I thought I'd read that even the very best players only average ~90% accuracy, whereas the best engines average 99.something%?
janalsncm•10m ago
tomku•10m ago
kmike84•2m ago
To compute accuracy, you compare the moves which are made during the game with the best moves suggested by the engine. So, the engine will evaluate itself 100%, given its settings are the same during game and during evaluation.
You get 99.9something% when you evaluate one strong engine by using another strong engine (they're mostly aligned, but may disagree in small details), or when the engine configuration during the evaluation is different from the configuration used in a game (e.g. engine is given more time to think).
fsckboy•1m ago
vunderba•18m ago
Cheating is as simple as having somebody feed you chess engine moves from a nearby laptop running stockfish.
[1] https://en.wikipedia.org/wiki/The_hand_of_God
rayng•12m ago
"... But had I started cheating in a clever manner, I am convinced no one would notice. I would've just needed to cheat one or two times during the match, and I would not even need to be given moves, just the answer on which was way better. Or, here there is a possibility of winning and here you need to be more careful. That is all I would need in order to be almost invicible."
https://www.youtube.com/watch?v=VcbHmHHwlUQ&t=313s