NTP at NIST Boulder Has Lost Power
As it stands at the minute, the clocks are a mere 5 microseconds out and will slowly get better over time. This isn't even in the error measurement range and so they know it's not going to have a major effect on anything.
When the event started and they lost power and access to the site, they also lost their management access to the clocks as well. At this point they don't know how wrong the clocks are, or how more wrong they're going to get.
If someone restores power to the campus, the clocks are going to be online (all the switches and routers connecting them to the internet suddenly boot up), before they've had a chance to get admin control back. If something happened when they were offline and the clocks drifted significantly, then when they came online half the world might decide to believe them and suddenly step change to follow them. This could cause absolute havoc.
Potentially safer to scram something than have it come back online in an unknown state, especially if (lots of) other things are are going to react to it.
In the last NIST post, someone linked to The Time Rift of 2100: How We lost the Future --- and Gained the Past. It's a short story that highlights some of the dangers of fractured time in a world that uses high precision timing to let things talk to each other: https://tech.slashdot.org/comments.pl?sid=7132077&cid=493082...
To put a deviation of a few microseconds in context, the NIST time scale usually performs about five thousand times better than this at the nanosecond scale by composing a special statistical average of many clocks. Such precision is important for scientific applications, telecommunications, critical infrastructure, and integrity monitoring of positioning systems. But this precision is not achievable with time transfer over the public Internet; uncertainties on the order of 1 millisecond (one thousandth of one second) are more typical due to asymmetry and fluctuations in packet delay.
[1] https://groups.google.com/a/list.nist.gov/g/internet-time-se...
How do those other applications obtain the precise value they need without encountering the Internet issue?
Alternate sources include the GPS signal, and the WWVB radio signal, which has a 60kHz carrier wave accurate to less than 1 part in 10^12.
edit: also the linked slides in TFA
Precision Time Protocol gets you sub-microsecond.
We actually disable NTP entirely (run it once per day or at boot) to avoid clocks jumping while recording data.
This doesn't seem right to me. NTP with default settings should be monotonic. So no jumps. If you disable it Linux enters 11-minute mode, IIRC, and that may not be monotonic.
Perhaps a bit more boring than one might assume :).
qmr•3h ago
Suggestions from the community for more reliable alternatives?
evanriley•1h ago
You still can...
If you're that considered about 5 microseconds: Build your own Stratum 1 time server https://github.com/geerlingguy/time-pi
or just use ntppool https://www.ntppool.org/en/
eddyg•1h ago
For example: unlike the IPv4 space, the IPv6 space is too big too scan, so a number of "researchers" (if you want to call them that) put v6-capable NTP servers in the NTP pool to gather information about active v6 blocks to scan/target.
ticoombs•32m ago
edoceo•28m ago
monster_truck•1h ago
ajkjk•1h ago
ianburrell•1h ago
Also, you can use multiple NIST servers. They have ones in Fort Collins, CO and Gaithersburg, MD. Most places shouldn't use NIST directly but Stratum 1 name servers.
Finally, NTP isn't accurate enough, 10-100 ms, for microsecond error to matter.
vel0city•7m ago
For instance, time-a-wwv.nist.gov.
One should configure a number of different NTP sources instead of just a single host.