it could also be gamma rays or a variety of problems that seem to appear and disappear between chairs and keyboards.

memes are jokes. people taking jokes as something other is the problem.

A lot of the time it's cabling.

Resolver limitations, as opposed to server or protocol issues, are in my view the main reason why "it is always DNS".

This is a beautifully designed page.

Well sure... it could be BGP

No, sometimes it is just Spanish football as for everything behind Cloudflare. Which is the case for this blog being blocked right now and redirecting to another page:

"El acceso a la presente dirección IP ha sido bloqueado en cumplimiento de lo dispuesto en la Sentencia de 18 de diciembre de 2024, dictada por el Juzgado de lo Mercantil nº 6 de Barcelona en el marco del procedimiento ordinario (Materia mercantil art. 249.1.4)-1005/2024-H instado por la Liga Nacional de Fútbol Profesional y por Telefónica Audiovisual Digital, S.L.U. https://www.laliga.com/noticias/nota-informativa-en-relacion..."

Paul Tagliamonte sounds like a nice guy who has thought about these issues at length. He's reached the second level of DNS enlightenment: "There's no way it's DNS".

Finality will arrive, and Paul will internalize the knowledge.

Nope, the other times it's CORS

> but it is not the operational hazard it’s made out to be

Until you flip that DNSSEC toggle

> a DNSSEC rollout bricking prod for hours

He links to the Slack incident. But that problem wasn’t caused by a DNSSEC rollout; the problem was entirely caused by a completely botched attempt to back out of DNSSEC, by doing it the worst way possible.

Tell that to AWS East 1

I had the CEO and CTO of our ccTLD registry give a guest lecture to my CS students, and one question came up regarding the AWS incident.

Prior to the question, the CEO boasted a 100% uptime (not just five nines), and the CTO said “We’re basically 30 people maintaining a 1GB text file.”

So the question was, “How come 30 people can have 100% uptime, and the biggest cloud with all of its expertise can’t? Sure, it was DNS, but are you even doing the same thing?”

And the answer was, (paraphrasing) “No, what we do is simple. They use DNS to solve all sorts of distributed problems.”

As did the CTO with all of these new record types embedding authentication. But running CoreDNS in a Kubernetes megacluster is not “maintaining a 1GB text file”.

The full maxim I was taught being, “it’s either DNS or permissions”.

The fatal design flaw for the Domain Name System was failure to learn from SCSI, viz. that it should always be possible to sacrifice a goat to whatever gods are necessary to receive a blessing of stability. It hardly remains to observe that animal sacrifice is non-normative for IETF standards-track documents and the consequences for distributed systems everywhere are plainly evident.

Goats notwithstanding, I think it is splitting hairs to suggest that the phrase “it’s always DNS” is erroneously reductive, merely because it does not explicitly convey that an adjacent control-plane mechanism updating the records may also be implicated. I don’t believe this aphorism drives a misconception that DNS itself is an inherently unreliable design. We’re not laughing it off to the extent of terminating further investigation, root-cause analysis, or subsequent reliability and consistency improvement.

More constructively, also observe that the industry standard joke book has another one covering us for this circumstance, viz. “There are only two hard problems in distributed systems: 2. Exactly-once delivery 1. Guaranteed order of processing 2. Exactly-once delivery”