Personally it happened to myself as well. I regularly rent a Tesla and once, I took a date on a trip and I drove us through the city, her riding shotgun. She said: "Look at the orange line on the screen. You're driving too close to the parked cars on our right".
I answered, "it always does that, the proximity sensor on these Teslas is way too nervous". She looked out of the window and said: "no, you are actually way too close to the parked cars!"
I had totally normalized the proximity warning.
This means, IMO, that any organisation with a healthy culture will also have a means to review and remove alarms and procedures if they are found to not be worthwhile. This substantially increases the chances the rest of them are respected and followed.
But…
> Humans are bad at reasoning about how failures cascade, so we implement bright line rules about when it's safe to deploy.
I think aggregate human intuition is often undervalued. It is the case that every bright line rule has a cost, and the total cost of its adherence must be weighed against the occasional cost of failure to adhere.
Benefits don’t exist in a vacuum.
Microsoft's security orientation must have peaked before this article (2015), and the culture slid back, because I see a lot of folks bagging on Microsoft security right now. If true, deviance was normalized at Microsoft, de-normalized, and the re-normalized.
For example, Azure offers Microsoft software with various proprietary protocols exposed to the Internet that would have been unthinkable for any competent administrator a decade ago. This includes the SMB file sharing protocol and the SQL Server TDS network protocol.
It's bizarre to me to see a file share and a SQL database just "on" the Internet, no firewall or anything.
From my experience the second part is somewhat true. I have yet to see a "data driven" decision that wasn't actually driven by a very political process of choosing what data to gather and how to interpret the results.
(This obviously doesn't mean you should ignore data and focus on politics. Focus on making the politics good so that data can be properly used.)
Good business is about hedging your bets. It's not about creating business processes that always make the right bets.
It's also not the case that good business is the only way to survive. Which is one of my favorite stories from FedEx's beginnings. They were short on cash and couldn't make fuel payments or payroll the next day. The CEO extract cash from the business, took it to Las Vegas, increased his holdings by gambling, then returned the money to the business the next day.
If it worked, then great, you "saved" the business. If it didn't, then bummer, you're now a felonious embezzler.
It’s even worse when you don’t have direct knowledge about what precisely it is that leadership wants. I’ve had multiple situations where I’ve worked hard on some analysis only to be told, when the line goes the wrong way, that the results “don’t look right” and that I should try something else.
A corollary to this is that I can not recall as single time when leadership reversed their initial decision/intuition when shown data that contradicts it.
> He acknowledged that my way reduced the chance of failure without making the technical consequences of failure worse, but it was more important that we not be embarrassed. Now that I've been working for a decade, I have a better understanding of how and why people play this game, but I still find it absurd.
If OP's embarrassment comment and the topic of normalization of deviance interest you then you might find this soft (Social) Science Fiction short story to be amusingly enlightening...
"The trouble with you Earth people" by Katherine MacLean (1968)
https://www.google.com/books/edition/The_Trouble_With_You_Ea...
^ link is to google books and their preview includes the entirety of the titular short story from the collection.
If ^ that short story is tl;dnr for you, Spoiler Alert:
Well meaning Alien POV discovery that Humankind is a self important and superstitious lot, and not mostly harmless.
dang•5h ago
Related. Others?
Normalization of Deviance (2015) - https://news.ycombinator.com/item?id=34791106 - Feb 2023 (219 comments)
Normalization of Deviance (2015) - https://news.ycombinator.com/item?id=22144330 - Jan 2020 (43 comments)
Normalization of deviance in software: broken practices become standard (2015) - https://news.ycombinator.com/item?id=15835870 - Dec 2017 (27 comments)
How Completely Messed Up Practices Become Normal - https://news.ycombinator.com/item?id=10811822 - Dec 2015 (252 comments)
What We Can Learn From Aviation, Civil Engineering, Other Safety-critical Fields - https://news.ycombinator.com/item?id=10806063 - Dec 2015 (3 comments)