frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

How to Build a Custom Mechanical Keyboard

https://www.alexotos.com/how-to-build-a-mechanical-keyboard/
1•pbardea•2m ago•0 comments

Show HN: SafeRate – AI chat-native mortgage lender

https://saferate.com/
2•dmhall23•3m ago•0 comments

Figma stock soars 230% in first day of trading, valuing company north of $40B

https://finance.yahoo.com/news/figma-stock-soars-230-in-first-day-of-trading-valuing-company-north-of-40-billion-132215034.html
1•sgerenser•3m ago•0 comments

Chasing Riskless Profits with Triangular Arbitrage Bot on Binance

https://shufflingbytes.com/posts/binance-triangular-arbitrage/
1•ValtteriL•5m ago•0 comments

My Heart of Hearts

https://www.astralcodexten.com/p/my-heart-of-hearts
1•Michelangelo11•5m ago•0 comments

What Can a Cell Remember?

https://www.quantamagazine.org/what-can-a-cell-remember-20250730/
2•chapulin•6m ago•0 comments

Sweeney confirms Epic Games Store launch on Play Store after Google loses appeal

https://www.neowin.net/news/fortnite-maker-confirms-epic-games-store-launch-on-play-store-after-google-loses-appeal/
2•bundie•7m ago•0 comments

Sendria is a test SMTP server

https://github.com/msztolcman/sendria
1•porridgeraisin•8m ago•0 comments

New ultrasound imaging to map drug delivery into the brain

https://medicalxpress.com/news/2025-07-ultrasound-imaging-drug-delivery-brain.html
1•PaulHoule•9m ago•0 comments

From Elvis to "Men in Black," the Story of the Hamilton Ventura

https://wornandwound.com/from-elvis-to-men-in-black-the-story-of-the-hamilton-ventura/
2•notagoodidea•10m ago•0 comments

Andrew Tate's "The Real World" is stealing my software (2022)

https://insrt.uk/post/andrew-tate-stealing-software-revolt/
2•boramalper•11m ago•0 comments

Show HN: DJ-style audio waveform thumbnailer in Python

https://github.com/Alzy/Xochi-Thumbnailer
2•Alfredo101•11m ago•0 comments

The missing trust model in AI tools

https://docs.freestyle.sh/blog/missing-trust-model
1•benswerd•11m ago•0 comments

Orch.space – A visual builder for AI reasoning workflows (BYOK)

https://www.orch.space/
1•steomb•12m ago•0 comments

Changes in Inflation by City

https://wallethub.com/edu/cities-inflation/107537
2•josephjrobison•15m ago•0 comments

Model Context Protocol, Product Demos, and the New App Store

https://wjgilmore.com/articles/model-context-protocol-product-demos-and-the-new-app-store
2•wjgilmore•17m ago•0 comments

Robotaxis are powered by human drivers as it launches ride-hailing in Bay Area

https://electrek.co/2025/07/31/teslas-robotaxis-officially-powered-by-human-drivers-as-it-launches-ride-hailing-bay-area/
2•TheAlchemist•18m ago•0 comments

Show HN: Microsoft research on impact of GenAI on jobs

https://www.youtube.com/watch?v=agceklIOXLU
1•mandarwagh•22m ago•0 comments

Intuitive explanation of LLM Transformers without math

https://www.youtube.com/watch?v=VlbBgj2lBls
1•i_dont_know_•23m ago•1 comments

Playwright users–wants design-partner to shape an AI tool that tames flaky tests

1•pratik-tgx•23m ago•0 comments

Our first outage from LLM-written code

https://sketch.dev/blog/our-first-outage-from-llm-written-code
3•mrw34•24m ago•2 comments

KDE Linux

https://kde.org/linux/
3•MBCook•25m ago•0 comments

Parsing Without ASTs and Optimizing with Sea of Nodes [video]

https://www.youtube.com/watch?v=NxiKlnUtyio
2•adamrezich•25m ago•0 comments

The Practicals of Writing: Paper and Pens

https://brianschrader.com/archive/the-practicals-of-writing-paper-and-pens/
3•sonicrocketman•27m ago•0 comments

What Is Space-Time? Einstein's Theory of Time and Gravity Explained

https://www.discovermagazine.com/the-sciences/what-is-space-time-einsteins-theory-of-time-and-gravity-explained
2•Bluestein•27m ago•0 comments

Released a lsp server for game maker language

https://github.com/Ralticks/gml_lsp
2•Okerew•31m ago•0 comments

Tesla hits a speed bump in its California 'Robotaxi' rollout: Permits

https://www.politico.com/news/2025/07/30/tesla-robotaxi-permit-problems-california-00486269
4•JumpCrisscross•31m ago•0 comments

Augment Code Brings Its Coding Agent to the Terminal

https://thenewstack.io/augment-code-brings-its-coding-agent-to-the-terminal/
1•gailaxelrod•32m ago•0 comments

In-Network Leaderless Replication for Distributed Data Stores [pdf]

https://www.vldb.org/pvldb/vol15/p1337-lee.pdf
2•todsacerdoti•32m ago•0 comments

Understanding Blockchains Through Java Code

https://blog.blockingqueue.com/understanding-blockchains-through-java-code
3•liviu31•34m ago•0 comments
Open in hackernews

You know what: Microsoft became miserably incompetent in IT

https://mikekaganski.wordpress.com/2025/07/25/microsoft-anybody-home/
199•doener•18h ago

Comments

gnabgib•18h ago
Far from the title: Microsoft, anybody home?
WarOnPrivacy•15h ago
But a dead ringer for the subtitle.

And most importantly, the rightest title for an article about the user-hating, CoPervy, chaos-hydra that is Microsoft.

avazhi•14h ago
I think the current title used by OP actually conveys the gist of the article the best.
gnabgib•14h ago
Perhaps, but HN guidelines request the original title without editorialising.

https://news.ycombinator.com/newsguidelines.html

altairprime•14h ago
Incorrect.

> please use the original title, unless it is misleading or linkbait; don't editorialize

The author’s title is significantly lower signal than the first sentence of their post. When the original title is defective in such a manner, the site accepts alternate titles, with preference given towards those that use the words of the post linked to compose a more informative title. The submitter does so here: “Microsoft, anyone home?” could be generically applied to hundreds of different Microsoft issues over time; the selected quote is much more precise. Plus informative, minus rhetoric is an excellent generalization of what makes a good HN title edit.

minebreaker•14h ago
I hope the guideline is clearer. "Unless it is misleading or linkbait; don't editorialize." is too ambiguous.
PaulHoule•17h ago
Hey, Copilot is doing Satya Nadella's job when it isn't summarizing podcasts for him.
GianFabien•16h ago
Never mind that ... look at next quarter's profit numbers and dividend. Like all corporations the only cohort they care about are their shareholders and executive suite.
bee_rider•14h ago
It is really shocking that their revenue keeps going up. I guess Azure makes sense.
burnt-resistor•14h ago
Munching up VMware on-prem and AWS/GCP cloud, especially when they support Linux and Windows, it makes it less risky for megacorps to shift traditional non-webops, on-prem operations to them.

What FOSS needs is saner and simpler type-1 hypervisor storage and compute appliances because the existing solutions are too difficult to deploy and maintain.

lightbritefight•13h ago
Look into XCP-NG. Its an excellent implementation of xenserver with a vsphere style single pain of glass called xen-orchestra. Its all FOSS and buildable from source.

It comes "batteries included" with backups/vsan/etc for an insanely reasonable "per host" license cost for support.

devjab•13h ago
Microsoft is still the best IT business partner for non-tech enterprise organisations. The sort of support you get as a tiny company or a private customer, like the stuff from the article, is similar to what you get from a lot of other tech companies. You get to talk with an AI if you're lucky enough that it's not just an automated system. Step into enterprise where Microsoft will call you when something goes wrong. Sure they've moved from old-scool product licenses to Azure, and their partner program is less "outsourced". The platform is still selling products that make it very easy to be an IT operations unit where your IT related corporate titles can tell the rest of the organisation that they are in constant talks with Microsoft who is working hard to resolve the issue.

It's an old tale, but Azure was sort of late to enter the market here in Europe compared to AWS (GCP was never really an option for a lot of sectors due to Google's inability to sell actual products without Big Brother). Anyway, in the early days AWS had terrible support, it was basically like trying to contact Google as a private customer even if you were a major company. Then came Azure and blew it's way into the enterprise sector, partly because Microsoft was already there but also because Microsoft did what they always do, and actually give you people to talk with. 3 months later we had an AWS representative calling us with a similar experience on sale. Google never adopted anything like it, but today AWS is frankly a great experience for enterprise as well.

The last part Microsoft has is that they bundle things. Teams didn't win because it was the best at what it did when it arrived. It won because nobody in non-tech enterprise would want to pay for a Teams alternative when they were already paying for Teams anyway. In Enterprise you don't pay the listing prices that you or I would if we were to buy office365, windows and some Azure services. You instead buy things through a partner program, and then some of the licenses are different than they otherwise would be.

There is no real alternative to it. All of Europe is looking into moving away from USA tech giants now. You can absolutely replace the Office365 platform with something like Open Deks, but you can't do it without having IT operations staff who can do a lot of the things Microsoft would typically do for you. Similarily you can absolutely run things on Hetzner or OVH, but not as easily as you could if you were doing the Azure vendor lock-in.

If it wasn't for the current geo-political situation, Microsoft would continue it's monopoly on non-tech organisations. For a good while anyway. Because ironically one of their strengths was familiarity. It used to be that when you hired someone, they knew Microsoft Office and they knew Windows, but they've lost that edge. Maybe because this blog entry is spot on as far as the private customer experience goes. Today the most frequently asked question from our new employees as far as IT goes is "can I get a mac?".

ammo1662•13h ago
They laid off a lot of people this year, so you can imagine how good the profits and financial reports must look.

No matter the cost, regardless of whether the company might fail, the executives and shareholders can always sell their shares early to make a huge profit and just walk away.

CCs•15h ago
LinkedIn (also MS) has the same problem.

Also: minimum age to sign up is 16. But you need to be 18 to verify your account. And if you’re locked out, you need to verify your account and log in to open a ticket saying that you can’t log in…

donatj•10h ago
I have a terrible fear that someday eBay is going to do the math on my 30 year old eBay account, figure out I was 9 when I opened it and lock me out.
charcircuit•14h ago
>OK, they have a stupid AI that is worse than good old filters.

This is an unsubstantiated claim. Right after he said he prefers to provide references for what he says.

zo1•13h ago
Except he's right. If you haven't noticed how crappy "AI" search is in Outlook and Teams then you probably don't use them. Even when it sorta works, it literally ignores search terms you put in the field, it gives you the search results first for what it thinks you want, then after that it gives you the actual results which are at least somehow using your search terms.

At this point they'd be better off throwing my entire inbox in a RAG workflow written by some junior halfway across the world. It's that bad. I find stuff quicker by remembering when I got an email and then scanning it myself with my own eyeballs and my finger on the scroll wheel.

Outlook used to be amazing. It used to be the hub to my work knowledge base and my memory. Now it's just the tool that does my emails mostly very well.

mikekaganski•13h ago
Are you sure that a text describing one's thoughts when facing something has the same requirements as a mail to a mailing list? And do you always think using quotations in your mind? Like "Oh I come across this interesting place; it's rather popular [quotation1] [qotation2]; my friend suggested me to visit it [a link to the friend's spoken advise]; etc."?
charcircuit•13h ago
It's not presented as a thought. This is an article criticizing Microsoft, and criticizes their antispam measures saying they are bad due to an anecdotal false positive without considering that the overall effectiveness may be better than older approaches.
mikekaganski•13h ago
No you obviously didn't read it. It writes exactly "OK, they have a stupid AI that is worse than good old filters. OK, they made it react immediately, as an undoubted authority. But that’s not a big problem, right? They provide a way to appeal! Let me do that."

If you say that it's not a depiction of what someone thinks and does facing something, you just need to improve reading skills.

And basically that wasn't something important in the article; I made it rather clear. Their following catch22 was the most likable thing IMO; that was what I explicitly emphasized.

I wonder why do I write all that, though. If someone just wants to find something irrelevant, and build their criticism on that, there's no way to prevent it.

charcircuit•12h ago
It was an unnecessary comment to include in the article in my opinion. It doesn't affect the overall story, is potentially libelous, and trivializes the complexity of fighting abuse at scale.

I agree it's irrelevant. It makes 0 sense to me to include unless it's backed in truth.

mikekaganski•12h ago
Irrelevant is "lack of references when one describes some reactions", not "this is what I thought, when I faced that; and I described my thoughts and feelings in a post of mine in my own blog dedicated to my thoughts and feelings".
charcircuit•12h ago
Fair. I could have raised skepticism of the claim without including the reference to references. But I wanted to make my comment longer, and the bit about references was mentioned the sentence before the sentence before the claim.

I also now realize that you are likely the author of the article that was posted, so I will wish you luck in resolving your account issues.

supportengineer•14h ago
Microsoft’s peak was Windows 7. It’s all been downhill after that.
burnt-resistor•14h ago
They had multiple peaks. DOS 6.22, Windows 2000, Windows XP/2K3 (kinda), and 7.

They're trying hard to make 11 rhyme with the histories of Vista and Bob.

(I only use Windows 11 Enterprise on my oscilloscope control and instrumentation logging lab box, and on a gaming-ish laptop with an eGPU.)

linguae•13h ago
Peak Microsoft to me was 1999-2000. Office 2000 (though Office 97 was also excellent), Windows 2000, and Visual Basic 6.0 were great.

In my opinion, things started going downhill with Windows XP, with the introduction of product activation and the later introduction of forced software updates in Service Pack 2. I also don’t like the UI changes that Microsoft made in the 2000s, notably the Luna theme in Windows XP and the introduction of the ribbon in Office 2007. I still don’t like the ribbon nearly 20 years after its introduction.

My take is that 1990s Microsoft software tended to get out of my way. From the 2000s onward, it seems that Microsoft software often gets in my way. I found a refuge in Mac OS X, but in the past decade macOS has gotten more annoying.

aucisson_masque•12h ago
> I still don’t like the ribbon nearly 20 years after its introduction.

> I found a refuge in Mac OS X, but in the past decade macOS has gotten more annoying.

Agree on both, macOS is still nowhere near windows level of annoyance but it’s getting worst and worst.

Each feature well integrated that made me like macOS was implemented at least 10 years ago, the recent feature are neat but don’t feel right. Like the window snapping, if you drag it to the top it will expand ‘almost’ fully but then if you didn’t stop at the right spot and kept dragging to the top your window goes instead to the virtual desktop screen. It’s extremely counterintuitive, and it keeps repeating with all these new features. Who use stage manager on macOS for instance ? Makes absolutely no sense.

At some point it won’t be worth the premium price they are asking for and I guess I’ll go Linux desktop. It’s broken, counter intuitive, little support but at least it’s getting better with time, not worst.

bigstrat2003•13h ago
> They're trying hard to make 11 rhyme with the histories of Vista and Bob.

Windows 11 wishes it was half as good as Vista. While much-maligned, Vista was decent enough. And it didn't show you ads, try to shove MS services down your throat, or any of the other user-hostile things 11 does.

pjmlp•12h ago
VB 6, nothing in .NET or the multiple C++ frameworks, has been as easy to use for COM.

MFC, although not as good as OWL/VCL/Qt, they never produced anything better in house for C++ developers.

herbst•11h ago
I don't see anything peeking after 95 when they invented the UX we still all kinda stuck with.
burnt-resistor•14h ago
Well, that's actually 3 failures: UX designed to create an impossible flow, AI-based user blocking, and a lack of customer service.

By comparison, something similar happened when I was hired by Meta. Visiting the 2FA page caused my account to be insta-locked. It turned out that I had another account under an email address domain I no longer had access to. They were eventually able to fix it with an Oops request.

In conclusion, it doesn't matter so much if tech is imperfect if the support is good. But without good support, any small inconvenience or issue can easily spiral to become a show stopper.

guappa•13h ago
Support at meta was only good because you worked there.
danlitt•12h ago
Seconded. I got banned by Facebook back in 2015, for no reason. I say no reason because they never told me a reason, even after following up to ask. And when I immediately created the same account all over again, same name, same photo, it stayed up & has for 10 years. Shrug.
herbst•12h ago
I have that with all meta products. As soon as I sign up my account gets locked, then I can unlock it normally most of the time. After a few hours or any interaction it gets locked again. When I usually start to see broken UX where there is no way to proceed anymore.
ulrikrasmussen•14h ago
And this is one of the (many) reasons that I have self-hosted my email for the last 20 years. Sure, I have had outages, and one or two times I have had to drop everything to fix the server due to a botched upgrade. But each time this happened it was my own fault and I myself was able to fix it. I have NEVER lost access due to some automated system putting me in the wrong category.

By the way, the biggest griefs I have had with self-hosting my email has been due to Microsoft. Their way of categorizing spam can pretty much be summed up to: does it come from an explicitly whitelisted commercial email provider? Probably not spam, otherwise: spam. They are criminally incompetent (also) in this area.

iforgotpassword•13h ago
Yep, on the rare occasion I deal with someone who uses Microsoft for email, I give them a heads-up that they probably miss an email every now and then. I remember my last employer's mails often didn't reach them, and at some point they added a big warning to their sign up page when you used one of the Microsoft domains for your email.
vachina•13h ago
Or don’t use Outlook (at least the retail version) for anything too important.
pityJuke•8h ago
Owning your own domain is good enough for me - I use Apple to host my mail, but should they suspend me I’d at least be able to move the domain to another provider.

I should consider a solution for backing up my email, though.

SSLy•7h ago
imapsync
kabes•14h ago
Also if you have multiple microsoft 365 accounts, switching between those in the webapp seems to be impossible for years already. There's a switch account option, you can click it and sign in to another account, except you just stay logged in as the previous account.
laserbeam•14h ago
Oh yeah, I can only actually use them via private browsing (or nuking cookies, of course)
jiggawatts•13h ago
Firefox "Containers" are great for this. I have color-coded tabs for various customer work accounts / tenants.
laurent_du•13h ago
Yes, they are a life-saver. I use them all the time for various AWS accounts. The guy who told me about it years ago has saved me countless hours.
grammarxcore•9h ago
AWS finally added the ability to swap between a few accounts [1]. There’s an arbitrary limit of 5 so it’s really bad if, say, you work for an enterprise and have lots of accounts or, say, you work for a smaller business following the AWS Well-Architected Framework and isolate things. Containers still win.

1: https://docs.aws.amazon.com/awsconsolehelpdocs/latest/gsg/mu...

guappa•13h ago
I am not sure… a microsoft login normally seems to jump across tens of different domains and my firefox containers get confused and it often fails if the stars are not well aligned.
chrismorgan•14h ago
This kind of issue is absurdly common. I’ve seen basically the same thing with Microsoft, Amazon and Google—personally and with others.

With Google, it’s well-documented that sometimes, even if you don’t have 2FA turned on and you know your password, they simply won’t let you log in without some sort of additional verification… which may not be possible. This hit me once while overseas, I think I managed to get in by tunnelling through my VPS in Australia. And if you can’t do this, you’re stuck, because Google simply doesn’t have support.

With Amazon, I was finally able to find one way of contacting them that didn’t require signing in, and when I finally got to a human, the problem was immediately fixed.

wat10000•13h ago
These businesses are all about scale. Each user makes them a paltry amount of money, but they have an enormous number of users.

A single support incident that reaches a paid worker on their side costs them more than the profit you’ve brought them for the year. It probably costs them more than the profit you’ll bring them for your entire life.

With less scale-y businesses, it’s worth it for them to put some effort into fixing problems, as an investment in your future business. Here, the only reason would be to avoid reputational damage. And who’s going to stop using Google or Microsoft because some guy had trouble getting support?

There’s a story (no doubt apocryphal) about Bill Gates telling people working on Windows that a customer calling their support line cost as much as the profit they made selling that copy of Windows. The point was to make it so Windows users didn’t need to call support by making Windows work better. The modern equivalent of this would be to make sure that users can’t reach expensive support in the first place.

_rm•13h ago
I feel like once they've gotten everyone on the planet using their services, and then fucked them all over, maybe then there'll be a market un-fucking them over.
danielscrubs•13h ago
With enough eyeballs all bugs are shallow.

What would be the opposite?

With enough blindfolds all bugs are unimportant?

aucisson_masque•12h ago
If business are all about scale then you are not fixing one user problem but you are fixing the same problem experienced by thousands or millions of user.

There is no excuse for being lazy.

GoblinSlayer•13h ago
>even if you don’t have 2FA turned on

Presumably the excuse is that it happens because you don't use 2fa and your ip changed, but I heard 2fa might not save you either.

random3•13h ago
Exactly! The OP’s surprisal is surprising, given that 95% of all support interactions across the board are exactly like this. In fact, the internal support - i.e. the support for employees is just as bad and nonsensical.

It goes pretty much like this. If your problem needs “reasoning” that entails more than matching a direct solution, you’re getting the default answer that’s the closest match to the solution.

The only difference between live support and support docs is that live support has access to some of your data and controls over some flows (e.g. reset password)

Now the real kick - it’s like this across the board outside of tech. If you go to a hospital with a problem that is not the 95 or 99 or whatever they are “tuned” to deal with, your pretty much fucked, unless you can be resourceful or luck enough to navigate the structure until you find someone that actually knows how to.

Bureaucracies? Same.

It’s effectively the same dumb machinery with dumb human cogs that you’re always dealing with.

I think it’s simply the “limit” of average human intelligence and find it hard to believe AI will end up being dumber than this.

mikekaganski•13h ago
> The OP’s surprisal is surprising, given that 95% of all support interactions across the board are exactly like this

If your point is "it is today's norm, let's accept it and shut up", I feel the opposite. If the "norm" is like that, expressing it aloud is my way of contributing to society.

carlosjobim•12h ago
That is clearly not his point.
sam_lowry_•12h ago
Nexus by Yuval Harari covers bureaucracy in the same vein.
tarsinge•11h ago
Bureaucracy didn’t emerge in the 21th, contrary to lack of customer support for big companies. Bureaucracy is a symptom part of the enshittification, the root cause is Monopoly (direct or through organized shareholders across an industry). You don’t need support or good service when customers don’t have the choice to go elsewhere.
danaris•8h ago
I would even go a step further, and say that bureaucracy, in and of itself, is not a bad thing. It is a natural and inevitable consequence of a large organization: you need more measures in place to keep track of things when there are too many for people to keep track of in their heads.

The problem arises when the bureaucracy becomes, at least for portions of the organization, the end in itself rather than a means to an end (the end being something like better service, efficiency, and accountability). That's when it starts to contribute to frustrations and bad experiences (and, if allowed to continue in this vein, enshittification).

random3•7h ago
Yes. In fact I was referring to bureaucracies in general as a state’s “support” organization.
mcosta•7h ago
They can go elsewhere , but then they would have to pay. And users hate to pay when there is a free offer.
samrus•10h ago
Thats last part is either naive or just shilling.

The problems isnt the cogs, its the business owners setting up the incentives. Providing reasoned support is expensive from a time and effort perspective, so they dont invest in it. Service reps are specifically trained and incentivized to just fuzzy match the SOPs

And AI cant reason either, so it wont be able to provide any real reasoned support either. And the pwners dont care because they already provide shit service, people like you will just hype the AI up and then make excuses for it when its just as shit

random3•10h ago
Shilling what? AI in general? What’s the incentive? Hype is harder to argue. LLMs are defective attempts at AI, but that’s also a huge blind spot— effectively confounding AI and AI progress with a specific instance like LLMs.

Business incentive— if you can make an argument on why would a corporation provide shit support to its own employees you win. Also make an argument on why support is actually good at Amazon and others like them. So I don’t buy your argument on incentives either. Although there are other implications related to incentives and AI in general, but those are off topic.

ethbr1•10h ago
The part that seems to be missing these days, that was historically present in support orgs, is escalation.

You’re always going to do the bulk of support through L1, and L1 support is going to be limited in terms of training and access.

But! The intended flow is that L1 recognizes when an issue exceeds their ability and escalates it up to L2+. And that’s where your talented (and empowered!) support engineers work.

That escalation just doesn’t seem to happen any more.

My hypothesis is that it’s a consequence of L1 being outsourced, where now they’re heavily incentivized to avoid escalating and impacting their probably-contractual KPIs.

Yet another cost for adding too many parties to something that could be better if it were simpler.

dragonwriter•9h ago
> Yet another cost for adding too many parties to something that could be better if it were simpler.

But its not a cost from the perspective of the companies doj the outsourcing: the incentives are the way they are because the result is intended. Its not a cost of adding an extra party, it is part of the reason the party is added.

ethbr1•9h ago
I think you’re overestimating intentionality in bad outcomes.

It’s entirely possible for a whole organization to want a good outcome, but construct an org chart where bad outcomes are guaranteed by the misaligned and incomplete incentives for sub-divisions.

E.g. the IT support org that is only measured on total spend and time to ticket closure

lcnPylGDnU4H9OF•7h ago
> It’s entirely possible for a whole organization to ... construct an org chart

You changed actors mid-sentence. The workers in the organization may want good outcomes but they are near-invariably separate from the people making decisions for the organization.

ethbr1•4h ago
No, I said it exactly as I meant it.

The organization (including the bulk of employees and executive leadership) may want a good outcome, but sub-divisions’ (aka the org chart) may have have their self-interests set to inevitably produce a bad outcome.

esulteric•12h ago
Google wiped all my codes from their authenticator app

It really drove home that trusting my identity and online presence to these entities that do not care or even know how to be responsible about it is a very bad idea.

mkesper•12h ago
Always use an open source OTP app for something like that.
rstuart4133•11h ago
Nowhere near paranoid enough.

Always take the "manual" OTP option, take a backup copy of the code you copy and pasted. I use my password manager

Then, always export the contents of you entire password manager database, encrypt it with the same password you use for you password manager with the pdkf2 rounds set to an absurdly high number and place it in a public place on the internet. Lots of places will host it for free.

waste_monk•12h ago
It is true that google's (lack of) customer support is awful.

>it’s well-documented that sometimes, even if you don’t have 2FA turned on and you know your password, they simply won’t let you log in without some sort of additional verification

However, this isn't quite fair on google - it's not arbitrary "sometimes", it's that accounts get locked if there's been unusual or suspicious activity that may indicate an attempt to compromise the account, such as failed login attempts or logins from a new location. The account is locked from single-factor (password only) authentication for a period of time to prevent compromise (which isn't disclosed but as I understand can vary and is usually a week or two), but will allow login with an MFA challenge.

If the account is locked and there are no MFA methods set up or other means of recovery like backup passwords, how are they meant to verify that it's you recovering the account and not an attacker?

Supposing you did manage to talk to a live google support agent, you'd still have the same problem of not being able to verify ownership of the account.

danlitt•12h ago
I think this is quite fair criticism of google. The idea of the account being locked and requiring MFA to verify is something they made up without consulting the user. If they're going to do this "suspicious account" locking, they must make MFA mandatory beforehand. If they haven't got MFA from the user, they must trust the password. If that means my account gets compromised, that's my problem.

Note that there is no question who owns the account. That's what the password is for!

waste_monk•12h ago
> If they're going to do this "suspicious account" locking, they must make MFA mandatory beforehand.

This should probably be the case - I am not a big fan of passkeys [1] but hopefully this will help make non-SMS, phishing resistant MFA available to the masses.

Still, Google are in a difficult position where on the one hand they have people angry about being locked out, and yet if they didn't they would have people angry about their accounts being compromised.

> If that means my account gets compromised, that's my problem.

It is also the problem of everyone on your contacts list, the operators of other services which use Google OAuth for login, anyone who might recieve spam from your compromised account, and so on.

Reducing harm by temporarily restricting the account when heuristics indicate it may be compromised is not unreasonable IMO.

>Note that there is no question who owns the account. That's what the password is for!

This is unfortunately not as true as we would like it to be.

Have a look at the list of compromised services on Have I Been Pwned [2] - not all of these include passwords, and many that do would be hashed (and hopefully salted) rather than plaintext, but it still happens often enough to be of great concern.

[1] WebAuthN / Passkey technology is fine in the general sense, and I am quite happy to use it with a hardware security key (such a Yubikey or Titan) or emulated in a password manager.

However I think the recent push towards having it tied to a TPM on a device such as a phone is a bit short-sighted, simply because these devices are prone to being lost or damaged. If you set up passkey MFA on the phone and don't enrol a backup method, then lose your phone, you have also lost the account. Probably these services should require 2+ MFA methods configured.

[2] https://haveibeenpwned.com/PwnedWebsites

rstuart4133•11h ago
> The idea of the account being locked and requiring MFA to verify is something they made up without consulting the user.

I'd be surprised if that was true. They don't just consult me about MFA, they downright pester me about it.

Besides if you wait long enough Google unlock the account anyway (if it's due to suspicious logins). I know, because it happened to me. I don't know how long it took, all I know is when I happen to check a few months after I thought I had lost access to the account, I could login again.

Finally, it's not like they have a choice. They have to protect the reputation of their domains and servers. It may seem unfair that you cop it if there are security issues, but you got the account for free and you refused to use the numerous MFA options the provide. If someone nerfarious takes over an accout to distribute spam or malware there is a mess that costs time and money to clean up.

There is a real point to be made about how Google / Microsoft and friends message accounts. But it's hard to sympathise with complaints about free ones. Paid for ones, where they promised a certain level of service and then withdraw service without warning and without the ability to recover your data (as I think happened to to the ICC) is a different matter.

a012•12h ago
Oh man, I’m pulling my hair right now because Google _decided_ consistently that someone else has my password and locked them out. They’re so proud of themselves about it I can imagine. But in reality, I’m trying to login my Gmail account on my second iPhone which is next to my original Android that has this Gmail account installed. When I logged in on the iPhone they showed a code and asked me to pick the same code on the Android, I did follow but they insisted that it wasn’t enough and still locked me out.
doener•11h ago
The complete lack of any human customer service that is actually answering is the reason I‘ll never again trust my private eMails to be managed by Google.

https://t3n.de/news/google-e-mails-geld-604458/

kazinator•44m ago
The common thread is that all these tech giants concentrate on scaling their ability to deliver the product and generate revenue, making service for exceptional situations a lower priority, with solutions that do not scale.

While you are consuming product and paying, you are basically identical to everyone else doing that (except for your varying personal info). That can be scaled in straightforward ways.

If you have some problem that requires attention to your account, that doesn't scale well. Even if it could, it's not worth it to them; it's off the happy, revenue-generating path.

vhguru•14h ago
Became?
kristianp•13h ago
I've encountered something similar with Apple. I tried to create an Apple id on a reconditioned Mac. It said the machine had been used too many times to create an Appleid. In the process it seemed to flag my phone number so I was unable to create an account on another device. The two support chats just told me to try again with no change. The first chat took about 40 minutes, there was a 5 minute pause on every response from them. At least on the 2nd chat I was able to ask for a 2nd level support call who fixed the problem.
LeoPanthera•13h ago
This is a bit different. You didn't even have an account yet. You had no data to lose.

Getting locked out of your own data is a far more serious issue.

doughnutstracks•13h ago
I've also been experiencing issues with the Authenticator app and Outlook, and it seems many others online are facing similar problems [0][1].

It's genuinely disappointing that using the Microsoft Authenticator app is a mandatory requirement for work, especially since they introduced the tap the correct number thing.

[0] https://www.reddit.com/r/Outlook/comments/1m4wp7h/microsoft_...

[1] https://learn.microsoft.com/en-us/answers/questions/4376965/...

0x_rs•13h ago
It's well known some things will trip those flags, probably not what all of them are or why, but most of them inappropriately (e.g. rating IP trustworthiness, but also simple HTTP requests that look "odd"). It's also well understood you have little to no options available as contacting support, live human or not as it may be, is made intentionally opaque and difficult or completely impossible. They just don't care, there's no reason to when you're one of the many hundreds of millions using their service most likely at no cost, and it's not unique to Microsoft. It's not that they became incompetent (they are, objectively), they simply never cared about you.
TZubiri•13h ago
Enshittification exhibit 2773

Cases were the software developer is not being directly paid for their services will be dismissed to make way in the docket for such enshittification cases

skirge•13h ago
Competent staff is expensive, because competent staff knows their value.
sunshine-o•13h ago
It feels the author is talking about a bank or the government. That feeling you get when you are dealing with insanity.

But this is not new, they were already like this a long time ago.

jeisc•13h ago
I think that they just wanted your mobile phone number in order to do 2 factor logins and they don't know how to do it without breaking the day with their inept system messaging and inputs.
hofrogs•13h ago
They rejected the number for unblocking purposes but I'm damn sure they saved it somewhere as a data point for that account to sell/use later.
eviks•13h ago
> But this, once an IT tech leader, became utterly incompetent in IT. And for me, it’s a pity.

The rose tint is strong in these glasses. When was their support great? Or is the author simply confused about what tech they were a lead in?

nicman23•13h ago
wasnt it in the vista days that ms went "let them eat forum" and basically yeeted all of their support >?
bryanrasmussen•13h ago
I bet support is probably pretty difficult to do well especially when you have lots of different products and a large international customer base. So I am quite willing to accept the idea that MS support was at one time the best one could get, for those particular very rare combination of problems, while also being apt to frustrate people using that support.

All that said - do you have a better example of support for a company with similar to MS needs?

rincebrain•12h ago
I have a lot of strong opinions about Microsoft, but at least during the Vista era, I once called their support without a support contract because someone's machine was very weirdly fucked after trying to upgrade to Vista, and I spoke to someone who spent an hour on the phone digging through weird registry crap and known issues to figure out how to successfully get the machine upgraded and working again. I was pleasantly surprised, having expected crappy support.

So I can confirm they did at least at one point have good support, and more broadly, I think it's not unfair to describe them as having had a great handle on what people wanted in user experiences and support, with how much they bent over backwards to fix compatibility issues for many years. I've also been told their support if you had a support contract was great, but I never was in a position to explore that firsthand.

Those days are, of course, gone, but they were known for that for a while.

II2II•12h ago
I can't speak for Microsoft since I've never dealt with them directly, but descriptions of the level of support offered to consumers in the eighties and early nineties would be beyond belief to those who didn't live through it. You would dial a toll-free number, talk to a real person who would diagnose the problem and fix it. If it was a known fixed bug, they would mail you a software update. If the bug didn't have a fix, there was still a chance it would be fixed in the current version. At least my experience with Corel, IBM, and, WordPerfect were all like that. How the companies could afford that level of support is beyond me.
aucisson_masque•12h ago
> How the companies could afford that level of support is beyond me.

They weren’t squeezing everything they can to get bigger profit to the shareholders ?

It’s not like Microsoft is barely surviving atm, they make excellent profit and could very well spend some on support but decided not to at some point.

pas•10h ago
Software was less complicated. Not that many layers, not that many supported platforms/configurations, etc.

Networking was simpler, if any.

And in general, pushing updates was not near zero-cost like today. So there was a bit of QA.

_DeadFred_•1h ago
In the 90s/2000s their Microsoft SQL Server support was amazing. They would go through traces with you and explain everything going on, what different techniques could be applied, performance considerations, etc from the highest level down to disk access. Their SQL support literally changed my skills to a point it changed my career. They could have recorded those calls and sold access they were so good and explained things so well.
GMoromisato•13h ago
I am disappointed that none of you are hating on Microsoft Teams. Maybe that's just too easy.
rf15•13h ago
Teams is such an ever-degrading experience it's now part of the indistinguishable cosmic background hate.
shim__•13h ago
I always make sure to provide an Jitsi room when inviting external people to an meeting to make sure they don't have an excuse to send me an teams link.
sexy_seedbox•7h ago
I deal with Microsoft Teams' embarrassing bugs on a weekly basis. Just yesterday discovered a bug with cropped / chopped off sentences on mobile version. But that's just one piece of M$ software.

I also have to deal with ScarePoint, MS Defender quarantining obviously safe messages, Outlook syncing problems, Windows 11 Account issues, many more...

kalleboo•13h ago
All of the tech giants have really broken account recovery flows. They've all been glomming on recovery options, single sign-on, 2FA, etc features and none of it is coherent, with lots of dead ends where the optional recovery option it suggests isn't actually supported.

Amazon has a lot of issues with accounts shared between their national storefronts. I lost a Google account that owns a YouTube channel since it wants to 2FA to a long-gone phone number. Apple has a lot of oddities when you used different emails for your Apple ID, iTunes Connect, and iTunes before they unified everything.

One great example though is Facebook

My wife's Facebook account recently got hacked, and I managed to recover it though this crazy workflow:

The hacker had removed her email address and phone number from the account, changed her password, and added their controlled Meta account as a connected account. This connected account had an email but no password, so it could not be removed without adding a password to it, which required verifying the attackers email address.

None of the account recovery tools worked (including the “this wasn’t me” link in the Facebook “did you just delete your phone number” email - what is the point of that link) - they couldn’t find her account by email or phone number, and even though the Facebook app itself was still logged in, none of the account center tools allowed us to do anything without the new password. It also did not allow us to remove the connection to the hackers Meta account or log it out from their devices because it had no password and it would become orphaned with no login.

What seems to have worked for us:

1. Open the still-logged-in FB Messenger app on her phone. It now asks to add a phone number to enhance security. We did this.

2. Now install WhatsApp and sign up using the phone number.

3. Now go into the Facebook app, change password, I forgot my password, and use WhatsApp as 2-factor authentication.

4. Now we have control of the password again! We also added a app (TOTP) 2-factor authentication and iOS passkey to her account at this point to add more options for control.

5. Go to Meta Quest website (meta dot com), and log in via Facebook. This logs us into the attackers account!

6. We could now add a 2-factor authentication to the hackers account, after which it also now let us change the password of the attackers account without knowing the old one.

7. With a password on the account, we can now log them out of all other devices (the attackers phone).

8. We could also now change the permissions so the attackers Meta account could not be used to log in to her Facebook account, but it’s still listed as a related account since it requires email confirmation to remove.

9. Managed to use the 2-factor code to reset the email address on the attackers meta account, so now we own it completely!

jkkola•12h ago
I just wanted to say that's equally impressive and terrifying. Good job though!
realusername•13h ago
I had the same thing with both Google and Apple.

I have a Gmail account lost because they unilaterally activated 2FA on it with a fake phone number (something like 111111111).

And for Apple, moving abroad made the appleid system crash in an loop and nobody in the support system has any idea how appleid works.

Do not use any of the tech giants for any important data.

DANmode•13h ago
Were either of these paid accounts?
realusername•12h ago
Only the Apple one was but anyways, that doesn't excuse borking up the login, even if it's free.

If they don't want us to use a free account, they can simply tell us.

DANmode•12h ago
Agreed!
tomesch1982•13h ago
Microsoft became miserably incompetent in IT in the 1980s. I'm saying this for decades now: If there is one thing that Microsoft is realy bad at then it's software. Almost every piece of software they produce is worse than the alternatives.
Tade0•12h ago
I wanted to snarkily ask "became?".

I recall how the very first edition of Win XP needed time to "cool off" after startup for a minute or two, otherwise it would be wildly unstable. Service Pack 2 resolved most of those issues, but the fact it was needed at all is already telling.

Similarly Windows 11 had regressions in the weirdest of places, like for instance volume controls.

Microsoft has a legacy of bad software.

y42•8h ago
What company is not incompetent in IT?

IT is not flawless. Because it's operated by people. I don't know any software company that offers bugfree software with a superb support and an innovative development roadmap.

Every company is, to a certain level, incompetent in IT. Because there is no perfect competency. The bigger the company, the more prominent this incompetency grows. Which misleads to a confirmation or selection biassed opinion, that this specific company is incompetent.

Take that small startup team that just developed a proof-of-concept, minor bugs as usual, small code-base, collecting 100 Mio. venture capital. Next year they grow to 10.000 employees and 100 Mio. customers. What also grows, inevitable, the code base and amount of bugs. At what state and why would you call them "incompetent in IT"?

PartiallyTyped•13h ago
And yet the stock reached new highs.
progx•13h ago
That only explains one thing: people love & buy shit.
wolvesechoes•11h ago
No, it explains the fact that it doesn't matter what customer wants or buys.
pacifika•12h ago
They expect things to get better, and if it’s this bad maybe they’re right.
AraceliHarker•13h ago
From their responses, could it be that even official Microsoft support is just Microsoft Community volunteers?
progx•13h ago
Win11 "Install update and SHUTDOWN"... computer restarts ;-)

It seems impossible to fix this.

ghishadow•12h ago
even updates are not reliable nowdays
nl•13h ago
Microsoft sign-in issues are so widespread it's amazing people tolerate it.

I have never worked in any place where someone being unable to login to their MS account wasn't a weekly occurrence.

kazinator•12h ago
`Tis ironic he was locked out trying to interact with LibreOffice.

I had a good interaction with the LibreOffice project. I reported something about LibreOffice Writer. Not even a bug, but just a user experience thing in some paragraph cohesion control UI.

My report was taken seriously, discussed and fixed, in a way that was fairly aligned with my suggestions, and it all happened on not a very long time scale, about within a month and a half.

https://bugs.documentfoundation.org/show_bug.cgi?id=156109

aucisson_masque•11h ago
I once reported an obvious bug on open source project (windows preview was misaligned on Linux mint desktop) and told it’s working as intended, case closed.

Open source support vary a lot, you can’t compare it to a business that is actually receiving money for service and take things seriously.

MrPowerGamerBR•2h ago
While true, at least with open source you can actually go into the code and try to fix the code if you really want to.

With a closed source business you are at the mercy of them to decide if they really want to fix your issue, even if you are a paid customer.

PeterStuer•12h ago
I remember a time long ago when Microsoft support was great. I had a problem with Windows 3.11 crashing on my computer. Called the free telephone support, and the person went through a rigorous unscripted troubleshooting session with me to finally find the problem. Turned out some dram timing was slightly of in the bios setting which was only tripped by Windows and not by other software.

That was in the 80's.

velocity3230•12h ago
Windows 3.11 was released in 1993.

Don't mess with the timeline, Marty!

PeterStuer•10h ago
You are right. Must have been early 90's. Might have been Windows 3 or 3.1 as well. I remember you had to install a 3rd party tcp/ip stack, as Windows did not come with one.
donatj•10h ago
Yes! I had an issue with my original Xbox where it wouldn't power on properly, blinking light. After some back and forth with a very Texan man, we determined that I just needed to reseat the video cable. It was fantastic support! I miss those days.
crocowhile•12h ago
It gets much worse than this if you think about what they decided to do with Minecraft. They obliged millions of kids - literally, kids - to have a microsoft account to play a game that has nothing to do with any other microsoft products. They apply sub-standard and child-unfriendly security measures and when armies of kids get hacked (mostly phished) every day the only thing they do is to close down their account and FORCE them to buy the game again.

I still have to understand whether it's incompetence or a business model.

BLKNSLVR•12h ago
If one company becomes massively profitable despite offering literally zero consumer level support, then why would any other company, that covets the same scalability, make any attempt at consumer level support at all?

Google has not scaled it's services, it has ignored hard problems of scaling in favour of solving (relatively) easy technical problems. It has not been punished in any meaningful way for this unbalanced approach, and so "the norm" shall it become.

To the detriment of us all.

ttiurani•12h ago
As a bonus, you can can get completely stuck if you made the mistake of creating a MS family when signing into Minecraft. In the forums there are hundreds of problems like mine where the family becomes completely uneditable and Minecraft offline-only.

But I should've never attempted anything that complicated with MS. They can barely manage simple cases, groups of users is way too hard.

crocowhile•11h ago
Just going through that myself. My son's Microsoft account was hacked, we contactes customer service and all they did was acknowledge the account was compromised, close it indefinitely asking us to buy the game again, and locked my account as the family manager. Fortunately I am a Linux guy and the last Microsoft thing I've touched was windows xp some 20 years ago. But imagine thinking this is acceptable?!

I guess they still suffer from monopoly syndrome. The EU should get them again.

bleh•12h ago
And for us schmucks who bought Minecraft licenses long ago, hope you created MS accounts for them all or they’re gone.
herbst•12h ago
Had one of the first 10k Minecraft accounts or so. Never changed the name or anything. When Microsoft bought it they spammed me that I would loose my account if I don't transfer it. So after the 3rd email or so I did. It got hacked a few days after that, took several weeks to resolve for whatever reason. Ive had the account a few months after that until it was "hacked" again.

The very few times I played Minecraft since then I just used a hacked client.

phba•51m ago
The business incentive is funneling new users into their walled garden. If they could, they would assign a MS account to every newborn. There are still loopholes to protect your children from Minecraft's new "features" (at least for Linux users, ask privately), but it's only a matter of time until they are closed to appease the gods of enshittification.
jauntywundrkind•12h ago
There was a post I enjoyed recently saying that basically Capitalism lost the plot & became a negative force as it became ever more the purpose of a company to diffuse responsibility, to not suffer greatly for doing a so so or even bad job of the thing.

Tech alas so often is the hammer in this bad relationhip, acts as such a horrible tool against humankind broadly. Doing so much to shield and protect the guilty, to make situations illegible & confounded. There's so many domain of concerns that overlap in so many complex ways, and so so so few people within companies and usually vastly less still outside who can see & view these domains & overlap.

It's only getting vastly worse. It feels like the fix so very nearly was in for the Feds to outlaw any regulation at all against AI in the US! AI is so much harder, so much worse than the bad systems work here, vastly more impervious to understanding than what has come before.

We are falling from god's light, into the deepest shadows of our own making. The machines keep getting more infernal.

tdiff•12h ago
I am still waiting for cases of ms locking people out of their windows accounts (which are now required to use windows, you know) for violation of some policy.
drum-computer•12h ago
I once had to go through a very similar process when I wanted to disallow my kid to play Minecraft online. I can't even tell you how much wtf's I've experienced along the way. It literally took me more than 40 minutes and I had to create two accounts, keep in mind I'm a software developer, so I'm a tech savvy person. Then after couple of month we tried to play together on the local lan and after spending another 40 minutes and choosing "allow" on every goddamn option on their settings portal it was still blocked...

I feel like Microsoft has a beta new great portal built just for what you're looking for but then when it becomes just mature enough for version 1.0 it immediately gets overruled by a newer, bigger, better portal that is currently in beta.

herbst•11h ago
As Linux "gamer" it's almost always just the launchers that make(/made) issues rarely the game itself.

More often than not did a proper crack that was a custom launcher basically fix "the whole game" for me.

ReptileMan•12h ago
Don't let me start you on their own proprietary 2fa ...
rich_sasha•11h ago
I'm not sure incompetence is the right word. More like, deliberately cutting corners?

I have a feeling that they know how to write good software, and when pressed, they do. Every now and then they produce something good.

But they also know that they can get away with releasing mediocre, cheaply-made software that works fine for 60% of people, 35% of users hate it but use it anyway, and 5% get locked out/run away to OSX and Linux.

I'm not splitting hairs here. There is a difference between "I can't" and "I can't be bothered".

hatly22•10h ago
Been hearing this sort of bait since the 90s
donatj•10h ago
Hear me out, it's industry wide. There is simply no real support anymore.

My aunt died unexpectedly a number of years ago. Within weeks some man from Nigeria took over her facebook profile. Changed her profile picture from a middle aged white woman to that of a Nigerian man, added a bunch of Nigerian friends. Otherwise the account is seemingly largely untouched. We, the family, have all contacted Facebook support multiple times. We have attempted to memorialize the account submitting proof of death. Nothing. It's beyond infuriating to see a cherished loved ones page vandalized like this regularly every time Facebook has its flashbacks.

Alternatively, a number of years ago I started a Twitter account for a side project that never got very far. Forgot about it.

Years later I make a little update to the side project and think "I could update the Twitter surely no one cares about" and discover to my annoyance that the account got hacked, posted crypto spam for a couple months and got shut down.

I try to contact support, explain the situation. Got into a loop of absolutely infuriating responses of "Your account has been banned. This ticket had been closed. You cannot reply to this message. There is no recourse to this action." No chance for literally any discussion.

I keep attaching longer and longer explanations until I eventually get a "Hmm, let me look into that" from someone that at least smelled minutely human. After a couple messages back and forth suddenly I get the "Your account had been banned. This ticket has been closed..." I throw my arms up and give up. One of the most infuriating experiences of my life. I'd spent over a week fighting to get a largely unused account back. It frankly instilled quite a bit of fear for if I ever lost access to my primary account.

kkfx•9h ago
It's simply the proof anyone need personal ownership in a free internet instead of being dependent on giants on their own computer. It's curious how many in IT fails to see this...
y42•8h ago
The problem is not that they are incompetent in IT, the problem is, IMHO, that they not scale their support with their product. Because they can't.

Yes, the process described is nuts. But it's not an IT process, it's a business process. And there it is about cost saving, trying to automate things and keep them secure at the same time. The reason is: They have to take care of hundreds of millions of customers. Eemploying what, 10.000 support personell? Doing simple math here: One support employee covers 10k customers. Impossible.

This happens everywhere in big tech. Accounts get banned, deleted, people loose data and whatever can go wrong. And all you have is - if at all - some anonymous e-mail address, automated answers and generic online documentation.

Problem is not that FAANG is incompetent in IT. It's just not feasible to cover Millions of individual problems if you grow to a certain size.

stolkin•2h ago
Facebook was even worse. My wife’s account was hacked and we noticed and right away clicked the link to do a reset. Then page then said something like: we have sent the reset link to hacker@hack.ru Which was the email associated with the account for an hour. That is stupid beyond belief. They should have sent that link to the email that had been on the act for the past several years.
bithead•1h ago
As a network engineer I once had a project involving an infrastructure for a deployment of thousands (so I was told) of SQLServers. My part was pretty straightforward and ready well ahead of schedule, but the DB team got stuck because the SQLServer build process kept getting hung. The only response ever heard from microsoft was "It's a network problem"

So I dug into packet captures and found that the switches were using all-zeros gratuitous arps to populate the forwarding table more quickly, which is a fairly common practice. The SQL servers saw this as in indication of a duplicate IP address (the garps were for 0.0.0.0).

Technically this did trigger the duplicate IP address detection RFC, but was never a problem for other server builds or any other server operational capacities. Cisco was the other vendor involved and came out with an update to allow for a timing delay for the all-zeros GARP. However given the timeline runway and the need to do a bug scrub for just one access area in a large data center with other much more lucrative capacity builds the only fiscally responsible thing was to look to the vendor at the heart of the problem.

However it was not possible to get any response from microsoft on it in spite of having the most expensive support contract they offered. The monthly late fees for the project were in the 8 figure range, so microsoft was dumped since they were not answering the phone. It was not for a lack of trying either. Our so-called "high touch" liaisons were equally frustrated.

It was weird since we were not a small customer - we were one of their largest. If a fortune 10 can't get microsoft to answer the phone, what hope does anyone else have?