frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Intel and AMD standardise ChkTag to bring Memory Safety to x86

https://community.intel.com/t5/Blogs/Tech-Innovation/open-intel/ChkTag-x86-Memory-Safety/post/172...
128•ashvardanian•6d ago•58 comments

Building a message queue with only two UNIX signals

https://leandronsp.com/articles/you-dont-need-kafka-building-a-message-queue-with-only-two-unix-s...
49•SchwKatze•2h ago•28 comments

Claude Code on the web

https://www.anthropic.com/news/claude-code-on-the-web
346•adocomplete•6h ago•209 comments

My trick for getting consistent classification from LLMs

https://verdik.substack.com/p/how-to-get-consistent-classification
92•frenchmajesty•1w ago•22 comments

Production RAG: what I learned from processing 5M+ documents

https://blog.abdellatif.io/production-rag-processing-5m-documents
291•tifa2up•9h ago•80 comments

BERT is just a single text diffusion step

https://nathan.rs/posts/roberta-diffusion/
340•nathan-barry•10h ago•83 comments

A laser pointer at 2B FPS [video]

https://www.youtube.com/watch?v=o4TdHrMi6do
188•thunderbong•1d ago•28 comments

Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

https://www.tomshardware.com/tech-industry/semiconductors/alibaba-says-new-pooling-system-cut-nvi...
330•hd4•12h ago•228 comments

Show HN: I created a cross-platform GUI for the JJ VCS (Git compatible)

https://judojj.com
61•bitpatch•9h ago•7 comments

Postman which I thought worked locally on my computer, is down

https://status.postman.com
182•helloguillecl•9h ago•83 comments

Today is when the Amazon brain drain sent AWS down the spout

https://www.theregister.com/2025/10/20/aws_outage_amazon_brain_drain_corey_quinn/
273•raw_anon_1111•4h ago•115 comments

x86-64 Playground – An online assembly editor and GDB-like debugger

https://x64.halb.it/
105•modinfo•7h ago•8 comments

Code from MIT's 1986 SICP video lectures

https://github.com/felipap/sicp-code
82•felipap•3d ago•6 comments

AWS Multiple Services Down in us-east-1

https://health.aws.amazon.com/health/status?ts=20251020
1574•kondro•17h ago•1776 comments

The scariest "user support" email I've ever received

https://www.devas.life/the-scariest-user-support-email-ive-ever-received/
144•hervic•5d ago•108 comments

Art Must Act

https://aeon.co/essays/harold-rosenberg-exhorted-artists-to-take-action-and-resist-cliche
20•tintinnabula•4d ago•0 comments

TernFS – an exabyte scale, multi-region distributed filesystem

https://www.xtxmarkets.com/tech/2025-ternfs/#posix-shaped
88•kirlev•7h ago•12 comments

How to stop Linux threads cleanly

https://mazzo.li/posts/stopping-linux-threads.html
164•signa11•5d ago•60 comments

Optical diffraction patterns made with a MOPA laser engraving machine [video]

https://www.youtube.com/watch?v=RsGHr7dXLuI
106•emsign•6d ago•18 comments

Atomic-Scale Protein Filters

https://press.asimov.com/articles/filters
11•mailyk•5d ago•0 comments

Space Elevator

https://neal.fun/space-elevator/
1471•kaonwarb•20h ago•336 comments

The longest baseball game took 33 innings to win

https://www.mlb.com/news/the-longest-professional-baseball-game-ever-played
38•mooreds•5d ago•54 comments

Old Is Gold: Optimizing Single-Threaded Applications with Exgen-Malloc

https://arxiv.org/abs/2510.10219
4•todsacerdoti•5d ago•2 comments

Docker Systems Status: Full Service Disruption

https://www.dockerstatus.com/pages/incident/533c6539221ae15e3f000031/68f5e1c741c825463df7486c
326•l2dy•17h ago•124 comments

J.P. Morgan's OpenAI loan is strange

https://marketunpack.com/j-p-morgans-openai-loan-is-strange/
199•vrnvu•5h ago•130 comments

Servo v0.0.1

https://github.com/servo/servo
459•undeveloper•12h ago•141 comments

DeepSeek OCR

https://github.com/deepseek-ai/DeepSeek-OCR
861•pierre•18h ago•219 comments

Why UUIDs won't protect your secrets

https://alexsci.com/blog/uuids-and-idor/
4•8organicbits•1h ago•0 comments

Americans can't afford their cars any more and Wall Street is worried

https://www.telegraph.co.uk/business/2025/10/20/americans-cant-afford-cars-any-more-wall-street-w...
56•zerosizedweasle•1h ago•51 comments

iOS 26.1 lets users control Liquid Glass transparency

https://www.macrumors.com/2025/10/20/ios-26-1-liquid-glass-toggle/
144•dabinat•5h ago•122 comments
Open in hackernews

Today is when the Amazon brain drain sent AWS down the spout

https://www.theregister.com/2025/10/20/aws_outage_amazon_brain_drain_corey_quinn/
267•raw_anon_1111•4h ago

Comments

jqpabc123•3h ago
Nothing gets sold or fixed without people who know how it's built.
nine_zeros•3h ago
But it is more important to keep the PIP, stack-ranking, ladder-climbing game running than keeping the people /s
bwfan123•2h ago
cue in: programming as theory building [1] or building systems as theory building, ie, Mental causal models of how and why things work the way they do. Mental models live in people's heads and walk out of the door when they do. Management learns this the hard way [2].

[1] https://pages.cs.wisc.edu/~remzi/Naur.pdf

[2] https://x.com/elonmusk/status/1980221072512635117

ChrisArchitect•3h ago
More discussion: https://news.ycombinator.com/item?id=45640838
ortusdux•3h ago
"If you were a ‘product person’ at IBM or Xerox: so you make a better copier or better computer. So what? When you have a monopoly market-share, the company’s not any more successful. So the people who make the company more successful are the sales and marketing people, and they end up running the companies. And the ‘product people’ get run out of the decision-making forums.

The companies forget how to make great products. The product sensibility and product genius that brought them to this monopolistic position gets rotted out by people running these companies who have no conception of a good product vs. a bad product. They have no conception of the craftsmanship that’s required to take a good idea and turn it into a good product. And they really have no feeling in their hearts about wanting to help the costumers.”

- Steve Jobs - https://en.wikipedia.org/wiki/Steve_Jobs:_The_Lost_Interview

ASalazarMX•2h ago
Great words, but he lost any right to them when he made famous the "You're holding it wrong" workaround. IMO that was the defining moment when Apple started its decline on product innovation.
JustExAWS•2h ago
It was a nothingburger. Apple sold the same GSM iPhone 4 for three years with no design changes and nothing else was said about it three months later.
bobbiechen•2h ago
Well, there was a software change to smooth out how the bars would display.. https://9to5mac.com/2025/10/08/a-15-year-mystery-solved-the-...
DecentShoes•2h ago
It was not nothing. The phone stopped working if the user held it naturally in their hand. I had one. Reception completely cut out.
JustExAWS•2h ago
If it were that big of deal, don’t you think Apple would have been forced to recall it and definitely couldn’t keep selling it for 3 years. True they did redo the antenna for the Verizon CDMA iPhone 4. But they never bothered to back port the changes to the GSM one.

I also had a GSM iPhone 4.

Compare that to how quickly they ran away from the shitty Intel modems when they were selling some made by Intel and some made by Samsung (?)

hluska•1h ago
Apple offered a free case to make the problem go away. My iPhone just had trouble typing Apple and free in the same sentence.
Mistletoe•1h ago
I’d add a corollary to Steve’s often quoted idea, that became even more relevant after Covid. Everyone competent that makes tons of money retired early. We are left with the dregs at all these companies, the newbies and those that didn’t budget well and plan for early retirement.

Any interaction you have with a company post-Covid you can feel it. Nothing works anymore and you can’t even tell anyone about it or why.

deepsquirrelnet•1h ago
FWIW this has happens in consulting too, not just product companies. Just swap “product” for “delivery”.
behnamoh•2h ago
> It is a fact that there have been 27,000+ Amazonians impacted by layoffs between 2022 and 2024, continuing into 2025. It's hard to know how many of these were AWS versus other parts of its Amazon parent, because the company is notoriously tight-lipped about staffing issues. Internal documents reportedly say that Amazon suffers from 69 percent to 81 percent regretted attrition across all employment levels. In other words, "people quitting who we wish didn't." The internet is full of anecdata of senior Amazonians lamenting the hamfisted approach of their Return to Office initiative; experts have weighed in citing similar concerns.

So the title is all speculation. The author put 2 and 2 together and concluded that 10 is greater than 9.

Worthless article.

samrus•2h ago
Just because its speculation doesnt mean its worthless. But yeah it should be taken as speculation rather than a validated and tested hypithesis
behnamoh•2h ago
Anything that wastes my time and only reveals its half-assed reasoning half way through the article is indeed worthless.
a0123•2h ago
You could make smart inferences based on past and very frequent occurrences.

Or you could just say "there is no way the thing that constantly happens over and over again has happened once again, just no way".

Staff cuts constantly happen in the name of maximising profits. They always yield poor results for a company's performance. Every time. Especially for the consumer's side of it (not the company's finances of course).

Every time.

But maybe this time it's different. That one time.

bryanlarsen•1h ago
First time reading El Reg?
toofy•1h ago
i mean, you can assume if its on theregister its not going to have some kind of academic rigor or whatever it might be you're looking for. its the register, same basic rigor quality as the ny post.

that said, my suspicion is they're likely on to something here regarding layoffs and quality degradation.

JustExAWS•2h ago
I worked at AWS and still have friends who work there. I don’t know any L5s who wouldn’t jump at a chance to leave if they even got a slightly worse offer than what they are making now. I know a few L6s and L7s that would stick around out of momentum.

But I know very few people in the industry who know about Amazon’s reputation that have a life long dream of working there given a choice.

I was 46 when I was hired there for a “permanently remote [sic] field by design role” in ProServe and it was my 8th job out of college. I went in with my eyes wide open. I had a plan, stay for four years, sell my RSUs as soon as they vested, pay off debt, save some money, put it on my resume to open doors and make connections and leave.

I was never expecting to make more when I left. I used the time to downsize and reduce my expenses - including moving to state tax free Florida.

When I saw the writing on the wall, I played the game while I was on focus to get my next vest and wait for the “get 40k+ severance and leave immediately or try to work through the PIP”.

I took the latter and had three offers within 3 weeks. This was late 2023.

azemetre•2h ago
How long did you stick it out? Were you close toward completing your plan?
JustExAWS•2h ago
Close enough. I missed 2 vesting periods. But the severance and rapidly having a job made up for one and I got refreshers my third year that I hadn’t counted on.

I left debt free, sold my old home for exactly twice what I had built for 8 years earlier, downsized to a condo half the price I sold it for (and 1/3 the size) and I was debt free with savings.

I’m now a staff consultant working full time at a 3rd party AWS consulting firm with a lot less stress and still remote. They were the last to fall. But AWS made their ProServe department return to office at the beginning of this year.

dijit•2h ago
Maybe its speculation, but I mean drawing conclusions becomes easy when 40% of devops staff being laid off by AWS was in the news three days ago.

https://blog.stackademic.com/aws-just-fired-40-of-its-devops...

placardloop•2h ago
AWS doesn’t even have a “devops team” nor even any devops job roles. AWS also does not use Terraform (which is what the article says everyone was replaced with) at any significant scale, so this article is similar junk.
dijit•2h ago
Uh, they still have roles open for DevOps:

https://amazon.jobs/en/jobs/3080348/devops-engineer-linux-re...

https://amazon.jobs/en/jobs/3082914/devops-systems-engineer-...

This one mentions terraform by name (though that doesn't necessarily imply its in use, though having worked in large companies I would argue that sweeping statements about a popular technology not being used is likely to be wrong)

https://amazon.jobs/en/jobs/3042892/delivery-consultant-devo...

placardloop•1h ago
AWS does not have dedicated devops roles. All AWS SWEs are expected to take oncall shifts and respond to incidents, manage build pipelines, etc rather than having specific devops people to do it for them. The article you linked claiming 40% of them were fired is total junk. You can believe that or not, I don’t care.

The last one is a ProServe role, which is a consulting role that spends their time working in customer environments, which is where they may encounter terraform. It does not mean anything about internal use of terraform.

dijit•1h ago
Again, I’d be wary making sweeping generalisations like that.

I already showed you that AWS has (or hires) DevOps people with publicly available information, maybe the article is incorrect but you’re clearly not better informed, so maybe cut it with the rude commentary.

placardloop•1h ago
Misunderstanding the things you are linking does not mean you proved anyone wrong.
dijit•1h ago
I’m going to link this again, what role is this please:

https://amazon.jobs/en/jobs/3080348/devops-engineer-linux-re...

placardloop•1h ago
Within AWS this role falls under the Systems Engineer job family. It is not a devops role, and its involvement in events like today would be the same involvement as every other SWE at Amazon.

Just do a quick google search for that “40% of devops laid off” and you’ll see that it’s actually an old article from months ago that multiple people, including AWS employees, are saying is bullshit and unsourced.

edit: found another source that says this 40% number came from an AWS consultant that worked with customers to help them be better at DevOps, and it was 40% of their specific team that was laid off. Even if it were true, it has nothing to do with the internal operations of AWS services. This is why it’s important to understand the information you’re sharing before making judgements off of it.

dijit•1h ago
All I am finding are corroborating articles, maybe you can help me here.

https://www.theregister.com/2025/07/18/aws_sheds_jobs/

Seems wild that you would promote job titles you don’t hire for, makes me think that it’s reasonable for news outlets to refer to those roles in the same way honestly.

placardloop•1h ago
There is nothing in that article that mentions either devops or 40% of any team or role being cut. It doesn’t corroborate anything.
dijit•59m ago
It makes a direct claim of hundreds of cloud staff being laid off.

You know what though? I’m not wasting my time with you, the fact was that this was all over social media. Then a huge outage- my original comment was factually accurate even if we contend that the article itself was bunk. And AWS clearly hires DevOps staff.

You’ve not even disproved anything you’re just making me play internet fetch. I’m not replying anymore.

anonymars•14m ago
> It makes a direct claim of hundreds of cloud staff being laid off.

I don't have any dog in this fight, but I don't see where this article makes your case. From your article:

> We understand around 100 jobs are at stake.

> Sources familiar with AWS operations who requested anonymity told The Register most of the layoffs affected people in marketing and outreach roles, although chatter on sites like Blind suggests folks in frontline support and in other positions may have been affected, too.

jrflowers•59m ago
You seem to be kind of annoyed that somebody on the internet hasn’t taken your assertion that you just sort of generally Know Better as strongly as you’d like. You could probably put this entire discussion to bed by clarifying your current position at AWS and how your job there gives you direct knowledge of their devops practices.
JustExAWS•1h ago
Source: Former AWS Professional Services employee.

Notice the job description:

As part of the AWS Managed Operations team, you will play a pivotal role in building and leading operations and development teams dedicated to delivering high-availability AWS services, including EC2, S3, Dynamo, Lambda, and Bedrock, exclusively for EU customers.

They aren’t looking for DevOpe engineers to work alongside the “service teams” - the teams that build and support internal AWS services. They are working with AWS customers who may already be using Terraform. AWS has a large internal consulting division staffed with full time employees. When they work with customers they will use Terraform if needed.

hinkley•1h ago
You could both be right if they are trying to expand terraform use from a beachhead to the entire company. You need to hire people with prior experience for such things.
jongjong•2h ago
Speaking of DNS, I still cannot comprehend why we still rely on the current complex, aging, centralized, rent-seeking DNS.

It's one one of the few parts of the internet which could potentially be replaced over time with very little disruption.

The hierarchy of resolvers could be replaced with a far simpler flat hierarchy Blockchain where people could buy and permanently own their domains directly on-chain... No recurring fees. People could host websites on the Blockchain from beyond the grave... This is kind of a dream of mine. Not possible to achieve in our current system.

madaxe_again•2h ago
To bake an apple pie from scratch, you must first overthrow capitalism.
chicagobuss•2h ago
perfection
tpmoney•2h ago
> People could host websites on the Blockchain from beyond the grave...

This is precisely why something like this isn't a popular solution lots of people are working towards. Domains broadly speaking aren't a finite resource, but usable domains using common words definitely are. As time marches on human readable/typeable "permanent identifiers" are going to have to go away. Email address, usernames and the like are all going to get recycled, just like phone numbers are. Domains are currently recycled and most people probably think that's a good thing.

tpmoney•2h ago
> People could host websites on the Blockchain from beyond the grave...

This is precisely why something like this isn't a popular solution lots of people are working towards. Domains broadly speaking aren't a finite resource, but usable domains using common words definitely are. As time marches on human readable/typeable "permanent identifiers" are going to have to go away. Email address, usernames and the like are all going to get recycled, just like phone numbers are. Domains are currently recycled and most people probably think that's a good thing (assuming they think about it at all)

yreg•2h ago
Also the potential for a domain to get irreversibly stolen is not a good feature for the security of the users.
tombert•2h ago
I think this would make the squatting problem that we already have way worse. There would be bots buying every single remotely usable domain, and there would be no incentive for them to sell it unless they get an absurdly large offer.

I bought tombert.com in 2014 and forgot to renew it in 2015, and it was auctioned off by GoDaddy. For like six years, it was owned by squatters, and they wanted thousands of dollars for the domain [1]. I called offering the $100 for it, and they claimed that they can't go below $1400 because this domain is in "extremely high demand". I finally was able to buy it back in 2021, presumably because the squatter purged out domains that hadn't been purchased for N years and they wanted to save money.

Now, you could argue "see! You wouldn't have had to worry about it expiring if it were permanent on the blockchain", and that's true, but if someone else had gotten to that domain first, then I would also never get it. I think the only thing that keeps the internet even remotely fair in this regard is that domain names cost some amount of money to keep.

[1] https://web.archive.org/web/20160219161720/http://www.hugedo...

bombcar•1h ago
Let’s solve rent-seeking by building a new system that’s even more rent-seeking!

At least currently death dissolves bonds.

chicagobuss•2h ago
internal reports from current AWS engineers seem to be confirming all of the speculation in this article. Shit's rotten from the inside out and you can pretty evenly blame AI, brain drain, and good old fashioned "big company politics"

https://forums.theregister.com/forum/all/2025/10/20/aws_outa...

int3trap•1h ago
There's been a massive talent exodus, especially among the principal and senior principal engineering roles, across all Amazon orgs since the RTO policies have been enforced. Its demoralizing to lose key engineers that you look up to and want to continue to learn from all because a few people far removed from the day to day make a bad call.

RTO in combination with Amazon being last place in AI innovation have led to departures of anyone that can leave, leaving.

mschuster91•1h ago
> So they can't suggest a fix even if they know 100% what it will be. Thats exactly what happened this time. EIGHT different staff members pointed to the underlying cause and were told (some literally) to "shut the f*ck up and get back to your job"

Jesus, if even an ounce of that is true... Yes, everyone on the internet is a cat clawing on a keyboard... but if a ton of people legitimately confirmed to be ex-AWS point to similar culture issues... probably it's AWS that's rotting.

pdonis•2h ago
One thing I love about El Reg is that they never shirk from calling a spade a spade.
freedomben•1h ago
I also love the humor and personality that authors are able to inject into their pieces
mrec•1h ago
I miss Lewis Page. It's a rare day when I'm so downhearted that the memory of https://www.theregister.com/2008/12/30/german_beaver/ can't raise at least a wan smile.
furyofantares•45m ago
They never shirk from calling anything a spade.
add-sub-mul-div•2h ago
Amazon has reportedly been a shitty place to work forever, so using issues that happen to be popular today to explain turnover is disingenuous.
sc68cal•1h ago
From the article.

> At the end of 2023, Justin Garrison left AWS and roasted them on his way out the door. He stated that AWS had seen an increase in Large Scale Events (or LSEs), and predicted significant outages in 2024. It would seem that he discounted the power of inertia

Your comment is relying on that referenced inertia. Things will continue to function for a period of time, but there exists an inflection point at which they no longer function as previously.

bombcar•1h ago
Arguably we had a perfect example with Twitter.
aerostable_slug•1h ago
In what way? It seems reliable enough. I admit I don't check it constantly, but I've not noticed outages impacting my perhaps middling usage.
mschuster91•1h ago
The year or so after Musk took over was brutal. The influx of far-right pests, troll farms and porn bots was one thing, but the reliability went down the drain.
bombcar•36m ago
Exactly / but it did not DIE as many were predicting (even on HN many were claiming it would go down immediately).

Inertia is a hell of a force.

mlhpdx•2h ago
This is the time to accept that the path forward is keeping people and giving them the best tools you possibly can to do their work. That is, the same as has been true for decades remains so.

Yes, development tools are better every day. Yes, you can downsize. No it won’t be felt immediately. Yes, it mortgages the future and at a painfully high interest rate.

Suspending disbelief won’t make downsizing work better.

add-sub-mul-div•1h ago
I don't think there's ignorance of the fact that turnover is bad, I think the field is being designed to homogenize staff and favor uniform mediocrity so that employees truly do become interchangeable. We're so close to just plain talent being likened to cowboyism.
bcrosby95•1h ago
Nah, the decades old crop of "new" big tech companies are just entering their IBM phase.
CobrastanJorji•1h ago
Seems like it worked fine. They laid off a quarter of their junior principal engineers, the stock went up. They had a massive outage a few months later, the stock went up again. Everything's working out fine for their strategy so far.
few•1h ago
I remember comments saying the stock went up because the average joe didn't realize how much of the internet was powered by AWS until all their day to day apps started failing. To most people Amazon is an online shopping site.
array_key_first•22m ago
It continues to work until eventually the debt is so high the company implodes.

See: general electric, RCA, Xerox, GM

foobarian•9m ago
You would think this would eventually show up on the balance sheets, right? Presumably a lot of their big customers have SLAs with money penalties, so maybe next quarter earnings? Or quarter after that?
nijave•1h ago
It was certainly suspicious that actual progress on the outage seemed to start right around U.S. west coast start of day. Updates before that were largely generic "we're monitoring and mitigating" with nothing of substance.
hinkley•1h ago
Now a post I read on Reddit this morning makes a lot of sense.
dehugger•1h ago
what was the post?
jftuga•46m ago
This was a funny take on it...

https://archive.ph/o4q5Z

swagmoose•1h ago
context... it's not just for LLMs
edoceo•1h ago
I thought the recovery was early AM Seattle time (like 4am). Where I think start-of-day is like 9am. Maybe recovery started early (6am) New York time?
nijave•1h ago
[09:13 AM PDT] We have taken additional mitigation steps to aid the recovery of the underlying internal subsystem responsible for monitoring the health of our network load balancers and are now seeing connectivity and API recovery for AWS services. We have also identified and are applying next steps to mitigate throttling of new EC2 instance launches. We will provide an update by 10:00 AM PDT.

[08:43 AM PDT] We have narrowed down the source of the network connectivity issues that impacted AWS Services...

[08:04 AM PDT] We continue to investigate the root cause for the network connectivity issues...

[12:11 AM PDT] <declared outage>

They claim not to have known the root cause for ~8hr

anon7000•40m ago
I don’t think that’s true, there was an initial Dynamo outage that was resolved in the wee hours that ultimately cascaded into the ec2 problem that lasted most of the day
nijave•30m ago
Was the Dynamo outage separate? My take was the NLB issue was the root cause and Dynamo was a symptom which they flipped some internal switches to mitigate the impact to that dependency
NKosmatos•1h ago
This is how articles should be written, this is why I’m reading El Reg (a.k.a. The Register) all these decades, this is what happens when high management cares only about profits and when real engineers don’t eat the RTO bullshit. Bravo for putting this online.

P.S. I’m not an Amazon hater, replace the company name with any other big one of your choice and the article will have the same meaning ;-)

add-sub-mul-div•1h ago
It's not your feeling about Amazon that would cast doubt on your take, it's that you've reduced it to one pet cause and decided a source is well written because it appeals to your dislike of RTO. Nowhere has any evidence of the relevance of that to this been presented, Amazon has had outages since before WFH even began, they've all always had their occasional outages and bad days.
christhecaribou•34m ago
There’s clear data that shows an increase in the frequency and severity of LSEs post RTO3 in 2023, and it looks like RTO5 has accelerated that trend.
orbital-decay•47m ago
The Register is an opinionated tech tabloid filled with outrage bait. This article is not an exception, drawing far reaching conclusions from little evidence.
christhecaribou•44m ago
“Little evidence?” If the “aws” partition doesn’t actually exist when IAD breaks, Amazon hasn’t even discovered how to make multi-region cloud infrastructure. That’s a big deal.
Dig1t•1h ago
>Amazon remained the single largest H-1B sponsor, increasing approvals from 9,257 in 2024 to 10,044 in 2025, an addition of 787 visas.

https://www.reddit.com/r/SeattleWA/comments/1ncm25p/amazon_m...

I’m confused how they can have such a failure, they are employing the best and brightest top tier talent from India.

Hopefully they can increase their H1B allotment even more next year to help prevent these types of failures.

browningstreet•1h ago
It depends on how the resources are assigned, what projects they’re asked to focus on, and which strategic tech debt initiatives aren’t approved.
hluska•1h ago
We’ve reached a point where I can no longer distinguish between people without experience and people repeating the talking points they’re told to repeat. That’s a major loss.

However, talent is a very small part of shipping a project. How that talent is resourced is far more important.

dafelst•1h ago
Given today is Diwali, perhaps the reason everything went down is because the best and brightest from India were all on vacation and weren't there to babysit/roll back the deployment that broke everything?
citizenpaul•1h ago
Its almost like institutional knowledge is a real thing that you cannot put on some BIG BRAIN MBA spreadsheet.
Groxx•19m ago
how many lines of code is it though? or how many tokens? we need something we can measure if we're gonna fire people!
g-b-r•1h ago
For those who worked there recently, how much does the comment at [1] reflect the current state of things?

[1] https://forums.theregister.com/forum/all/2025/10/20/aws_outa...

bithead•1h ago
What - their AI couldn't find it sooner? Better get those RAGs in order.
charcircuit•1h ago
This fails to recognize that the people who designed everything to rely on us east 1 did so a long time ago. "Brain drain" could just mean that they've had their fun and now want other people to deal with their mess.

>I've seen zero signs that this stems from a lack of transparency, and every indication that they legitimately did not know what was breaking for a patently absurd length of time.

That information is under NDA, so it's only natural you aren't privy to it.

Esophagus4•1h ago
I’ve seen this happen with startups as well -

They’ll get acquired and top people leave as their stock vests or get pushed out because the megacorp wants someone different in the seat.

The people who knew the tech are gone and you’re left with an unmaintainable mess that becomes unreliable and no one knows how to fix it.

pinkmuffinere•1h ago
> one really gets the sense that it took them 75 minutes to go from "things are breaking" to "we've narrowed it down to a single service endpoint, but are still researching," which is something of a bitter pill to swallow

Is 75 minutes really considered that long of a time? I don't do my day-job in webdev, so maybe I'm just naive. But being able to diagnose the single service endpoint in 75 minutes seems pretty good to me. When I worked on firmware we frequently spent _weeks_ trying to diagnose what part of the firmware was broken.

estimator7292•1h ago
75 minutes is damn good turnaround for any major problem, IMO
__loam•1h ago
Amazon is supposed to have the best infrastructure in the business because everyone else runs on it. They should have access to the sre talent that can quickly mitigate this kind of issue
Freedom2•30m ago
What if the SRE talent has a lot of real-life experience but can't pass LeetCode puzzles that have nothing to do with the job?
ecnahc515•1h ago
Also it's pretty likely it took less time than that to get an idea, but generally for public updates you want to be very reserved, otherwise users get the wrong impressions.
michaelt•49m ago
> Is 75 minutes really considered that long of a time? [...] When I worked on firmware we frequently spent _weeks_ trying to diagnose what part of the firmware was broken.

One might spend weeks diagnosing a problem if the problem only happens 0.01% of the time, correlated with nothing, goes away when retried, and nobody can reproduce it in a test environment.

But 0.01%-and-it-goes-away-when-retried does not make a high priority incident. High priority incidents tend to be repeatable problems that weren't there an hour ago.

Generally a well designed, properly resourced business critical system will be simple enough and well enough monitored that problems can be diagnosed in a good deal less than 75 minutes - even if rolling out a full fix takes longer.

Of course, I don't know how common well designed, properly resourced business critical systems are.

causal•1h ago
Yeah. They will identify the cause, but not the cause behind the cause.
president_zippy•1h ago
Between the engineering staff and the warehouse workers, I wonder how long it will be until they have already fired everyone who ever would have been willing to work there.

Even with candidate pools of hundreds of thousands of H1-B engineers and tens of millions of illegal immigrant warehouse workers, there still comes a point where such a big company firing so many people so quickly exhausts all their options.

It reminds me of the Robot Chicken Sketch where Imperial Officers aboard the Death Star all pretend to be force choked to death by Darth Vader so they can avoid getting killed by lightsaber, then come back in under different names in different jobs. It's worse though for Amazon: nobody wants to come back.

https://www.youtube.com/watch?v=fFihTRIxCkg

steve-atx-7600•1h ago
Seriously. I don’t know any half way decent engineer that would ever work there twice.
president_zippy•42m ago
I like to think I'm halfway decent at my job, and I wouldn't work there once. During undergrad, my landlord working for AMZN on the opposite end of the country offered me an interview, but it was during final exam week.

I asked if I could schedule the interview after my final exams, and his arrogance really showed when not only did he refuse, but then insisted my exams are not don't even register on the same scale of importance as the opportunity to work for Amazon.

Somewhat related: a recruiter at Google cold-called me a couple months into my first job out of undergrad back in 2016 and was similarly condescending about "the chance" to work for Google compared to everything else. I already had a low opinion of them when they gave my then-girlfriend an introductory O'Reilly book on Java after she failed their interview.

I regret being born too late to work somewhere like Bell Labs, SGI, or Sun. I had a ton of graybeard wizard coworkers from these places, and they were all a pleasure to learn from and even better friends. For the first 2 years of my first job, every day of work was like walking into the Shire and talking magic spells with 20 Gandalfs.

That job was great until I got put on a team with a guy who was a former middle manager at some IBM-like company and went from being surrounded by people lightyears ahead of me to being surrounded by Dilbert characters. The messed-up part was that it wasn't even punishment. I was rewarded after completing a project with my choice of which team I joined next, and I joined the wrong one. I assumed that joining a new team to utilize this newfangled "cloud computing" thing would be trailblazing, and I didn't do any diligence on who I would work with.

To this day, I still regret not rejoining the first team I worked for, otherwise I would still be at that company and happy about it. Then again, the boredom and discontent while being on that sucky team is the reason I started investing, and now I can buy a house in cash and fund myself to do whatever I want for at least a decade. Hard to complain about the way things turned out.

le-mark•32m ago
> to being surrounded by Dilbert characters.

As a real life Wally I appreciate this comment.

kaladin-jasnah•27m ago
> I regret being born too late to work somewhere like Bell Labs, SGI, or Sun.

I'm not even out of college, and I feel the same way. Especially for Sun, everything they did was so cool. "The network is the computer" and all that.

mjamil•7m ago
The coolness peaked before the “the network is the computer” phase, IMO. Late 80s vs mid 90s.
nradov•48m ago
Are there really many illegal aliens working in the warehouses? I know that Amazon does verify employment eligibility and checks documents. There may be some committing identity theft but I doubt that it's a large proportion.
ahi•32m ago
I assume the parent was being hyperbolic. Illegal immigrants is the barrel scraping bottom of the work force.
catskull•10m ago
I don’t think it’s identify theft so much as it’s SSN “renting” where the citizen is in on it and will then file a fraudulent tax return to collect the illegal aliens tax return. It’s a win-win.
mmonaghan•48m ago
Ehh I trust the reporting and generally agree that RTO was/is executed hamfisted but I dunno if this particular incident "makes" the narrative. IIRC LSE rate has been increasing for many years, maybe most of AWS's existence. This is part and parcel of building something so complex that continues to grow and evolve.

I do expect much better of them and they certainly have problems to solve but this is a big company evolution thing and not an Amazon-specific thing imo.

christhecaribou•46m ago
The real story from this incident is that Amazon’s “aws” partition doesn’t actually have multiple regions - effectively, it’s all IAD in a trench-coat.

This is a big deal. My employer has already started to look at bringing back our old racks from storage and switching back to on-premises. Cannot imagine he’s alone in that.

anigbrowl•43m ago
One of the best-written articles I've read in a long time. I wish general news coverage had this tight blend of fact, context, and long-term perspective.
the_real_cher•35m ago
AWS has been having issues like this for years.
benjaminclauss•6m ago
big "who is John Galt" vibes in these comments lol