frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Trillions Spent and Big Software Projects Are Still Failing

https://spectrum.ieee.org/it-management-software-failures
48•pseudolus•5h ago

Comments

franktankbank•59m ago
> Phoenix project executives believed they could deliver a modernized payment system, customizing PeopleSoft’s off-the-shelf payroll package to follow 80,000 pay rules spanning 105 collective agreements with federal public-service unions.

Somehow I come away skeptical of the inevitable conclusion that Phoenix was doomed to fail and instead that perhaps they were hamstrung by architecture constraints dictated by assholes.

ruralfam•39m ago
My reaction also. 80K payroll rules!!! Without much prompt effort, I got about 350K Canada Federal Service employees (sorry if not correct).
dmix•26m ago
Sounds like they put zero effort into simplifying those rules the first time around.

Now in the new project they put together a committee to attempt it

> The main objective of this committee also includes simplifying the pay rules for public servants, in order to reduce the complexity of the development of Phoenix's replacement. This complexity of the current pay rules is a result of "negotiated rules for pay and benefits over 60 years that are specific to each of over 80 occupational groups in the public service." making it difficult to develop a single solution which can handle each occupational groups specific needs.

stackskipton•19m ago
I have worked on government payroll systems, simplifying those rules is almost impossible from political PoV. They are generally a combo of weird laws, court cases, union contracts and more.

Any time you think about touching them, the people who get those salaries come out in droves and no one else cares so government has every incentive to leave them alone.

franktankbank•17m ago
Oh great a committee!
ZeroConcerns•58m ago
Yup, and with an equal amount of mindblowing-units-of-money spent, infrastructure projects all around me are still failing as well, or at least being modified (read: downsized), delayed and/or budget-inflated beyond recognition.

So, what's the point here, exactly? "Only licensed engineers as codified by (local!) law are allowed to do projects?" Nah, can't be it, their track record still has too many failures, sometimes even spectacularly explosive and/or implosive ones.

"Any public project should only follow Best Practices"? Sure... "And only make The People feel good"... Incoherent!

Ehhm, so, yeah, maybe things are just complicated, and we should focus more on the amount of effort we're prepared to put in, the competency (c.q. pay grade) of the staff we're willing to assign, and exactly how long we're willing to wait prior to conceding defeat?

JohnMakin•55m ago
> "Why worry about something that isn’t going to happen?”

Lots to break down in this article other than this initial quotation, but I find a lot of parallels in failing software projects, this attitude, and my recent hyper-fixation (seems to spark up again every few years), the sinking of the Titanic.

It was a combination of failures like this. Why was the captain going full speed ahead into a known ice field? Well, the boat can't sink and there (may have been) organizational pressure to arrive at a certain time in new york (aka, imaginary deadline must be met). Why wasn't there enough life jackets and boats for crew and passengers? Well, the boat can't sink anyway, why worry about something that isn't going to happen? Why train crew on how to deploy the life rafts and emergency procedures properly? Same reason. Why didn't the SS Californian rescue the ship? Well, the 3rd party Titanic telegraph operators had immense pressure to send telegrams to NY, and the chatter about the ice field got on their nerves and they mostly ignored it (misaligned priorities). If even a little caution and forward thinking was used, the death toll would have been drastically lower if not nearly nonexistent. It took 2 hours to sink, which is plenty of time to evacuate a boat of that size.

Same with software projects - they often fail over a period of multiple years and if you go back and look at how they went wrong, there often are numerous points and decisions made that could have reversed course, yet, often the opposite happens - management digs in even more. Project timelines are optimistic to the point of delusion and don't build in failure/setbacks into schedules or roadmaps at all. I've had to rescue one of these projects several years ago and it took a toll on me I'm pretty sure I carry to this day, I'm wildly cynical of "project management" as it relates to IT/devops.

BirAdam•54m ago
I study and write quite a bit of tech history. IMHO from what I've learned over the last few years of this hobby, the primary issue is quite simple. While hardware folks study and learn from the successes and failures of past hardware, software folks do not. People do not regularly pull apart old systems for learning. Typically, software folks build new and every generation of software developers must relearn the same problems.
tristor•52m ago
This is one part of the issue. The other major piece of this that I've seen over more than two decades in industry is that most large projects are started by and run by (but not necessarily the same person) non-technical people who are exercising political power, rather than by technical people who can achieve the desired outcomes. When you put the nexus of power into the hands of non-technical people in a technical endeavor you end up with outcomes that don't match expectations. Larger scale projects deeply suffering from "not knowing what we don't know" at the top.
cjbgkagh•26m ago
Sometimes giving people what they want can be bad for them; management wants cheap compliant workers, management gets cheap compliant workers, and then the projects fall apart in easily predictable and preventable ways.

Because such failures are so common management typically isn’t punished when they do so it’s hard to keep interests inline. And because many producers are run on a cost plus basis there can be a perverse incentive to do a bad job, or at least avoid doing a good one.

wesammikhail•34m ago
Agree 100%.

I know a lot of people on here will disagree with me saying this but this is exactly how you get an ecosystem like javascript being as fragmented, insecure, and "trend prone" as the old school Wordpress days. It's the same problems over and over and every new "generation" of programmers has to relearn the lessons of old.

mstipetic•24m ago
I was so annoyed when I found out the OTP library and realized we’ve been reinventing things for 20+ years
hackthemack•15m ago
I have a theory that the churn in technology is by design. If a new paradigm, new language, new framework comes out every so many years, it allows the tech sector to always want to hire new graduates for lower salaries. It gives a thin veneer of we want to always hire the person who has X when really they just do not want to hire someone with 10 years of experience in tech but who may not have picked up X yet.

I do not think it is the only reason. The world is complex, but I do think it factors into why software is not treated like other engineering fields.

bane•11m ago
I've also considered a side-effect of this. Each generation of software engineers learns to operate on top of the stack of tech that came before them. This becomes their new operating floor. The generations before, when faced with a problem, would have generally achieved a solution "lower" down in the stack (or at their present baseline). But the generations today and in the future will seek to solve the problems they face on top of that base floor because they simply don't understand it.

This leads to higher and higher towers of abstraction that eat up resources while providing little more functionality than if it was solved lower down. This has been further enabled by a long history of rapidly increasing compute capability and vastly increasing memory and storage sizes. Because they are only interacting with these older parts of their systems at the interface level they often don't know that problems were solved years prior, or are capable of being solved efficiently.

I'm starting to see ideas that will probably form into entire pieces of software "written" on top of AI models as the new floor. Where the model basically handles all of the mainline computation, control flow, and business logic. What would have required a dozen Mhz and 4MB of RAM to run now requires TFlops and Gigabytes -- and being built from a fresh start again will fail to learn from any of the lessons learned when it was done 30 years ago and 30 layers down.

malfist•7m ago
I work at $FANG, every one of our org's big projects go off the rails at the end of the project and there's always a mad rush at the end to push developers to solve all the failures of project management in their off hours before the arbitrary deadline arrives.

After every single project, the org comes together to do a retrospective and ask "What can devs do differently next time to keep this from happening again". People leading the project take no action items, management doesn't hold themselves accountable at all, nor product for late changing requirements. And so, the cycle repeats next time.

I led and effort one time, after a big bug made it to production after one of those crunches that painted the picture of the root cause being a huge complicated project being handed off to offshore junior devs with no supervision, and then the junior devs managing it being completely switched twice in the 8 month project with no handover, nor introspection by leadership. My manager's manager killed the document and wouldn't allow publication until I removed any action items that would constrain management.

And thus, the cycle continues to repeat, balanced on the backs of developers.

pphysch•5m ago
There are rational explanations for this. When software fails catastrophically, people almost never die (considering how much software crashes every day). When hardware fails catastrophically, people tend to die, or lose a lot of money.

There's also the complexity gap. I don't think giving someone access to the Internet Explorer codebase is necessarily going to help them build a better browser. With millions of moving parts it's impossible to tell what is essential, superfluous, high quality, low quality. Fully understanding that prior art would be a years long endeavor, with many insights no doubt, but dubious.

mdavid626•48m ago
It’s so “nice” to know, that trillions spent on AI not only won’t make this better, but it’ll make it significantly worse.
supportengineer•46m ago
The purpose of a system is what it does.

1. Enable grift to cronies

2. Promo-driven culture

3. Resume-oriented software architecture

mariopt•44m ago
> IT projects suffer from enough management hallucinations and delusions without AI adding to them.

Software is also incredibly hard, the human mind can understand the physical space very well but once we're deep into abstractions it simply struggles to keep up with it.

It is easier to explain how to build a house from scratch to virtually anyone than a mobile app/Excel.

dmix•37m ago
So in the 1990s Canada failed to do a payroll system where they paid Accenture Canada $70M

Then in 2010s they spent $185M on a customized version of IBM's PeopleSoft that was managed directly by a government agency https://en.wikipedia.org/wiki/Phoenix_pay_system

And now in 2020s they are going to spend $385M integrating an existing SaaS made by https://en.wikipedia.org/wiki/Dayforce

That's probably one of the worst and longest software failures in history.

bryanlarsen•28m ago
Oh, it's much more interesting than that. Phoenix started as an attempt to create a gun registry. Ottawa had a bunch of civil servants that'd be reasonably compotent at overseeing such a thing, but the government decided that it wanted to build it in Miramichi, New Brunswick. The relevant people refused to move to Miramichi, so the project was built using IBM contractors and newbies. The resulting fiasco was highly predictable.

Then when Harper came in he killed the registry mostly for ideological reasons.

But then he didn't want to destroy a bunch of jobs in Miramichi, so he gave them another project to turn into a fiasco.

bigbuppo•26m ago
As someone that has seen technological solutions applied when they make no sense, I think the next revolution in business processes will be de-computerization. The trend has probably already started thank to one of the major cloud outages.
add-sub-mul-div•19m ago
An endless succession of new tools, methodologies, and roles but failure persists because success is rooted in good judgment, wisdom, and common sense.
jmyeet•11m ago
This has dot-com bubble written all over it. But there are some deeper issues.

First, we as a society should really be scrutinizing what we invest in. Trillions of dollars could end homelessness as a rounding error.

Second, real people are going to be punished for this as the layoffs go into overdrive, people lose their houses and people struggle to have enough to eat.

Third, the ultimate goal of all this investment is to displace people from the labor pool. People are annoying. They demand things like fair pay, safe working conditions and sick leave.

Who will buy the results of all this AI if there’s no one left with a job?

Lastly, the externalities of all this investment are indefensible. For example, air and water pollution and rising utility prices.

We’re bouldering towards a future with a few thousand wealthy people where everyone else lives in worker housing, owns nothing and is the next incarnation of brick kiln workers on wealthy estates.

ThaDood•9m ago
So, I'm not a dev nor a project manager but I found this article very enlightening. At the risk of asking a stupid question and getting a RTFM or a LMGTFY can anyone provide any simple and practical examples of software successes at a big scale. I work at a hospital so healthcare specific would be ideal but I'll take anything.

FWIW I have read The Phoenix Project and it did help me get a better understanding of "Agile" and the DevOps mindset but since it's not something I apply in my work routinely it's hard to keep it fresh.

My goal is to try and install seeds of success in the small projects I work on and eventually ask questions to get people to think in a similar perspective.

Apt Rust requirement raises questions

https://lwn.net/SubscriberLink/1046841/5bbf1fc049a18947/
171•todsacerdoti•3h ago•263 comments

Launch HN: Onyx (YC W24) – The open-source chat UI

74•Weves•3h ago•62 comments

FLUX.2: Frontier Visual Intelligence

https://bfl.ai/blog/flux-2
60•meetpateltech•1h ago•18 comments

The 101 of Analog Signal Filtering

https://lcamtuf.substack.com/p/the-101-of-analog-signal-filtering
20•harperlee•4d ago•0 comments

Meta Segment Anything Model 3

https://ai.meta.com/blog/segment-anything-model-3/?_fb_noscript=1
151•alcinos•6d ago•33 comments

Human brains are preconfigured with instructions for understanding the world

https://news.ucsc.edu/2025/11/sharf-preconfigured-brain/
304•XzetaU8•10h ago•207 comments

Pebble Watch software is now open source

https://ericmigi.com/blog/pebble-watch-software-is-now-100percent-open-source
1173•Larrikin•22h ago•212 comments

Making Crash Bandicoot (2011)

https://all-things-andy-gavin.com/video-games/making-crash/
121•davikr•5h ago•8 comments

Orion 1.0 – Browse Beyond

https://blog.kagi.com/orion
122•STRiDEX•1h ago•52 comments

Trillions Spent and Big Software Projects Are Still Failing

https://spectrum.ieee.org/it-management-software-failures
51•pseudolus•5h ago•25 comments

Most Stable Raspberry Pi? Better NTP with Thermal Management

https://austinsnerdythings.com/2025/11/24/worlds-most-stable-raspberry-pi-81-better-ntp-with-ther...
234•todsacerdoti•10h ago•75 comments

Unpowered SSDs slowly lose data

https://www.xda-developers.com/your-unpowered-ssd-is-slowly-losing-your-data/
642•amichail•21h ago•260 comments

Ozempic does not slow Alzheimer's, study finds

https://www.semafor.com/article/11/25/2025/ozempic-does-not-slow-alzheimers-study-finds
4•danso•47m ago•0 comments

Brain has five 'eras' with adult mode not starting until early 30s

https://www.theguardian.com/science/2025/nov/25/brain-human-cognitive-development-life-stages-cam...
152•hackernj•3h ago•139 comments

Nearby peer discovery without GPS using environmental fingerprints

https://www.svendewaerhert.com/blog/nearby-peer-discovery/
39•waerhert•4d ago•17 comments

Broccoli Man, Remastered

https://mbleigh.dev/posts/broccoli-man-remastered/
96•mbleigh•5d ago•47 comments

Claude Advanced Tool Use

https://www.anthropic.com/engineering/advanced-tool-use
593•lebovic•22h ago•244 comments

Using an Array of Needles to Create Solid Knitted Shapes

https://dl.acm.org/doi/10.1145/3746059.3747759
58•PaulHoule•3d ago•13 comments

Explaining, at some length, Techmeme's 20 years of consistency

https://news.techmeme.com/250912/20-years
26•nhf•3d ago•10 comments

Show HN: I built an interactive HN Simulator

https://news.ysimulator.run/news
439•johnsillings•23h ago•198 comments

Three Years from GPT-3 to Gemini 3

https://www.oneusefulthing.org/p/three-years-from-gpt-3-to-gemini
330•JumpCrisscross•2d ago•259 comments

How the Atomic Tests Looked Like from Los Angeles

https://www.amusingplanet.com/2016/09/how-atomic-tests-looked-like-from-los.html
92•ohjeez•3d ago•67 comments

Windows GUI – Good, Bad and Pretty Ugly (2023)

https://creolened.com/windows-gui-good-bad-and-pretty-ugly-ranked/
56•phendrenad2•11h ago•119 comments

A million ways to die from a data race in Go

https://gaultier.github.io/blog/a_million_ways_to_data_race_in_go.html
114•ingve•3d ago•98 comments

Implications of AI to schools

https://twitter.com/karpathy/status/1993010584175141038
308•bilsbie•23h ago•355 comments

Cool-retro-term: terminal emulator which mimics look and feel of CRTs

https://github.com/Swordfish90/cool-retro-term
276•michalpleban•23h ago•104 comments

Build a Compiler in Five Projects

https://kmicinski.com/functional-programming/2025/11/23/build-a-language/
175•azhenley•1d ago•38 comments

Choosing a hash function for 2030 and beyond: SHA-2 vs. SHA-3 vs. BLAKE3

https://kerkour.com/fast-secure-hash-function-sha256-sha512-sha3-blake3
8•unsolved73•55m ago•5 comments

Show HN: OCR Arena – A playground for OCR models

https://www.ocrarena.ai/battle
189•kbyatnal•4d ago•57 comments

Rethinking C++: Architecture, Concepts, and Responsibility

https://blogs.embarcadero.com/rethinking-c-architecture-concepts-and-responsibility/
51•timeoperator•5d ago•54 comments