For an interesting example of the outcome of strongly optimising for essential complexity, look at the demoscene. They have done some amazing things that were thought to be impossible.
See the eternal urge of gamedevs to build engines instead of building the game.
And if you do stick to purpose-specific design and management asks for a seemingly minor expansion of scope, the “this is a small change, why did we make the system so rigid that it’s expensive to update?” is an uncomfortable conversation.
Everyone loves a bit of abstraction that encapsulates a given complexity and make it a single simple concept. This is not: "a complex bit of technology that has pros and cons and will lock you in a particular architecture and vendor and would almost always necessitate 2 architectures because not all tasks/scenarios would be suitable for it". It is a "Lambda". So simple. Just a `handle(event) {}`, how much simpler can this be? No complex runtimes, VMs, environments, etc. It's just a `handle(event) {}` no complexity to see here. good luck.
On the other hand, sometimes people who "champion" simplicity are also the same people who don't care about the engineering part of it. The "Hey, I added a full text search. It's implemented as a:
select *
from posts
where column1::text like $1
or column2::text like $1
...
or columnN::text like $1
Lets push it. Oh out database CPU is burning, search is so slow. What a surprise this is. I guess it validates the popularity of search. Can we optimize it? Maybe we cache the result? Maybe we add an LLM to normalize queries so we can cache results more effectively? I'm trying to keep things simple"There is also the requirement or ask that is trying to hide a particular complexity from one side by shoving it all under a rug somewhere. Once the complexity has been sufficiently shoved under the rug, we can think of the rug as just a rug. We can move it around and use it like a rug. Surely the shit underneath will never have any impact on anyone any time.
The hard part is in knowing which is which and I don't think anyone real has a tried and true solution for it.
I think because its really about correctly predicting the future in a n continually evolving environment.
Those "simple" solutions can be piled on top of each other turn into an absolute monstrosity over time.
And then engineered solution can be overengineered and overly complex when it turns out the problems it solves never ended up happening.
On top of that people don't even think of simplicity in the same way. One person's simple is another's overly complex.
What seems to work for me is try give yourself outs/alternative paths.
Hedge your bets by overengineering slightly on where you think there is more risk. And keep it simple where there is less risk. Then reevaluate.
If every dev took the path of least resistance, and did the easiest thing that did its task, we would lose 80% of all tech jobs. This would only funnel more money to billionaires and share holders instead of the employees. Why would we want that.
I am seeing a huge uptick in this style of "accidental" complexity.
I recently had the pleasure of observing a perfectly functional T-SQL script be refactored to use entity framework, resulting in a ~50x decrease in throughput.
Ignorance is something I am willing to entertain. Arrogance is way, way more difficult to deal with. One is an educational opportunity. The other is game of thrones.
Let's pat ourselves on the back. Their code is exciting and ego-driven. Our code is boring and adds value. They think they are facebook; we do not. Their complexity is accidental. Our complexity is essential. Their engineers should stop adding features until their codebase is like ours.
Also, framing it as "us vs them" misses the point. For instance, in my world (Data & Analytics), there's no universe where Spark makes sense for <1TB data or using Kafka for what could be solved using with more simple and predictable batch processing, yet we still see them used in wild implementations that clearly don't need that scale. Those implementations can end up needing a lot of headcount and budget for support without adding any value whatsoever and sometimes even ending up in a more brittle platform. Complexity often comes from misguided choices, not just ego.
I've seen enormous data teams at startups, by that I mean bigger than WhatsApp engineering team in its prime pre-Meta (~30 people). That's a red flag, especially when they're processing far less data.
I admit I did not read the article. But I have done so before I guess, I remember the headline.
The headline is enough to provoke thought. See
https://world.hey.com/dhh/merchants-of-complexity-4851301b
And after clicking, I see, this article is very recent. Strange...
Accidental complexity is compressable, but essential complexity is not. At some point, you cannot compress further without losing nuance.
In compiler design, there's a concept called the waterbed theory of complexity which states that you can try to abstract complexity away, but it'll just show up elsewhere.
• plain poor design
• complexity in operation rather than design (or technology)
• inescapable real-world complexity
• lots of moving pieces and details, straining working memory
• abstract or novel concepts that are hard to learn up-front, but easy longer-term
Some of these are practically opposites! At the pareto frontier of good design, there is a pretty fundamental trade-off between having more abstract concepts that are harder to learn up-front, and exposing more details that make systems hard to work with on an ongoing basis. People just call both of these "complexity"! These are two concepts that absolutely should not be conflated.
I've seen lots of other patterns that are misleadingly described as "complex". For example, some of the most effective software I've seen has been situated software[1]; that is, software built for, and largely in a specific social context. Think seemingly messy code that is incestuously coupled to a single person's or team's workflow. This software might actually be perfectly simple in context, but it's going to seem painfully baroque to anybody outside that context.
[1]: https://gwern.net/doc/technology/2004-03-30-shirky-situateds...
Given all I've seen, I've come to the conclusion that generic exhortations about "complexity" are actively harmful. If you're going to write a universally applicable rant, just write about how bad design is bad and good design is good! At least that's something that people will disagree with—I've met far more people who insist there is no such thing as "good" or "bad" design than people who insist that complexity is actually better than simplicity.
stego-tech•2h ago
It drives leadership nuts, but I find asking the "why" or "what problem does this solve" questions helps the organization remain focused on what actually delivers value versus what's just hype we can ignore or throw away.