Don’t reinvent the wheel on a tight deadline while the rest of the team does useful work. Don’t reinvent the wheel and then expect to be rewarded or respected as if you had really invented something.
I love my rabbit holes, but at work, it's often not viable to explore them given deadlines and other constraints. If you want your wheel to be used in production though, it better be a good wheel, better than the existing products.
Chesterton himself was using it as a religious metaphor, and I think most of us agree that software engineers are not literal gods.
Also, in software, you may find the fence sometimes exists in higher dimensions than the usual three, can turn invisible for part of its length, and occasionally exists but is four inches tall and blocks only mice. And it may also be designed, at least in part, by a blind idiot god called ChatGPT on whose altar consciousness itself is sacrificed. At this point it's worth considering whether keeping the fence around is worse than letting the bull out.
Of course mismanagement happens but the implied value of understanding why the fence was erected is to understand the expected value it would bring and understand the problem it was trying to solve. This does not imply that it should have been erected, just that there were others before you trying to solve a problem and if they're failing it's important to know why so you don't fail in the same way.
But it was also low-value data and even in the worst case we surmised the most we'd do is anger someone in marketing briefly.
I think so long as you can ascertain that the stakes are low, this is a good tactic.
Same can be said of learning a large inherited codebase. If you're assigned to a large piece of functionality and you need to understand it, your first instinct shouldn't be to rewrite the whole thing from scratch just to understand it; it should be to either read the code and documentation, or if those are impossible to comprehend, ask around what it's supposed to do, maybe even write tests to get a general sense of the behavior.
I don't expect programmers to be cognitive psychologists here but the thing about learning is that you must do a "progressive overload" of complexity as you go, which is why building on top of what exists is the best way to go. You get a general idea first of what something is supposed to do, and then when you're good enough, that's when you can work backwards to building something from scratch where all the complexity at lower levels of abstraction wouldn't be too much for your brain to handle. Starting with a large amount of complexity will impair your learning and only discourage you.
Personally, I’m a fan of learning by doing. Reinventing the wheel works better for me than just about anything I’ve tried.
The expense (time or otherwise) follows from how intimately you have to get to know the subject. Which is precisely why it's the best way to learn. It's not always viable, but when you can spare the expense nothing else compares.
True for life in general. We have limited lifespans. Aging and death really are the great grand daddy of all problems.
Yea, reinventing the wheel is a great way to learn. You're not going to hear an educator tell you to not reinvent the wheel.
That menas that almost by definition, if a library is popular, it contains huge amounts of code that just isn't relevant to your use case.
The tradeoff should be whether you can code your version quickly (assuming it's not a crypto library, never roll your own crypto), because if you can, you'll be more familiar with it and carry a smaller dependency.
> if a library is popular, it contains huge amounts of code that just isn't relevant to your use case.
It is true that many libraries do contain such code, whether or not they have dependencies. For example, SQLite does not have any dependencies but does have code that is not necessarily relevant to your use. However, some programs (including SQLite) have conditional compilation; that sometimes helps, but in many cases it is not suitable, since it is still the same program and conditional compilation does not change it into an entirely different one which is more suitable for your use.
Also, I find often that programs include some features that I do not want and exclude many others, and existing programs may be difficult to change to do it. So that might be another reason to write my own, too.
Maybe it shouldn't be necessary to have 120 different external libraries just to run bone-stock app? They should be placed in standard library.
It's okay to write a compiler or a database if you only know the basic principles, but it's not okay to write an encryption algorithm, or even an implementation of one, using only basic principles, because someone who knows more than you will break it.
For instance, were you aware that every time you write array[index], it leaks data to other threads in other processes on the same CPU, including JavaScript code running inside web browsers?
I really doubt people give it even a second thought.
The problems introduced in xz are still fresh, but Dual_EC_DRBG[0] also comes to mind within the cryptography context.
(Besides, getting cryptography right goes way beyond "just writing a library". As the parent commenter wrote, simple operations are the tip of the iceberg with regards to a correct implementation)
If it's in a standard library of an open source programming language, I'm less inclined to fully check the implementation.
That actually happened once.
I’d rather install date-fns or moment and let it decide what the fourth Sunday of a given month is in 2046, and also audit for the latest browser attack vectors.
popularity != bloat
So I had to write a simple queue, but since I wanted demos to work in the browser it has a IndexedDB backend, and I wanted demos it to work in an Electron app, so there is a SQLite backend, and I’ll likely want a multi-user server based one so there is a Postgres backend.
And I wanted to use it for rate limiting, etc, so limiters were needed.
And then there is the graph stuff, and the task stuff.
There are a lot of wheels to-create actually, if you don’t want any dependencies.
I do have a branch that uses TypeBox to make and validate the input and output json schemas for the tasks, so may not be dependency free for the core eventually.
Our reinvented wheel using posgresql, rabbitmq and EC2 runners has ~10x better throughput and scales linearly with the number of pending tasks, whereas the airflow falls apart and fails to keep the runners fully occupied the moment you out any real load on it.
Also, the dependencies often have a lot of extra "baggage," and I may only want a tiny bit of the functionality. Why should I use an 18-wheeler, when all I want to do, is drive to the corner store?
Also, and this is really all on me (it tends to be a bit of a minority stance, but it's mine), I tend to distrust opaque code.
If I do use a dependency, it's usually something that I could write, myself, if I wanted to devote the time, and something that I can audit, before integrating it.
I won't use opaque executables, unless I pay for it. If it's no-money-cost, I expect to be able to see the source.
But it is literally a one liner to declare a Pair<> type in java
``` record Pair<S, T>(S first, T second) {} ```
It might also change your psychological relationship with the dependecy. Instead of being disugsted by yet another external dependecy bringing poorly understood complexity into your project you are thankful that there exists a piece of code maintained and tested by someone else that does the thing you know you need done and lets you remove whole mess of complexity you yourself constructed.
This is also generally helpful when you have performance requirements, as often 3rd party code even when optimized in general, isn't very well optimized for any particular use case.
I find that in many cases you can cut out 80 percent of the code of the original library.
Most of the deleted code is flexibility and features we don't need.
It's surprising how small the essence of a solution can be.
When Primeagen was once interviewed, he built out a whole Java architecture; the interviewer asked him, "Have you heard of grep?" And that started a journey.
If it were to happen to me, feels like a full circle to go from glue and dependencies to pointers and data structures. A welcome reverie.
You will also rarely build a better implementation of these things than whatever is in the standard library or even some other library that already exists. If anything, it's better to, if one have a better idea, to contribute one's patches there.
A standard library data strucute or algorithm has to be something for everyone, so it can't be truly great at a specific thing. If you understand your specific use case extremely well it (and are competent...) it can be very easy to run circles around the standard library.
There is lot of complexity that mature wheels have taken into account or have had to solve and you are likely miss lot of it. Not that building your own does not help you to understand it.
Still, I wouldn't replace wheels on my car with ones I made myself from scratch... Just like I wouldn't replace reasonable complex library.
While sometimes reinventing the wheel is a useful exercise, as TFA lays out, this is often a symptom of a larger Not Invented Here mentality. This is generally a harmful tendency in some organizations that leads to derailing project deadlines, and misdirecting resources towards building and maintaining software that is not core to the company's mission.
So in most cases the advice to not reinvent the wheel is more helpful. Deciding to ignore it, especially within a corporate environment, should be backed by very good reasons.
Even if the open source alternatives already exist, does not necessarily mean that they do what you want them to do. In some cases they can be fixed to do what you need (since it is open source, that is an advantage), but sometimes it cannot really be done without rewriting it and making a new one.
As a team you constantly need to assess whether a dependency should be brought in or of a re-implementation is not better.
Sometimes, the re-implementation is so specific to you actual problem that maintenance becomes almost free. A generic solution always open more doors to problems and require more effort to maintain.
For a rich client web application, you need a really good reason not to bring an external dependency such as React.
Plus, most professionally programmed React apps from the last 10 years were created like this. Vite only replaced CRA as the recommended tool quite recently.
>3 million weekly downloads
Dear God.
[1] https://github.com/workofart/ml-by-hand
[2] https://github.com/workofart/ml-by-hand/blob/main/examples/c...
[3] https://github.com/workofart/ml-by-hand/blob/main/examples/g...
The advice as I would give it is:
"Try to re-invent the things you are interested in".
"Do not underestimate the value of the continued interaction between reality and established solutions."
I would agree about this 15 or 20 years ago, where I saw some monstrosities by people who didn't knew that frameworks or ORMs existed, but the pendulum has swung too far in the other direction today.
The "crappy square wheels" of 2025 are libraries that don't fully solve problems and are forced upon others by developers who are hostile to any approach that doesn't involve reaching out to a 20 github stars library from their package manager.
And this has become extremely difficult to discuss because the discussion has also become binary with people treating any criticism as a sign of NIH.
accept that there are compatibility boundaries such that it is sometimes quicker to create a new X than locate it on the market, or that X is too expensive and it's time to pursue vertical integration
but teams who can't do build vs buy properly are kind of doomed, sentenced to endless cycles of Not Invented Here syndrome which block other work.
if you're in a meeting and someone says 'we can't launch our website until we develop a platform-native way to host websites' you're in the wrong part of the curve
Against all odds, I not only succeeded (mostly thanks to ignorance and stubbornness), but my wheel turns out to be unbelievably good at what it does. Possibly even world-class. After further experimentation, it also enables feats that can only be described as pure heresy with troubling ease. Time passes and some people from that niche start picking up my wheel. They all hold it wrong at the beginning because it's so alien, but once they get the hang of it they never go back.
I get bug reports and feature requests from all over the world for the oddest of use-cases and workflows. I have deep, in-depth technical discussions with brilliant people I would've never met otherwise. I've witnessed achievements done by others with my wheel beyond my wildest dreams. I discover things that keep me awake at night. I get kicks out of melting down the brains of my uninitiated coworkers and colleagues explaining what my wheel does and what I can do with it.
Don't be afraid to reinvent the wheel. You never know what crazy, wild path it might roll you down to.
It's a Ghidra extension that can export relocatable object files from any program selection. In other words, it reverses the work done by a linker.
I originally built this as part of a video game decompilation project, having rejected the matching decompilation process used by the community at large. I still needed a way to divide and conquer the problem, which is how I got the funny idea of dividing programs. That allows a particular style of decompilation project I call Ship of Theseus: reimplementing chunks of a program one piece at a time and letting the linker stitch everything back together at every step, until you've replaced all the original binary code with reimplemented source code.
It's an exquisitely deep and complex topic, chock-full of ABI tidbits and toolchains shenanigans. There's next to no literature on this and it's antithetical to anything one might learn in CS 101. The technique itself is as powerful as it is esoteric, but I like to think that any reverse-engineer can leverage it with my tooling.
In particular, resynthesizing relocations algorithmically is one of those problems subject to the Pareto principle, where getting 80% of them right is reasonably easy but whittling down the last 20% is punishingly hard. Since I refuse to manually annotate them, I've had to relentlessly improve my analyzers until they get every last corner case right. It's by far the most challenging and exacting software engineering problem I've ever tackled, one that suffers no hacks or shortcuts.
Once I got it working, I then proceeded in the name of science to commit countless crimes against computer science with it (some of those achievements are documented on my blog). Cross-delinking in particular, that is delinking an artifact to a different platform that it originates from, is particularly mind-bending ; I've had some successes with it, but I sadly currently lack the tooling to bring this to its logical conclusion: Mad Max, but with program bits instead of car parts.
Ironically, most of my users are using it for matching decompilation projects: they delink object files from an artifact, then typically launch objdiff and try to create a source file that, when compiled, generates an object file that is equivalent to the one they ripped out of the artifact. I did not expect that to happen at all since I've built this tool to specifically not do this, but I guess when everything's a nail, people will manage to wield anything as a hammer.
A common challenge is decompiling and reverse engineering a compiled program, e.g. a game. The author realized that it's an interesting approach to do a sort of reverse-linking (or I guess unlinking) process of a program into pieces and then focusing on reverse engineering or reimplementing those pieces instead of the whole.
When it comes to decompiling games, enthusiasts in that hobby want to be able to reproduce/obtain source code that compiles to exactly the same instructions as the original game. I think there is usually also a differentiation between instruction matching and byte for byte matching reproduction. But it definitely seems like a simpler and better approach to do this piece by piece and be able to test with each new piece of progress.
That's my layman understanding of it having only dabbled in decompiling stuff.
The key insight of delinking is that object files are relocatable and the process of linking them together (laying their sections in memory, computing all the symbol addresses, applying all the relocations) removes that property. But by reversing this process (creating a symbol table, unapplying the relocations/resynthesizing relocation tables, slicing new sections), code and data can be made relocatable again.
Since properly delinked object files are relocatable, the linker can process them and stitch everything back together seamlessly, even if the pieces are moved around or no longer fit where they used to be (this makes for a particularly powerful form of binary patching, as constraints coming from the original program's memory map no longer apply). Alternatively, they can be fed to anything that can process object files, like a disassembler for example.
Of course, the real fun begins when you start reusing delinked object files to make new programs. I like to say that it breaks down the linear flow of toolchain from compilation to assembly to linking into a big ball of wibbly-wobbly, bitsy wimey stuff. Especially if you start cross-delinking to a different platform than the original pieces of the program came from.
That was never satisfying to me because recompilation (decompiling to source code or IR that can be recompiled for another platform) is an academic field of study [1], so the problem of converting a shared library to a static one should be easier. After all, you have all the assembly and symbol information right there, it seemed like all you needed to do was massage it the right way.
[1] https://rev.ng/
Shared objects are compiled in a way that the exact same code works when loaded at any address, which typically requires base+offset addressing and is less performant, but the same physical pages holding the object can be mapped into different processes at different addresses, ie shared.
Static object addresses the code and and data directly and does not waste a register and requires the addresses in its code to be modified depending on where it is loaded, and the code cannot be shared between different processes.
You can't convert a static library to shared because you can't.
You can convert a shared library to static but there is no point.
My understanding a shared library is closer to an executable than it is to an object file, so it is in that sense that it is "prelinked and ready to map into memory" just as an executable is. (Once loaded dyld still has to do fixups in the GOT, but that's not touching the text segment.)
I mostly agree with your definitions, but you _can_ create a shared library out of a static library assuming the static library was compiled with fPIC, no? I mean I've never tried it but most linkers seem to allow it [1], and given that creating a full executable is similar to created a shared library, I don't see why it wouldn't work.
And if the code wasn't compiled with fPIC, I still feel that it should be possible in theory since the whole point of object files is that they support relocation. Linking a non-PIC object to produce a shared object would just require patches within the text segment at runtime right? Modern linkers probably don't like doing this for security reasons. But otherwise seems isn't some fundamental limitation, just modern OSs and linkers not supporting an obscure use-case with security risk.
When people say that you can't convert a shared library to a static one (or equivalently statically link against a shared library), they usually cite the lack of relocation table info as the reason. Like they have some hierarchy in mind that goes from source code -> object file -> executable/shared library, and so they say it's "obvious" that you can't go the other way. Which is mostly true in terms of the "golden path" and what you'd want to limit yourself to for production, but in a hacker sense you can obviously go the other way with assemblers and decompilers.
So if someone claims that it's impossible to convert a shared library into a static library, there better be good justification for such a hard claim. And on the face of it, the claim doesn't seem watertight because the _stronger_ claim that you can't convert an executable back into recompilable code is falsified by the existence of decompilers that lower down to LLVM IR.
[1] https://stackoverflow.com/questions/655163/convert-a-static-...
My interpretation is it’s basically a hack because C programmers can’t make their libs reliable/memory safe, and it throws some logs under attacker’s feet.
There was this software called MagicErmine or statifier that could convert the two.
Tools like Statifier to convert an executable + its dylibs into a statically linked executable are sort of an in-between hack that does the equivalent of prelinking. But that's not converting a shared library to a static library, because the result isn't an object file that can be linked.
I guess I never stated the punchline explicitly, but boricj's tool seems to prove that there is in fact no such theoretical limitation in converting from shared -> static library. It's a bit hacky of course, but you can indeed reconstruct relocation tables with decompiler-like analysis to go back from executable -> object file for any function, and this also implies you can go from shared library -> static library.
As far as the traditional linker goes, sure. This whole shtick relies on the fact that a platform has ABI conventions, so with a naive toolchain any translation unit (and function) compiled in isolation must be interoperable as-is, because the linker is completely oblivious to whatever's going on within an object file section (relocations excluded). Even then, platform-specific considerations might get in the way.
For artifacts that went through inter-procedural and especially link-time optimization, this breaks down. The former will take advantage of the fact that anything goes as long as the visible interface of an object file section is ABI compliant, the latter effectively turns an entire program into one big translation unit. Among other things, functions may exhibit non-standard calling conventions, which wreaks havoc as soon as you start trying to meld delinked bits with freshly built ones, because the optimizer is unlikely to make the same choices in its custom calling conventions.
I have a user who is decompiling a link-time optimized artifact built over 15 years ago. In order to graft newly compiled code, they binary patched the original compiler to basically add support for __usercall, in order to coerce the toolchain into making the same optimization decisions. If you're up against an artifact built with a non-traditional toolchain with optimizations on, delinking alone most likely won't cut it and additional magic will be required.
I guess the one catch with shared libraries is that if shared library A itself depends on another dylib B, then calls within A to functions of B would go via PLT indirection. So then creating an object file out of this would involve not just moving code and creating relocation entry but also possibly patching the assembly itself to remove the call indirection.
Thinking about it a little bit, I'd say the major challenge for converting from shared to static library is that shared libraries have resolved relocations from text to other segments.
In order to create a static library, you need to make it so that text and data can be moved relative to each other again.
At a minimum, this requires disassembling the text segment to find PC-relative addresses and recreate relocations for them. That doesn't sound impossible, but there may be pitfalls I'm not thinking of right now that make it hard to find all necessary fix ups.
Shared libraries could be converted to static. You would lose the ability of static libraries to only include part of the library.
Windows in 16 and 32bit code defaults to non-PIC code and thus considerable work has been done to ensure system libraries do not overlap in addresses.
What happens when addresses overlap? When runtime linker detects conflicts (or just wants to load a non-PIC library/executable at different address), it utilizes "relocation data" that is shipped with the code which contains essentially information on "where are pointers and how to rewrite them when you move the code"
But yeah, one recurring problem is that without literature on this esoteric and complex topic, any discussion remotely technical about it (especially without prior context) quickly bogs down, as it needs to be bootstrapped from first principles in order to make any sense.
I've been meaning to write the Necronomicon of delinking to at least be able to point people to a document that contains everything they need to know in order to understand or at least decently grasp this. At the very least it should make for a good horror bed time story for linker developers.
As for the reinvention part, I was referencing the wheel of video game decompilation specifically. By that point Super Mario 64 was already fully decompiled and several other projects were well underway. The established wheel was matching decompilation (either instruction or byte level) and the community was producing tooling for this sole purpose.
In a sense, I did reinvent the wheel of video game decompilation in stark contrast to everything else that was out there at the time. It just turned out horribly right, because before I knew it I was making chimeras executables made out of bits of programs coming from different platforms.
By platforms you mean different binary formats, right? Not CPUs (just making sure I'm not going crazy here :))
I don't have the tooling to meld together incompatible ABIs. I tried once with two really similar ABIs (MIPS PIC and non-PIC) and while I've managed to work around incompatibilities with assembly trampolines, the debugging experience was so poor that I rage quit. Never tried with different CPUs.
I'm crazy enough to think it ought to be possible (there's plenty of prior art for most of the needed pieces), but I'm already stuck inside a rabbit hole so deep I can't see the bottom... I'll let someone else investigate that.
Also, my decompilation project's goal was a remaster in the spirit of REDRIVER2, with bugfixes, new features, quality of life improvements and a PC port. The end result would not have been matching or even functionally equivalent anyways.
https://people.scs.carleton.ca/~soma/pubs/bfoster-gecco-2010...
It does have the potential to simplify and improve the decompilation over directly decompiling a linked binary, as it more naturally reverses the process.
I’m definitely going to use this for solving some problems that i’m currently facing.
For anyone curious about what hare-brained scheme that one could hatch with this, i’d like the ability to do something like a PGO’d shared library —- watch the process run and exit, tracing all dlopen’s to create a new shared library with only the right functions from all the referenced libraries.
Hopefully this works, and if not, i’ll at least fail at something interesting :)
I've fallen into this trap enough times to recognize it as a trap now. Good for you that it works for you, but in general, automation of loosely-defined problems just doesn't get 100% of cases right and once it works, you need to streamline the manual process instead.
Given the amount of information loss that occurs at the linking stage, delinking is an extremely intricate puzzle to solve. It's also very hard to be sure that you got it right, because there are many, many ways it can go wrong in ways that are a nightmare to troubleshoot. The scale of the endeavor can be also staggering: one user has over 250 000 relocation spots in their artifact, pretty much all of them must be spot-on otherwise glitches and crashes will occur.
The only way I've managed to pull it off is with an analyzer design that makes no assumptions, double-checks everything, logs anything that couldn't be successfully processed and heavily investing into regression testing. The manual process then consists of fixing inaccuracies within the Ghidra database, or (rarely nowadays) fix an unhandled corner case within the analyzer. Also, no manual annotations of relocations means they cannot go stale as the Ghidra database is modified.
I also happened to attempt this first on possibly the worst architecture possible, MIPS. I won't go into the details, but split HI16/LO16 relocations, register dependency graphs and branch delay slots can produce some absolutely gnarly cases to untangle. Let's just say that my MIPS analyzer contains among other things one recursive function that takes 6 parameters, 4 of them shifts one place to the left at each iteration. I'm not exactly sure how it works.
Why do you add a dependency? Because you need a certain functionality.
The alternative to adding this dependency is to write the code yourself.
That is totally feasible for a lot of dependencies.
That is totally infeasible for a lot of dependencies.
It's a trade off, as always. The fact is, that most of us, who do this for a living, need to ensure that our software runs and continues to run and may be adapted in a timely manner, when new requests come in from the client/owner.
Using dependencies or not using depdencencies isn't gonna change that.
Now, granted, some ecosystems are a bit extreme on the "let's add a dependency for this 1 line of code."
On the other hand: should I really roll my own crypto? (Jk)
Should I really write those 5000 lines of well tested code myself, just because I can? And because MAYBE it's not touched in a month from now?
Every (later executed) line I add to the project, be it one written by myself or an external dependency, becomes part of the code. So I have to be able to maintain it. Might be easier if I write it myself. Might be way more difficult and time consuming if I write it myself...
I have so many tasks to do and while I enjoy coding, I have to make my top priority a working system.
So should I mindlessly add dependencies? Of course not!
Should I just reinvent the whole world? Of course not! (Unless the client absolutely wants me to. And pays me to do it)
When I give this advice it usually means I don't think the output is better than the existing thing and the dependency cost is better paid in the form of integration.I probably don't think you'll really maintain your creation or think about others using it when you do.
As long as we are throwing shitty incentives around.
But on a more neutral note, it's a tradeoff with many moving parts. Different choices for different scenarios.
It's always internet strangers who are too dogmatic even though there's zero context.
People might add a redundant or dangerous library because they don’t know better but they don’t fight for it, like here.
> unless the particular wheel you're reinventing is core to the product/service your startup makes
If it's core to your product/service, absolutely reinvent the hell out of that wheel. That's your business. Put pneumatic spokes on it. Add a touchscreen. Embed a bespoke LLM. Go nuts.
Just stay focused on your customers. Don't do a bunch of things that are frankly just resume builders for your senior engineers and don't add value for your customers.
At our company we reinvented a handful of wheels because the existing solutions were too complex.
And complexity slows you down.
It's not this clear cut.
Your job as the decision-making engineer is to develop the expertise to be able to make the right choice more than 95% of the time.
I'm all for reusing frameworks, standard libraries, crypto. Sure, those are things we don't want to recklessly reinvent.
But there's a limit. There should be way more pushback against things like this: https://news.ycombinator.com/item?id=39019001
I’ve certainly done it at work because I didn’t have time (or desire) to learn a library.
But sometimes you have to understand the wheel to reinvent it.
I’m not gonna let a someone roll their own user authentication and authorization system in a rails application.
There’s so many moving pieces you won’t even think about and there’s gems that solve this extremely well. Study the gem and you will learn a lot more. Then reinvent the wheel.
especially in an industry context where you are being paid (there’s that pesky denominator again). R&D, which is what this is, is a very expensive and risky endeavor. 99% of startup projects fail, for example. They would not fail given infinite time, but you cant have that because time is not free:
Interest Is The Price Of Time
A great friend of mine once told me the following quote from an unknown author: "Reinvent the wheel, not because we need more wheels but because we need more inventors." That quote has brought my mind and heart to some modicum of tranquility at various occasions when I wanted to learn some concept and resolved to write my own "toy version" of a software library, framework et cetera. Later on, when I learned about Feynman's quote “What I cannot create, I do not understand”, amalgamated the sentiment that it is okay to build something in order to learn one concept. I have thus far learned that in every new journey to reinvent the wheel, so to speak, often led me to paths where my intuitions about the initial concept got stronger and beyond that, I learned several other concepts.
Many of my side projects are just my own version of things that are specifically tailored to my preferences. I have a notes web app that backs up to S3. I have my own music streaming service. I have my own music discovery algorithm.
Sure these things exist, but because I made them, they do exactly what I want them to do. There is no one "wheel" - there are infinite permutations of what a "wheel" can be.
A huge benefit to not re-inventing is that standardization is a big value. Compare auto tires to phone batteries (back when they were replaceable). Auto tires are standardizes whereas every phone used its own slightly differently shaped battery.
> those who tried to invent a wheel themselves and know how hard it is
> those who never tried to invent a wheel and blindly follow the advice
There's a third, and I think more common group: folks who know all that's involved with reinventing the wheel, and how to do it, and know the juice of doing it isn't worth the squeeze, and there's no value in doing it yourself, educational or otherwise.
I think this view of things is usually related to the fact that large parts of this group are simply too bad at reinventing wheels and in large part are gun-shy because of previously failed attempts at doing a good job.
One of my former leads had worked for 7+ years in the same company by the time I showed up and he was absolutely useless as a programmer beyond just gluing together libraries because that's all he had ever done. He barely also knew the platform we ran on because he simply had developed no skill when it came to understanding things.
For him and people like him reinventing wheels is simply not worth the time and since they're so bad at it and always fail they'll never actually learn anything. I heard from him last year (7 years after we both left the company we worked together at) and he still only glues libraries together, but he does it in Rust instead of Erlang nowadays.
How often do I see people metaphorically trying to use a car tire on bicycle with thee excuse of not re-inventing the wheel. There can be great benefits for the parts of your system to be tailor made to work together.
- wheels that force you to do things that way the maker wants, instead of the way that suits you;
- wheels that are fine going downhill ... but nowhere else;
- wheels require you to attach an ever more expensive widget;
- wheels that promise but never quite deliver;
- wheels that keep pushing you into services that you just know are going to be a pain.
Often your own wheel is the best.
- A Formula 1 wheel when all you need is a bicycle wheel, but the person in charge of choosing wheels chooses it on the basis of “if we want to be as good as Formula 1, then we need to use the same wheels as Formula 1”
Sometimes there are very good reasons to reinvent the wheel, where the buisness demands a faster x, or there are privacy concerns for y. But most of the time its just a bad idea to re-invent the wheel. Do that shit on your own time, and keep it out of our production codebase.
I know there will be lots of people moaning about curiosity and the like. I somewhat agree. However imagine you are building a new table. Nothing interesting just metal frame with a composite wooden top. You are prototyping is, and suddenly, one of your engineers decides to reinvent screws. Not because there is a need, but because they are bored.
Its not like you need hidden, or fancy screws/attachement devices, the engineer is just curious about trying out a lathe. Not that you have an "automatic" lathe, so it not like you can scale up production.
Its fucking stupid right? yes.
Sure, test out how to make your own fasteners. Even do it on company time. Just don't try and sneak it into production. Its a bad idea.
> A: While I am a strong believer in the future of the rotary dial as an input device, Linus Torvalds might disagree
* HHS -> FDA -> CBER
It's important IMO (IMO only NOT AN EXPERT) because it helps you understand first principles better. As fundamentals change, its helps me to reevaluate these things even though I know nothing will ever come of them.
I am 422 agencies in so far, hoping to finish in-time for Juneteenth. Cant post her because........... but yea.
Re-invent the wheel!
This baffles me, because a basic HTTP/1.0 server is about 20 lines of printing certain human-readable text over a socket in Perl, and I'm not cheating by just importing a library that does all the things.
I know because I have a www.pl script sitting around that I've frequently using for testing and such. I'm sure it violates various parts of the RFCs in different ways, but... sometimes that's what I'm testing anyway, and when I do want it to work, it simply prints file contents over a socket with a content-type header based on the file extension. It's even smart enough not to be vulnerable to directory traversal (i.e. it'll only serve files from the content directory), etc.
Sure, that's in some sense cheating by leaving out a lot of hard parts from HTTP/1.1 or 2.0, but really, you're just reading a string coming over a network socket, responding with another string, and then closing the connection with HTTP/1.0. You're really just worried about printing out the right thing in response to input from the client.
It's not magic and it's not even very complicated. At least crypto has a lot of formulas to keep track of, timing attacks to worry about, etc. You can have a simple web server that's too dumb to be vulnerable to most web attacks with a few lines of scripting, most of which involve formatting strings and printing them.
> There are great reasons to reinvent the wheel:
> Build a better wheel (for some definition of better)
You can invent an improvement, no need to reinvent the existing wheel
> Learn how wheels are made
Read a book
> Teach others about wheels
Read a book and tell others about it
> Learn about the inventors of wheels
Read a history book
> Be able to change wheels or fix them when they break
Read an engineering manual
etc.
I've often found that it was not until I tried to build that model for real that I had missed significant crucial parts, not just a matter of 'incompleteness' or a too high level abstraction, but that my model no matter how elegant was just wrong or unworkable.
It's fine to invent your own wheel. But that doesn't mean you should put it in production or feel any entitlement for anybody else to use it there, just because you put personal effort into it. It's going to need to be at least as good, if not better, not just in whatever novel genius way you made it unique, but in all the boring ways - testing, documentation, supportability, etc.
Ancient wheels supported weight through compression: the cart pressed down, the wheel transferred that force straight into the ground. Simple and solid.
Modern wheels? Totally different. Bicycle wheels hold weight through tension. If a bike is standing still, you could cut the bottom spokes and it would stay upright. Cut the top ones and it collapses—the rim is literally hanging from the hub.
Even car tires follow this principle. It’s not just a balloon of air under the wheel. The internal pressure forms a tensioned ring that holds the car up. The car is suspended from the tire, not resting on it.
So while the article encourages reinventing the wheel metaphorically, I can’t help but admire how we’ve already reinvented the wheel literally—by flipping the physics entirely.
If you're visual like me, this video illustrates it perfectly: https://youtu.be/8I7QsgYm_NU?si=Vz1aqpHBAPjayqlG
But in ubiquitous solutions, we also find assumptions and trade-offs that require constraining the application to fit the solution. Thus it is there, in those assumptions and trade-offs, as these literal wheel examples demonstrate, where we find the ability to usefully reinvent.
I have to laugh a little seeing your reference to Mecanum wheels and your post being dated 6 hours ago. 12 hours ago, I rewatched the Star Trek reboot film. In it, there is a scene where the characters are all loading into shuttles to head off to their assignments. In the background, we are supposed to be impressed with the technical advancement of the machinery in view, e.g. how something as familiar as a forklift has "futuristic" details like strange wheels. And those wheels on that forklift are Mecanum wheels. In-story, that makes them something like ~300 year old technology.
That said there are also contexts in which the existing system that was built sucks so bad that rewriting it usually a boon, even if the new wheel sucks, it sucks less from the start.
You at a minimum should engage with the existing wheels and their users to find the ways in which they do and don't work.
In your own time I think it is great to tinker, pull apart, assemble your own things. Every Jedi makes her own light saber right?
In particular, Google-scale frameworks are usually harmfully over-engineered for smaller companies. Their challenge is solving the Google scale, not the problem at hand. At the same time, their complexity implies that a small-scale solution will likewise require a hundred-programmer team to implement.
But I find that far from the truth. In fact, many problems benefit from a purpose-built, small-scale solution. I've seen this work out many times, and result in much simpler, easier-to-debug code. Google-scale frameworks are a very bad proxy for estimating the complexity of a task.
- you can afford it (money)
- you have enough time or your customer can wait
- provide significant benefit to you or your customers (have nothing better to do)
- you don't have fierce competition in your domain
- you are pretty much retired (or could) and treating it more like art/craft/passion project
I compare it with building some custom made designer house and furniture. Many would want that, only some can afford it, even less will pay for it.
Back in 2010, I started an open source project for identifying audio files based on fingerprints. I specifically designed everything, so that in the end I need to do super fast i32->i32 lookups on scale. There were many inverted indexes, but they are usually for text search, which has a lot of overhead I don't need. I ended up writing a completely custom solution, and it allowed me to scale to millions and millions of fingerprints on such small servers that it was unimaginable to anyone who ever ran audio fingerprinting system before.
As others said reinventing the wheel is fantastic for learning, but not so for production / real life stuff.
So for example implementing your own transformers from scratch - great for learning but for 99% it would not meaningfully change how you use LLMs for mundane things.
So only when the domain is really really meaningful to you, you could consider reinventing the wheel and even then do not expect any immediately payoffs.
Those 4000 weeks pass by awfully quickly.
You have to pick your battles / wheels.
I think every codebase needs a healthy and pragmatic balance of "this particular wheel we can and should build ourselves, and it'll work better (for us at least!) than any other wheels out there" vs "there is a standard wheel for this, 10 million other horse-and-carriages out there use the same wheel, it's not worth us using a non-standard wheel for this thing".
Often, it is to give a variant proposal to something, rarely a bad thing.
For, instance in software, alternative implementations are somewhat critical. It is actually a necessity "in the current economy" in order to sanely "work".
Reinvent means changing something, hopefully for the better:
so long as you have the time and your manager isn't rushing you to deploy that feature "ASAP" ;)
Nevermind that a much simpler and efficient piece is possible, we have to ship ASAP so don't bother to spend any time on it.
Include a whole set of dependencies just because we need a function or two in a huge library or framework. Worry about optimizations later (i.e. never).
Reinventing the wheel can be seen as an attempt to escape these local optima, exploring new approaches that better suit specific needs or constraints. This process involves effort and risk, just like tweaking optimization parameters in training, but it can lead to innovative and more tailored solutions.
So, rather than blindly accepting existing solutions, sometimes it’s worth challenging them and trying to find a different path—just as in deep learning, where escaping local optima can improve model performance.
How did we sink this low?
moron4hire•1mo ago
People who say "don't reinvent the wheel" should come up with something new to say.
But they can't, because they hate innovation so much.
mcnamaratw•1mo ago
hoppp•1mo ago
I think reinventing the wheel would be like creating a wheel that does not spin or somethin... some youtubers have done it actually. Its fun to mess around with but hard to create something valuable.
Ekaros•1mo ago