You cannot predict the future and chances are there will be some breaking change forced upon you by someone or something out of your control.
/v1/downloadFile
/v2/downloadFile
Is much easier to check for a v3 then
/api/downloadFile
/api/downloadFileOver2gb
/api/downloadSignedFile
Etc. Etc.
It's typically to declare bankruptcy on the entirety of /v1 and force eventual migration of everyone onto /v2 (if that's even possible).
For example dup(), dup2(), dup3() and pipe(), pipe2() etc
LWN has an article: https://lwn.net/Articles/585415/
It talks about avoiding this by designing future APIs using a flags bitmask to allow API to be extended in future.
What actually happens as the API grows-
First, the team extends the existing endpoints as much as possible, adding new fields/options without breaking compatibility.
Then, once they need to have backwards-incompatible operations, it's more likely that they will also want to revisit the endpoint naming too, so they'll just create new endpoints with new names. (instead of naming anything "v2").
Then, if the entire API needs to be reworked, it's more likely that the team will just decide to deprecate the entire service/API, and then launch a new and better service with a different name to replace it.
So in the end, it's really rare that any endpoints ever have "/v2" in the name. I've been in the industry 25 years and only once have I seen a service that had a "/v2" to go with its "/v1".
This is an interesting empirical question - take the 100 most used HTTP APIs and see what they do for backward-incompatible changes and see what versions are available. Maybe an LLM could figure this out.
I've been just using the Dropbox API and it is, sure enough, on "v2". (although they save you a character in the URL by prefixing "/2/").
Interesting to see some of the choices in v1->v2,
https://www.dropbox.com/developers/reference/migration-guide
They use a spec language they developed called stone (https://github.com/dropbox/stone).
One issue I have with weird resources are those that feel like unnecessary abstraction. It makes it hard for the human to read and understand intuitively, especially someone new to these set of APIs. Also, it makes it so much harder to troubleshoot during an incident.
If at all possible, take your time and dog-food your API before opening it up to others. Once it's opened, you're stuck and need to respect the "never break userspace" contract.
There’s a lot of things you can do with internal users to prevent causing a burden though - often the most helpful one is just collaborating on the spec and making the working copy available to stakeholders. Even if it’s a living document, letting them have a frame of reference can be very helpful (as long as your office politics prevent them from causing issues for you over parts in progress they do not like.)
Things that's missing from this list but that were important for me at some points:
1. Deadlines. Your API should allow to specify the deadline after which the request is no longer going to matter. The API implementation can use this deadline to cancel any pending operations.
2. Closely related: backpressure and dependent services. Your API should be designed to not overload its own dependent services with useless retries. Some retries might be useful, but in general the API should quickly propagate the error status back to the callers.
3. Static stability. The system behind the API should be designed to fail static, so that it retains some functionality even if the mutating operations fail.
It illustrates that the reminder isn't "never change an API in a way that breaks someone", it's the more nuanced "declare what's stable, and never break those".
So Linux is opinionated in both directions - towards user space and toward hardware - but in the opposite way
It is a bit ironic and a little funny that Windows solved this problem a couple decades ago with redistributables.
musl libc has a more permissive licence, but I hear it performs worse than GNU libc. One can hope for LLVM libc[1] so the entire toolchain would become Clang/LLVM, from the compiler driver to the C/C++ standard libraries. And then it'd be nice to whole-program-optimise from user code all the way to the libc implementation, rip through dead code, and collapse binary sizes.
But I don't think I've ever seen anybody actually do this.
For end points it’s a bit different. You don’t know what are they or user facing or programmer facing.
I wonder if someone has a good take on this. I’m curious to learn.
Why does the type of I/O boundary matter?
Some applications live in a single process, while others span processes and machines. There are clear differences, but also enough in common to speak of “APIs” for both
The format and protocol of communication was never fixed.
In addition to the rest api’s of today, soap, wsdl, web sockets could all can deliver some form of API.
Shudder...
In my circles this is usually (perhaps incorrectly) called REST API.
Versioning, etc. matter (or don’t) for binary UDP APIs (aka protocols) just as much as for any web API.
I'd like to introduce more fields or flags to control the behavior as params, not asking user to change the whole base url for single new API.
When an API commits to /v1 it doesn't mean it will deprecate /v1 when /v2 or /v3 come out, it just means we're committing to supporting older URI strategies and responses.
/v2 and /v3 give you that flexibility to improve without affecting existing customers.
Cursor based pagination (using the ID of the last object on the previous page) will give you a new list of items that haven't been viewed. This is helpful for infinite scrolling.
The downside to cursor based pagination is that it's hard to build a jump to page N button.
You can do some other cool stuff if they're opaque - encode additional state within the cursor itself: search parameters, warm cache / routing topology, etc.
But "API" means "Application Programming Interface". It was originally for application programs, which were... programs with user interfaces! It comes from the 1940's originally, and wasn't referred to for much else until 1990. APIs have existed for over 80 years. Books and papers have been published on the subject that are older than many of the people reading this text right now.
What might've those older APIs been like? What were they working with? What was their purpose? How did those programmers solve their problems? How might that be relevant to you?
I'm not sure how would storing a key in Redis achieve idempotency in all failure cases. What's the algorithm? Imagine a server handling the request is doing a conditional write (like SET key 1 NX), and sees that the key is already stored. What then, skip creating a comment? Can't assume that the comment had been created before, since the process could have been killed in-between storing the key in Redis and actually creating the comment in the database.
An attempt to store idempotency key needs to be atomically committed (and rolled back in case it's unsuccessful) together with the operation payload, i.e. it always has to be a resource-specific id. For all intents and purposes, the idempotency key is the ID of the operation (request) being executed, be it "comment creation" or "comment update".
This is not just true for authentication. If you work in a business setting, your APIs will be used by the most random set of users. They be able to google for how to call your api in python, but not be able to do things like converting UTC to their local time zone.
cyberax•6h ago
Sigh... I wish this were not true. It's a shame that no alternatives have emerged so far.
TrueDuality•5h ago
rahkiin•5h ago
marcosdumay•1h ago
The separation of a refresh cycle is an optimization done for scale. You don't need to do it if you don't need the scale. (And you need a really huge scale to hit that need.)
maxwellg•4h ago
If a client is accessing an API on behalf of itself (which is a more natural fit for an API Key replacement) then we can use client_credentials with either client secret authentication or JWT bearer authentication instead.
TrueDuality•2h ago
There doesn't need to be any OIDC or third party involved to get all the benefits of them. The keys can't be used by multiple simultaneous clients, they naturally expire and rotate over time, and you can easily audit their use (primarily due to the last two principles).
0x1ceb00da•48m ago
I never understood why.
TrueDuality•3m ago
1. Generate your initial refresh token for the user just like you would a random API key
2. The client sends the refresh token to an authentication endpoint. This endpoint validates the token, expires the refresh token and any prior bearer tokens issued to it. The client gets back a new refresh token and a bearer token with an expiration window (lets call it five minutes).
3. The client uses the bearer token for all requests to your API until it expires
4. If the client wants to continue using the API, go back to step 2.
The benefits of that minimal version:
Client restriction and user behavior steering. With the bearer tokens expiring quickly, and refresh tokens being one-time use it is infeasible to share a single credential between multiple clients. With easy provisioning, this will get users to generate one credential per client.
Breach containment and blast radius reduction. If your bearer tokens leak (logs being a surprisingly high source for these), they automatically expire when left in backups or deep in the objects of your git repo. If a bearer token is compromised, it's only valid for your expiration window. If a refresh token is compromised and used, the legitimate client will be knocked offline increasing the likelihood of detection.
Audit and monitoring opportunities. Every refresh creates a logging checkpoint where you can track usage patterns, detect anomalies, and enforce policy changes. You get visibility into which clients are actively using the API versus just sitting on old static keys. This also gives you natural rate limiting and abuse detection points.
Most security frameworks (SOC 2, ISO 27001, etc.) prefer time-limited credentials as a basic security control. Might not be relevant in the context of this post, but its an easy win.
Add an expiration time to refresh tokens to naturally clean up access from broken or no longer used clients. Example: Daily backup script. Refresh token's expiration window is 90 days. The backups would have to not run for 90 days before the token was an issue. If it was still needed the effort is low, just provision a new API key. After 90 days of failure you either already needed to perform maintenance on your backup system or you moved to something else without revoking the access keys.
pixelatedindex•5h ago
And what time frame is “long-lived”? IME access tokens almost always have a lifetime of one week and refresh tokens anywhere from 6 months to a year.
rahkiin•5h ago
cyberax•4h ago
OAuth flows are not at all common for server-to-server communications.
In my perfect world, I would replace API keys with certificates and use mutual TLS for authentication.
nostrebored•4h ago
I hate mTLS APIs because they often mean I need to change how my services are bundled and deployed. But to your point, if everything were mTLS I wouldn’t care.
cyberax•44m ago
Both, really. mTLS deployment is the sticking point, but it's slowly getting better. AWS load balancers now support it, they terminate the TLS connection, validate the certificate, and stick it into an HTTP header. Google Cloud Platform and CloudFlare also support it.
pixelatedindex•3h ago
cyberax•2h ago
pixelatedindex•1h ago
If you have sensitive resources they’ll be blocked behind some authz anyway. An exception I’ve seen is access to a sandbox env, those are easily generated at the press of a button.
cyberax•44m ago
Some way to break out of the "shared secret" model is needed. Mutual TLS is one way that is at least getting some traction.
smj-edison•4h ago
> ...You’re building it for a very wide cross-section of people, many of whom are not comfortable writing or reading code. If your API requires users to do anything difficult - like performing an OAuth handshake - many of those users will struggle.
Sounds like they're talking about onboarding specifically. I actually really like this idea, because I've certainly had my fair share of difficulty just trying to get the dang thing to work.
Security wise perhaps not the best, but mitigations like staging only or rate limiting seem sufficient to me.
pixelatedindex•3h ago