Author here! Agreed that are different levels of "engaged" from users, which is okay. The concerning part of this finding is that even dependent users that I know to be highly engaged didn't respond to the deprecation warnings, so they're not working for even the most engaged users.

Why is it that CI tools don't make warnings visible? Why are they ignored by default in the first place? Seems like that should be a rather high priority.

You can turn warnings into errors with the `-Werror` option. I personally use that in CI runs, along with the `-X dev` option to enable additional runtime checks. Though that wont solve the author's problem, since most Python devs don't use either of those options

> If you are a good developer, you'll have extensive unit test coverage and CI. You never see the unit test output (unless they fail) - so warnings go unnoticed.

In my opinion test suites should treat any output other than the reporter saying that a test passed as a test failure. In JavaScript I usually have part of my test harness record calls to the various console methods. At the end of each test it checks to see if any calls to those methods were made, and if they were it fails the tests and logs the output. Within tests if I expect or want some code to produce a message, I wrap the invocation of that code in a helper which requires two arguments: a function to call and an expected output. If the code doesn't output a matching message, doesn't output anything, or outputs something else then the helper throws and explains what went wrong. Otherwise it just returns the result of the called function:

  let result = silenceWarning(() => user.getV1ProfileId(), /getV1ProfileId has been deprecated/);
  expect(result).toBe('foo');

This is dead simple code in most testing frameworks. It makes maintaining and working with the test suite becomes much easier as when something starts behaving differently it's immediately obvious rather than being hidden in a sea of noise. It makes working with dependencies easier because it forces you to acknowledge things like deprecation warnings when they get introduced and either solve them there or create an upgrade plan.

When I update python version, python packages, container image, etc for a service, I take a quick look at CI output, in addition to the all the other checks I do (like a couple basic real-world-usage end-to-end usage tests), to "smoke test" whether something not caught by outright CI failure caused some subtle problem.

So, I do often see deprecation warnings in CI output, and fix them. Am I a bad developer?

I think the mistake here is making some warnings default-hidden. The developer who cares about the user running their the app in a terminal can add a line of code to suppress them for users, and be more aware of this whole topic as a result (and have it more evident near the entrypoint of the program, for later devs to see also).

I think that making warnings error or hidden removes warnings as a useful tool.

But this is an old argument: Who should see Python warnings? (2017) https://lwn.net/Articles/740804/

If you are a good developer, you consider warnings to be errors until proven otherwise.

There was this one library we depended on, it was sort of in limbo during the Python 2 -> 3 migration. During that period is was maintained by this one person who'd just delete older versions when never ones became available. In one year I think we had three or four instances where our CI and unit tests just broke randomly one day, because the APIs had changed and the old version of the library had been yanked.

In hindsight it actually helped us, because in frustrations we ended up setting up our own Python package repo and started to pay more attention to our dependencies.

That's why you, very early on, release code that slows the API down once it has been deprecated. Every place you issue a deprecation warning, you also sleep 60. Problem solved.

OS software maintainers don't like maintaining legacy ugly APIs forever and want to refactor/remove legacy code to keep themselves sane and the project maintainable.

I think this is a valid question for this specific case, but may not always be possible. That said, I think as a user I would probably prefer it if under the hood the old function called the new so they can deprecate the behavior without breaking the API. In that way you can still emit the deprecation warning while also only having one actual code path to maintain.

+1

The industry has a solution for the exact problem the urllib is having (semver). Urllib just actively refuses to use it.

Python itself doesn't use semver.

This, I think, is the crux of the matter.

As an example, I always knew urllib3 as one of the foundational packages that Requests uses. And I was curious, what versions of urllib3 does Requests pull in?

Well, according to https://github.com/psf/requests/blob/main/setup.cfg, it's this:

    urllib3>=1.21.1,<3

That is exactly the kind of dependency specification I would expect to see for a package that is using semver: The current version of urllib3 is 2.x, so with semver, you set up your dependencies to avoid the next major-version number (in this case, 3).

So, it seems to me that even the Requests folks assumed urllib3 was using semver.

If you (either directly or from SerpApi) are supporting the urllib3 folks (through a Tidelift subscription), then yes, that is a valid point.

Otherwise, I'd say that's a very brave comment you are making.

> Fixing deprecations is unfortunately the lowest prio of any kind of work for majority of the projects.

... and the right answer to that is to make it entirely their problem.

> Part of the problem is probably lack of pressure to do so it if the timeline is unclear. What if this is actually never removed?

In this case, the warnings said exactly what release would remove the API. Didn't help.

> Why going through the pain?

Because you're not a feckless irresponsible idiot? I don't think it's an accident that the projects they said didn't react were an overcomplicated and ill-designed management layer for an overcomplicated and ill-designed container system, a move-fast-and-break-things techbro company, and what looks to be a consolation project for the not-too-bright.

You probably get an extra measure of that if you're operating in the Python ecosystem, which is culturally all about half-assed, 80-percent-right-we-hope approaches.

The right answer is to remove it when you say you're going to remove it, and let them pick up the pieces.

It also helps if you design your API right to begin with, of course. But this is Python we're talking about again.

> After 12/24 months, cut a new semver-major release. People notice the semver-major through the dependency management tools at some point, an maybe they have a look at changelog.

The urllib3 package doesn't use SemVer.

This will just waste CI compute and not solve anything.