frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

Open in hackernews

Making PyPI's test suite 81% faster – The Trail of Bits Blog

https://blog.trailofbits.com/2025/05/01/making-pypis-test-suite-81-faster/
67•rbanffy•3d ago

Comments

ustad•4h ago
The article uses pytest - does anyone have similar tips when using pythons builtin unittest?
masklinn•4h ago
The sys.monitoring and import optimisation suggestions apply as-is.

If you use standard unittest discovery the third item might apply as well, though probably not to the same degree.

I don’t think unittest has any support for distribution so the xdist stuff is a no.

On the other hand you could use unit test as the API with Pytest as your test runner. Then you can also use xdist. And eventually migrate to the Pytest test api because it’s so much better.

kinow•3h ago
I wwsn't familiar with this sys.monitoring option for coverage. Going to give it a try in my test suite. At the moment with docker testcontainers, gh actions test matrix for multiple python versions, and unit + regression + integration tests it is taking about 3-5 minutes.
darkamaul•3h ago
Warning, there is a change in coverage 7.7.0 that disables sysmon support for coverage if using branch coverage _and_ a version of Python before 3.14alpha6.

[0]: https://coverage.readthedocs.io/en/7.8.0/changes.html#versio...

kinow•1h ago
Ah, thank you! I think you just saved me some time!
anticodon•1h ago
I profiled a huge legacy tests collection using cProfile, and found lots of low hanging fruits. Like some tests were creating 4000x3000 Pillow image in memory just to test how image saving code works (checkign that filename and extension are correct). And hundreds of tests had created this huge image for every test (in the setUp method) because of unittest reliance on inheritance. Reducing size image to 10x5 made the test suite faster for like 5-7% (it was long time ago, so I don't remember exact statistics).

So, I'd run the tests under cProfile first.

cocoflunchy•4h ago
I don't understand why pytest's collection is so slow.

On our test suite (big django app) it takes about 15s to collect tests. So much that we added a util using ripgrep to find the file and pass it as an argument to pytest when using `pytest -k <testname>`.

piokoch•3h ago
Ehhh, those pesky Python people, complaining and complaining, average Spring Boot application takes 15s to start even looking if the code compiled ;)
thom•3h ago
Lest we start to malign the JVM as a whole, my Clojure test suite, which includes functional tests running headless browsers against a full app hitting real Postgres databases, runs end to end in 20s.
ffsm8•41m ago
The spring tests are generally quicker then the equivalent python test, so ime - the jvm is mostly to blame.

How much time actually goes by after you click "run test" (or run the equivalent cli command) until the test finished running?

Any projects using the jvm I've ever worked on (none of which were clojure, admittedly) have always taken at least 10-15s until the pre-phases were finished and the actual test setup began

thom•10m ago
If I completely clear all cached packages maybe, but I never do that locally or in CI/CD, and that's true of Python too (but no doubting UV is faster than Maven). Clojure/JVM startup time is less than half a second, obviously that's still infinitely more than Python or a systems language but tolerable to me. First test runs after about 2s? And obviously day to day these things run instantly because they're already loaded in a REPL/IPython. Maybe unfair to compare an interpreted language to a compiled one: building an uberjar would add 10 seconds but I'd never do that during development, which is part of the selling point I guess. Either way, I don't think the JVM startup time is really a massive issue in 2025, and I feel like whatever ecosystem you're in, you can always attack these slow test suites and improve your quality of life.
kinow•3h ago
In their case I think they were no specifying any test path. Which would cause pytest to search or tests in multiple directories.

Another thing that can slow down pytest collection and bootstrap is how fixture are loaded. So reducing number or scope of fixtures may help too.

boxed•1h ago
I've done some work on making pytest faster, and it's mostly a case of death by a thousand paper cuts. I wrote hammett as an experimental benchmark to compare to.
throwme_123•2h ago
Is Trail of Bits transitioning out of "crypto"?

Imho, they are one of the best auditors out there for smart contracts. Wouldn't be surprising to see some of these talented teams find bigger markets.

bgwalter•2h ago
I get that pytest has features that unittest does not, but how is scanning for test files in a directory considered appropriate for what is called a high security application in the article?

For high security applications the test suite should be boring and straightforward. pytest is full of magic, which makes it so slow.

Python in general has become so complex, informally specified and bug ridden that it only survives because of AI while silencing critics in their bubble.

The complexity includes PSF development processes, which lead to:

https://www.schneier.com/blog/archives/2024/08/leaked-github...

williamdclt•1h ago
> it only survives because of AI

I don't disagree that it's "complex, informally specified" (idk about bug ridden or silencing critics), but it's just silly to say it only survives because of AI. It was a top-used language before AI got big for web development, data science and all sorts of scientific analysis, and these haven't gone away: I don't expect Python lost much ground in these fields, if any.

bgwalter•1h ago
Dropbox moved parts from Python to Golang already in 2014. Google fired the Python team last year and I hear that it does not use Python for new code. Instagram is kept afloat by gigantic hacks.

The scientific ecosystem was always there, but relied on heavy marketing to academics, who (sadly) in turn indoctrinate new students to use Python as a first language.

I did forget about sysadmin use cases in Linux distributions, but they could be easily replaced by even Perl, as leaner BSD distributions already do.

NeutralForest•1h ago
Pretty good article, it's really a challenge to properly isolate DB operations during testing so having a difference instance per worker is nice. I remember trying to use different schemas (not instances) but I had a hard time to isolate roles as well.

I ruined my vacation by reverse engineering WSC

https://blog.es3n1n.eu/posts/how-i-ruined-my-vacation/
196•todsacerdoti•7h ago•79 comments

Plain Vanilla Web

https://plainvanillaweb.com/index.html
1092•andrewrn•18h ago•513 comments

Continuous Thought Machines

https://pub.sakana.ai/ctm/
182•hardmaru•8h ago•15 comments

Intellect-2 Release: The First 32B Model Trained Through Globally Distributed RL

https://www.primeintellect.ai/blog/intellect-2-release
131•Philpax•9h ago•37 comments

Armbian Updates: OMV support, boot improvents, Rockchip optimizations

https://www.armbian.com/newsflash/armbian-updates-nas-support-lands-boot-systems-improve-and-rockchip-optimizations-arrive/
18•transpute•3h ago•0 comments

Making PyPI's test suite 81% faster – The Trail of Bits Blog

https://blog.trailofbits.com/2025/05/01/making-pypis-test-suite-81-faster/
67•rbanffy•3d ago•18 comments

Why Bell Labs Worked

https://1517.substack.com/p/why-bell-labs-worked
223•areoform•14h ago•165 comments

Dart added support for cross-compilation

https://dart.dev/tools/dart-compile#cross-compilation-exe
25•Alifatisk•3d ago•21 comments

Car companies are in a billion-dollar software war

https://insideevs.com/features/759153/car-companies-software-companies/
353•rntn•17h ago•604 comments

Absolute Zero Reasoner

https://andrewzh112.github.io/absolute-zero-reasoner/
82•jonbaer•4d ago•16 comments

High-school shop students attract skilled-trades job offers

https://www.wsj.com/lifestyle/careers/skilled-trades-high-school-recruitment-fd9f8257
194•lxm•19h ago•310 comments

Show HN: Vom Decision Platform (Cursor for Decision Analyst)

https://www.vomdecision.com
5•davidreisbr•3d ago•3 comments

The Academic Pipeline Stall: Why Industry Must Stand for Academia

https://www.sigarch.org/the-academic-pipeline-stall-why-industry-must-stand-for-academia/
103•MaysonL•8h ago•79 comments

Scraperr – A Self Hosted Webscraper

https://github.com/jaypyles/Scraperr
193•jpyles•16h ago•68 comments

Ask HN: Cursor or Windsurf?

149•skarat•6h ago•199 comments

Writing an LLM from scratch, part 13 – attention heads are dumb

https://www.gilesthomas.com/2025/05/llm-from-scratch-13-taking-stock-part-1-attention-heads-are-dumb
284•gpjt•3d ago•57 comments

Title of work deciphered in sealed Herculaneum scroll via digital unwrapping

https://www.finebooksmagazine.com/fine-books-news/title-work-deciphered-sealed-herculaneum-scroll-digital-unwrapping
214•namanyayg•21h ago•96 comments

A formatter for your kdl files

https://github.com/hougesen/kdlfmt
3•riegerj•3d ago•1 comments

Why alien languages could be far stranger than we imagine Essays

https://aeon.co/essays/why-alien-languages-could-be-far-stranger-than-we-imagine
7•rbanffy•1h ago•6 comments

One-Click RCE in Asus's Preinstalled Driver Software

https://mrbruh.com/asusdriverhub/
469•MrBruh•1d ago•224 comments

LSP client in Clojure in 200 lines of code

https://vlaaad.github.io/lsp-client-in-200-lines-of-code
147•vlaaad•17h ago•18 comments

ToyDB rewritten: a distributed SQL database in Rust, for education

https://github.com/erikgrinaker/toydb
97•erikgrinaker•15h ago•12 comments

How friction is being redistributed in today's economy

https://kyla.substack.com/p/the-most-valuable-commodity-in-the
213•walterbell•3d ago•96 comments

White House fires head of Copyright Office amid Library of Congress shakeup

https://www.washingtonpost.com/politics/2025/05/11/white-house-copyright-office-director-fired/
51•handfuloflight•3h ago•36 comments

Burrito Now, Pay Later

https://enterprisevalue.substack.com/p/burrito-now-pay-later
137•gwintrob•15h ago•230 comments

Show HN: Codigo – The Programming Language Repository

https://codigolangs.com
42•adamjhf•2d ago•13 comments

A simple 16x16 dot animation from simple math rules

https://tixy.land
457•andrewrn•2d ago•91 comments

Lazarus Release 4.0

https://forum.lazarus.freepascal.org/index.php?topic=71050.0
239•proxysna•5d ago•136 comments

Avoiding AI is hard – but our freedom to opt out must be protected

https://theconversation.com/avoiding-ai-is-hard-but-our-freedom-to-opt-out-must-be-protected-255873
175•gnabgib•11h ago•103 comments

3D printing in vivo for non-surgical implants and drug delivery

https://www.science.org/doi/10.1126/science.adt0293
22•Phreaker00•1d ago•5 comments