frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

Open in hackernews

Ask HN: What is the most impressive test suite youve ever worked with

2•escot•4h ago
Did it have near 100% coverage or did it make tradeoffs on what to test?

Did it include UI tests which are notoriously difficult, and if so how did it handle issues like timeouts and the async nature of UI?

Did it have rigid separation of concepts between unit vs integration tests etc, or more fluid?

Could you refactor internal code without changing tests — the holy grail.

Comments

tsm•4h ago
I've found the Metabase test suite[0] to be very good considering it's real-world software written by a for-profit company. Coverage is good, the correct tests usually break when doing a refactor (stuff like "Oh, I thought this change was harmless but actually it breaks the permissions model"), etcetera. But the most important thing is that there's a strong team culture of a) demanding good tests on each PR b) hunting down flaky tests and other sources of friction.

Another neat thing was that there used to be a full-time SDET who spent a lot of time writing Cypress reproductions for known bugs. When you picked up the bug, you could un-skip the test that was right there waiting for you.

All that said, of course it's far from perfect!

0: https://github.com/metabase/metabase/ Backend unit tests are in test/, Frontend unit tests are in frontend/test, end-to-end tests (Cypress) are in e2e.

MoreQARespect•4h ago
The test suite was 90% "end to end" unit tests - no real infrastructure was used it was all faked. Only interactions with the outside world (web client, LLM, database) were tested and all interactions were faked.

(This is not feasible on every project but it was on this one, database interactions were simple)

There were a small number (~5%) of slow tests that used a real LLM, database, infrastructure, etc. and a small number of very low level unit tests (~5%) surrounding only complex stateless functions with simple interfaces.

Refactoring could be done trivially without changing any test code 98% of the time.

Additionally, the (YAML) tests could rewrite their expected responses based upon the actual outcome - e.g. when you added a new property to a rest api response you just reran the test in update mode and eyeballed the test.

There was also a template used to generate how-to markdown docs from the YAML.

Test coverage was probably 100% but I never measured it. All new features being written with TDD/documentation driven development probably guaranteed it.

The Evaluation of Engineering Artificial General Intelligence

https://arxiv.org/abs/2505.10653
1•sirregex1•1m ago•0 comments

Alpenglow: A New Consensus for Solana

https://www.anza.xyz/blog/alpenglow-a-new-consensus-for-solana
1•lawrenceyan•1m ago•0 comments

Removing the weakest link in electrified, autonomous transport: humans

https://arstechnica.com/cars/2025/05/removing-the-weakest-link-in-electrified-autonomous-transport-humans/
1•rntn•2m ago•0 comments

Game theory illustrated by an animated cartoon game

https://ncase.me/trust/
1•felineflock•2m ago•0 comments

New Parquet writer allows easy insert/delete/edit

https://github.com/apache/arrow/pull/45360
1•sps44•2m ago•0 comments

Microsoft Build Conf – Day 1

https://build.microsoft.com/en-US/home
2•vyrotek•2m ago•0 comments

Edit is now open source

https://devblogs.microsoft.com/commandline/edit-is-now-open-source/
2•ingve•3m ago•0 comments

Ask HN: How do you handle licensing and revenue leaks for self-hosted software?

1•lexokoh•4m ago•0 comments

Keeping Ahead of Contagion

https://www.asimov.press/p/contagion
1•mailyk•6m ago•0 comments

The impact of types of childhood maltreatment on PTSD and personality disorders

https://www.tandfonline.com/doi/full/10.1080/20008066.2025.2491172
1•rendx•8m ago•0 comments

Microsoft's Windows Subsystem for Linux is now open-source

https://www.theverge.com/news/669286/microsoft-windows-subsystem-for-linux-open-source
2•el_duderino•8m ago•0 comments

Microsoft Open Sources Copilot Exclusive Interview W Erich Gamma Kai Maetzel

https://www.youtube.com/watch?v=GMmaYUcdMyU
2•torartc•9m ago•0 comments

Teaching a 4B LLM to Guess a Number (Without Cheating)

https://medium.com/@hbbio/guess-what-im-thinking-8e0e628094b4
1•hbbio•11m ago•0 comments

GitHub Copilot Coding Agent

https://github.blog/changelog/2025-05-19-github-copilot-coding-agent-in-public-preview/
6•net01•12m ago•0 comments

Ask HN: Tech Behind the "Magic" Company?

1•ChicagoBoy11•13m ago•0 comments

Hosted GitHub Actions Runners Now Available for IBM Z, IBM LinuxONE, IBM Power

https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/elizabeth-k-joseph1/2025/05/19/expanding-open-source-access-hosted-github-actions
1•rbanffy•13m ago•1 comments

Show HN: I built ServBay – An integrated local dev env for Mac (Windows coming)

https://www.servbay.com
2•Hayatoo•14m ago•2 comments

Biomimetic Flapping Mechanism for Insect Robots Driven by Indirect Muscles

https://www.mdpi.com/2313-7673/10/5/300
1•PaulHoule•15m ago•0 comments

Computation Isn't Consciousness: The Chinese Room Experiment [video]

https://www.youtube.com/watch?v=gleyKDvZ3x0
1•bitbasher•15m ago•0 comments

Inside the story that enraged OpenAI

https://www.technologyreview.com/2025/05/19/1116614/hao-empire-ai-openai/
1•miles•15m ago•0 comments

xAI's Grok 3 comes to Microsoft Azure

https://techcrunch.com/2025/05/19/xais-grok-3-comes-to-microsoft-azure/
1•mfiguiere•16m ago•0 comments

The Windows Subsystem for Linux is now open source

https://blogs.windows.com/windowsdeveloper/2025/05/19/the-windows-subsystem-for-linux-is-now-open-source/
2•pentagrama•16m ago•0 comments

Why Arabs Lose Wars (1999)

https://www.meforum.org/middle-east-quarterly/why-arabs-lose-wars
2•Tomte•18m ago•0 comments

Why Bloom filters work the way they do (2012)

https://michaelnielsen.org/ddi/why-bloom-filters-work-the-way-they-do/
1•Tomte•18m ago•0 comments

Microsoft Open Sources Copilot

https://code.visualstudio.com/blogs/2025/05/19/openSourceAIEditor
8•riejo•21m ago•0 comments

I 3D-printed Luigi Mangione's 'Ghost Gun' [video]

https://www.youtube.com/watch?v=tQB_ib-KQXA
2•sandebert•23m ago•0 comments

GitHub Copilot now has a coding agent embedded

https://twitter.com/ashtom/status/1924496497543901407
4•kesor•24m ago•0 comments

Digital Social Norm Enforcement: Online Firestorms in Social Media (2016)

https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0155923
3•mooreds•24m ago•0 comments

European Investment Bank to inject €70B in European tech

https://ioplus.nl/en/posts/european-investment-bank-to-inject-70-billion-in-european-tech
4•saubeidl•25m ago•0 comments

How to Get 600 EUR Flight Cancel Compensation

https://www.josecasanova.com/blog/flight-cancel-compensation
2•jcsnv•25m ago•0 comments