frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Dev-Owned Testing: Why It Fails in Practice and Succeeds in Theory

https://dl.acm.org/doi/10.1145/3780063.3780066
23•rbanffy•1h ago

Comments

__alexs•1h ago
This just seems to be basically a blog post that somehow got published in ACM?
pjdesno•1h ago
No, they got it published in ACM SIGSOFT Software Engineering Notes.

That's one of the things that publication is for.

The paper is a well-supported (if not well-proofread) position paper, synthesizing the author's thoughts and others' prior work but not reporting any new experimental results or artifacts. The author isn't an academic, but someone at Amazon who has written nearly 20 articles like this, many reporting on the intersection of academic theory and the real world, all published in Software Engineering Notes.

As an academic (in systems, not software engineering) who spent 15 years in industry before grad school, I think this perspective is valuable. In addition academics don't get much credit for this sort of article, so there are a lot fewer of them than there ought to be.

kayo_20211030•1h ago
A nice piece that outlines all the challenges, the opportunities, and the cultural and social adjustments that need to be made within organizations to maximize the chance of left-shifted testing being successful.

IMPO, as a developer, I see QA's role as being "auditors" with a mandate to set the guidelines, understand the process, and assess the outcomes. I'm wary of the foxes being completely responsible for guarding the hen-house unless the processes are structured and audited in a fundamentally different way. That takes fundamental organizational change.

brap•1h ago
Do people actually send PRs with no tests? That is so bizarre to me
pjdesno•55m ago
If your review was based on features shipped, and your bosses let you send PRs with no tests, would you? And before you say "no" - would you still do that if your company used stack ranking, and you were worried about being at the bottom of the stack?

Developers may understand that "XYZ is better", but if management provides enough incentives for "not XYZ", they're going to get "not XYZ".

hamdingers•9m ago
Breaking prod repeatedly probably impacts your stack ranking too.
Jtsummers•5m ago
[delayed]
brianwawok•49m ago
When I spell text wrong! Or want to add a log. Lots of reasons something is too silly to need a test.
weinzierl•39m ago
> "Do people actually send PRs with no tests?"

Rarely

Do people send PRs with just enough mostly useless tests, just to tick the DoD boxes.

All the time.

liampulles•28m ago
I've seen it many times. I think it often arises in business that are not very technical at their core.
xyzzy123•25m ago
It depends on the application but there are lots of situations where a proper test suite is 10x or more the development work of the feature. I've seen this most commonly with "heavy" integrations.

A concrete example would be adding say saml+scim to a product; you can add a library and do a happy path test and call it a day. Maybe add a test against a captive idp in a container.

But testing all the supported flows against each supported vendor becomes a major project in and of itself if you want to do it properly. The number of possible edge cases is extreme and automating deployment, updates and configuration of the peer products under test is a huge drag, especially if they are hostile to automation.

vrighter•16m ago
Once, for a very very critical part of our product, apart from the usual tests, I ended up writing another implementation of the thing, completely separately from the original dev, before looking at his code. We then ran them side by side and ensured that all of their outputs matched perfectly.

The "test implementation" ended up being more performant, and eventually the two implementations switched roles.

weinzierl•56m ago
The article argues that Dev-Owned testing isn't wrong but all the arguments it presents support that it is.

I always understood shift-left as doing more tests earlier. That is pretty uncontroversial and where the article is still on the right track. It derails at the moment it equates shift-left with dev-owned testing - a common mistake.

You can have quality owned by QA specialists in every development cycle and it is something that consistently works.

TheSoftwareGuy•47m ago
I'm interested, as I've never been in an org with QA specialists. What does that look like?
weinzierl•36m ago
The are breakers. Good devs are makers.

You can be both but I have yet to meet someone who is equally good in both mindsets.

bluGill•18m ago
You do everything the same as today. Then you turn it over to QA who keep finding weird things that you never thought of. QT finds more than half your written bugs (of course I don't write a bug everytime a unit test fails when doing TDD, but sometimes I find a bug in code I wrote a few weeks ago and I write that up so I can focus on the story I'm doing today and not forget about the bug)

QA should not be replacing anything a developer does, it should be a supplement because you can't think of everything.

We also use QA because we are making multi-million dollar embedded machines. One QA can put the code of 10 different developers on the machine and verify it works as well in the real world as it does in software simulation.

thmpp•44m ago
If as a developer you want to be seen as someone advancing and taking ownership and responsibility, testing must be part of the process. Sending an untested product or a product that you as a software engineer do not monitor, essentially means you can never be sure you created an actual correct product. That is no engineering. If the org guidelines prevent it, some cultural piece prevents it.

Adding QA outside, which tests software regularly using different approaches, finding intersections etc. is a different topic. Both are necessary.

sigmoid10•31m ago
The problem in big companies is that as a developer, you are usually several layers of people removed from the people actually using the product. Yes you can take ownership and implement unit tests and integration tests and e2e tests in your pipeline, to ensure the product works exactly as you intended. But that doesn't mean it works as management or marketing or the actual user intended.
mrits•43m ago
Author-Owned proof reading is next
sethammons•32m ago
While good points are made, I worry this gives the wrong impression. The paper doesn't say it is impossible, just hard. I have, very successfully, worked with dev owned testing.

Why it worked: the team set the timelines for delivery of software, the team built their acceptance and integration tests based on system inputs and outputs based on the edges of their systems, the team owned being on-call, and the team automated as much as possible (no repeatable manual testing aside from sanity checks on first release).

There was no QA person or team, but there was a quality focused dev on the team whose role was to ensure others kept the testing bar high. They ensured logs, metrics, and tests met the team bar. This role rotated.

There was a ci/cd team. They made sure the test system worked, but teams maintained their own ci configuration. We used buildkite, so each project had its own buildkite.yml.

The team was expected by eng leaders to set up basic testing before development. In one case, our team had to spend several sprints setting up generators to make the expected inputs and sinks to capture output. This was a flagship project and lots of future development was expected. It very much paid off.

Our test approach was very much "slow is smooth and smooth is fast." We would deploy multiple times a day. Tests were 10 or so minutes and very comprehensive. If a bug got out, tests were updated. The tests were very reliable because the team prioritized them. Eventually people stopped even manually verifying their code because if the tests were green, you _knew_ it worked.

Beyond our team, into the wider system, there was a light weight acceptance test setup and the team registered tests there, usually one per feature. This was the most brittle part because a failed test could be because another team or a system failure. But guess what? That is the same as production if not more noisy. So we had the same level of logging, metrics, and alerts (limited to business hours). Good logs would tell you immediately what was wrong. Automated alerts generally alerted the right team, and that team was responsible for a quick response.

If a team was dropping the ball on system stability, that reflected bad on the team and they were to prioritize stability. It worked.

Hands down the best dev org I have part of.

regularfry•21m ago
I've worked in a strong dev-owned testing team too. The culture was a sort of positive can-I-catch-you-out competitiveness that can be quite hard to replicate, and there was no concept of any one person taking point on quality.
boxed•29m ago
I think they've got that the other way around.
OptionOfT•28m ago
The conversation is usually: devs can write their own tests. We don't need QA.

And the first part is true. We can. But that's not why we have (had) QA.

First: it's not the best use of our time. I believe dev and QA are separate skillset. Of course there is overlap.

Second, and most important: it's a separate person, an additional person who can question the ticket, and who can question my translation of the ticket into software.

And lastly: they don't suffer from the curse of knowledge on how I implemented the ticket.

I miss my QA colleagues. When I joined my current employer there were 8 or so. Initially I was afraid to give them my work, afraid of bad feedback.

Never have I met such graceful people who took the time in understanding something, and talking to me to figure out where there was a mismatch.

And then they were deemed not needed.

KingOfCoders•14m ago
Developers want things to work.

QA wants things to break.

What worked for me, devs write ALL the tests, QA does selective code reviews of those tests making devs write better tests.

I also wrote the failure of Dev-Owned Testing: "Tests are bad for developers" https://www.amazingcto.com/tests-are-bad-for-developers/

darkwater•7m ago
First they came with the NoOps movement, and you were happy cause those damned ops people were always complaining and slowing you down. I can manage .y infra!

Then, they came with the dev-owned testing and fired all the QAs, and you were happy because they were always breaking your app and slowing you down. I can write my tests!

Now, they are coming with LLM agents and you don't own the product...