Ask HN: What's the future of software testing and QA?

21•sjgeek•1d ago

Hello everyone, I have spent a decade in software testing and QA. I see Al taking over the field very fast. I want to prepare for the next five or ten years. According to you how the software testing field will evolve in the future? What should I prepare for it?

Comments

Sevii•1d ago

LLMs writing test cases, LLMs writing Selenium tests, LLMs doing exploratory testing, LLMs used for canary deployments. All that testing that people didn't do before because it was too hard and took too long? LLMs will be used to do it.

qsera•1d ago

Wouldn't there be a lot more of QA jobs. If software gets written faster by AI, humans sure will need to test that it meets the requirements.

So I see a great future for testing and QA.

vivzkestrel•1d ago

LLMS are still very bad at mocking external dependencies. i have had a lot of trouble with vitest mocks for postgres and redis so far. I still have a playwright test for an SSR page that does a server request. At the moment it still does an actual request to the server and none of the LLMs have managed to fix that

HiPhish•1d ago

I really hope that functional programming and property-based testing [1][2] get taken seriously by real engineers who understand that it is important to know and understand what the program is doing. Something LLMs by their very nature cannot do.

I was writing a React application at work based on React Flow[3] and I was mucking about with state management libraries (because that's what the React Flow manual recommends). Maybe it was a skill issue on my part, but I had a hard time with the Zustand library. Then I read up on reducers in React and everything was perfectly clear. A reducer is just a pure function, it takes an existing state and an action and returns the new state. That's simple, I can wrap my brain around that. Plus, I know how to test a pure function, there is nothing to mock, stub or wrap. States and actions are just plain JavaScript objects, there is no history, no side effects, no executable code.

And this is where property-based testing comes in: if history does not matter it means I can randomly generate any valid state and any valid action, then apply the reducer function and verify that the resulting state has all the required properties. Only a formal proof would give me more certainty.

I fully understand people who want to use LLMs to write tests for them. Writing test cases is boring and tedious. There are many edge cases a human might miss. But relying on a guessing machine and praying it does not write nonsense is just plain irresponsible for someone who calls himself an engineer. People rely on the quality of our software for their work and personal safety. Property-based testing frees us from the tedium and will generate many more tests and we could write by hand, but it does so in a predictable manner that can be fully reasoned about.