Ask HN: What parts of software testing can realistically be autonomous today?
1•nishilpatel•1d ago
Modern systems have good observability and better tooling than ever, yet testing often remains manual and slow.
In your experience, what aspects of testing can actually run autonomously today, and what still seems fundamentally hard or unsolved?
Comments
jeffreygoesto•1d ago
Testing is answering "does it do, what it is supposed to do?" and autonomous means "according to it's own law(s)". Sounds like a contradiction to me. I'd answer with "none".
pfdietz•1d ago
One can define properties the software is supposed to have, then autonomously test for those properties (as in, initiate a process that spends arbitrary amounts of time running new tests to try to show the software fails to have those properties.)
Is this not autonomous because the properties weren't created without humans being involved? How could that even be possible?
linkedinlobster•1d ago
We did not fail to automate testing. We automated the easy parts and kept calling it progress. The hard part was never running tests. It was deciding what actually matters. We cannot automatically detect risk. Risk lives in context, trade-offs, and user expectations...none of those are deterministic.
stellarvore•13h ago
Historically speaking, if an autonomous system found a bug [monkeys and fuzzers have done that for years ;)], you'd just get a "test failed" log. You'd still need to manually dig through the stack traces to pinpoint the "why" of it. Well, an Ai app can tell you that the system crashed or a button moved 5px to the left. But it still can't tell you if that movement broke the UX without a human explicitely defining the heuristic first.
IMO, the hard parts that actually getting solved right now is the autonomous serialization of state; context capture and issue repro steps to be precise (and definitely not the "finding of bugs" part).
jeffreygoesto•1d ago
pfdietz•1d ago
Is this not autonomous because the properties weren't created without humans being involved? How could that even be possible?