In Java I find agents write a lot more tests than I do, particularly agents will write tests that exercise each and every exception handling path. The code might throw an AException or a BException or a CException and I guess it is good to test that case B happens and throws a BException and I guess it means you get better logging but in the end the important thing is that when unhappy case B happens some Exception is thrown. Tests like this are likely to break when you change the implementation and waste some time but these aren't too hard to live with.
For one system I work on though, React tests are already disastrous, we're still running an older version of React where the framework can't really know when the component is done updating itself so adding a test can add seconds to the build multiplied by all the times I'm going to build. I am happy writing tests for functions that are really functions but until we get our React updated (update all our dependencies, many of which are no longer maintained, to use React 17+) I can't really test components so I don't want an agent writing tests or running tests if components are involved. It's not the fault of the AI, it's the fault of test infrastructure that sucks.
PaulHoule•3mo ago
In Java I find agents write a lot more tests than I do, particularly agents will write tests that exercise each and every exception handling path. The code might throw an AException or a BException or a CException and I guess it is good to test that case B happens and throws a BException and I guess it means you get better logging but in the end the important thing is that when unhappy case B happens some Exception is thrown. Tests like this are likely to break when you change the implementation and waste some time but these aren't too hard to live with.
For one system I work on though, React tests are already disastrous, we're still running an older version of React where the framework can't really know when the component is done updating itself so adding a test can add seconds to the build multiplied by all the times I'm going to build. I am happy writing tests for functions that are really functions but until we get our React updated (update all our dependencies, many of which are no longer maintained, to use React 17+) I can't really test components so I don't want an agent writing tests or running tests if components are involved. It's not the fault of the AI, it's the fault of test infrastructure that sucks.