My colleague sent me an awesome suggestion by copilot. It introduced a classic SQL injection vulnerability. I copy pasted it into Claude and said “any problems with this?” Yep! Critical sql injection vulnerability!
So even when they know they can’t be trusted (granted this is two different models). But the secret is that they often do know best practices and they simply don’t follow them. I personally have experienced half a dozen pretty severe problems with ai code.
They’re like lying, idiot-savant teenagers. They will 100% forget to propagate variable names (and claim to have checked for that when challenged), fail to follow best security practices, lie, claim to have done things when they have not, fail to think carefully.
And they can write 2,000 lines of reasonably good code an hour in any language you like. And they’re not bad as pair programmers.
Oh and they can review your code reasonably well.
Maybe with some good code security cursor rules they can get better, but I’ll believe it when I see it.
So no, don’t trust them.
dyl000•8mo ago