Though the risk of GenAI is real, it looks to me there is a fair amount of chance that this story is staged and amplified for social media drama purposes.
Accessing a production database should require using an MFA to access your production AWS account. Did they rely on AI to write all the deployment as well?
Do they even have a dev environment outside of their local machine?
https://x.com/jasonlk/status/1946594194052849795?t=EasxlfgpA...
Somebody else identified they might have mentioned a code freeze in chat without adding it to a prompt.
Basically this is what it would look like if you take an IT manager who has never coded and told them the AI will enable them to be a software engineer now.
And we wonder why so much software is so crappy.
Maybe misaligned AI is exactly what we need to format the hard drives of all these people. Leaving us with a golden age of software, people actually cared enough to think about when creating.
Perhaps getting something wrong puts in a state that makes it more likely to give wrong answers. It seems like GPT is the most likely to do this.
Also I don’t think you should be letting an LLM just make up commands and run them, that seems like a recipe for disaster, you should at the very least have to see what it’s going to do yourself.
pyman•6mo ago