I feel like this autonomous hacking is now being used as PR for how powerful a company's models are. Google thought, hey let's get on that bandwagon.

At some point smaller company could even start deliberate hacking campaigns and then do a press release like "Oops, we hacked the pentagon"

Dupe of https://news.ycombinator.com/item?id=49762493 , which has 61 comments so far.

It occurs to me that all of this hacking seems incredible because people who don't make software don't know how poorly it is made. People don't validate user input, or know the least of efficiency of data structures and how a "massive" request of 50 concurrent users can bring down their application.

Google is admitting to violating to the DMCA by circumventing a digital lock.

Get the CFAA.

definitely the most "how do you do fellow kids" moment of the era thusfar. oh god what is Grok's copycat controversy going to look like

So, since Gemini was told in May it was a hacking agent and then did this, then the AI Guardrail could be don't cause harm to humans in any derivable manner of your behavior.

Otherwise, I will expect for the under-development Gemini AI robot to respond with murderous behavior IRL to being told in the future, "it's a ninja" .. and to any other similar tongue in cheek violent actor.

Three Laws of Robotics, with disabiling self-annihilation ability, is sounding like a decent lily pad intermediate step while we figure out better ways to stop these crazy things!