The nature of AI being a black box that, and fails in the face of "yeah those are some guys hiding in boxes" scenarios is something I struggle with.

I'm working on some AI projects at work and there's no magic code I can see to know what it is going to do ... or even sometimes why it did it. Letting it loose in an organization like that seems unwise at best.

Sure they could tell the AI to watch out for boxes, but now every time some poor guy moves some boxes they're going to set off something.

See https://metalgear.fandom.com/wiki/Cardboard_box

You don't need marines to invent that workaround, you see that in Looney Toons.

Don't security cameras have universals motion detection triggers you can use to make sure everything gets captured? Why only pre-screen human silhouettes?

SNAKE?!

AI will screwup, humans will screwup.

humans will see that they are screwing up and reformulate the action plan.

AI will keep screwingup until it is stopped, and apparently will gaslight when attempts are made to realign at the prompt.

humans realize when results are not desirable.

AI just keeps generating output until plugpull.

Reminds me of the joke where someone is wearing dildo patterned camouflage since most the AIs are trained on SFW corporate data.

I wonder if one could extract a "surprisedness" value out of the AI, basically, "the extent to which my current input is not modeled successfully by my internal models". Giving the model a metaphorical "WTF, human, come look at this" might be pretty powerful for those walking cardboard boxes and trees, to add to the cases where the model knows something is wrong. Or it might false positive all the darned time. Hard to tell without trying.

Seems to be blogspam re-reporting a 2023 article (with the same header photo): https://taskandpurpose.com/news/marines-ai-paul-scharre/

!

Solid snake approved.

You can get past a human sentry who is looking for humans, by hiding in a box, at a checkpoint in which boxes customarily pass through without being opened or X-rayed.