"Generate a pure white image." "Generate a pure black image." Channel diff, extract steganographic signature for analysis.
> "Generate a pure white image."
It refused no matter how I phrased it ¯\_(ツ)_/¯
> "Generate a pure black image."
It did give me one. In a new chat, I asked Gemini to detect SynthID with "@synthid". It responded with:
> The image contains too little information to make a diagnosis regarding whether it was created with Google AI. It is primarily a solid black field, and such content typically lacks the necessary data for SynthID to provide a definitive result.
Further research: Does a gradient trigger SynthID? IDK, I have to get back to work.
Gemini really doesn't like generating pure-white images but you can ask it to generate a "photograph of a pure-white image with a black border" and then crop it. So far I've just been looking at pure images and gradients, it's possible that more complex images have SynthID embedded in a more complicated way (e.g. a specific pattern in an embedding space).
In a sense, the identifier company can be an arbiter of the truth. Powerful.
Training people on a half-solution like this might do more harm than good.
Marking real from lense through digital life is more practical. But then what do we do with all the existing hardware that doesn't mark real and media that preexisited this problem.
In the end, society has always existed on human chains of trust. Community. As long as there are human societies, we need human reputation.
None of these schemes for validation of digital media will work. You need a web of trust, repeated trustworthy behavior by an actor demonstrating fidelity.
You need people and institutions you can trust, who have the capability of slogging through the ever more turbulent and murky sea of slop and using correlating evidence and scientific skepticism and all the cognitive tools available to get at reality. Such people and institutions exist. You can also successfully proxy validation of sources by identifying people or groups good at identifying primary sources.
When people and institutions defect, as many legacy media, platforms, talking heads, and others have, you need to ruthlessly cut them out of your information feed. When or if they correct their mistake, just follow tit for tat, and perhaps they can eventually earn back their place in the de-facto web of trust.
Google's stamp of approval means less than nothing to me; it's a countersignal, indicating I need to put even more effort than otherwise to confirm the truthfulness of any claims accompanied by their watermark.
Note that your cell phone camera is using gen AI techniques to counteract sensor noise.
Was that famous person in the background really there, or a hallucination filling in static?
Who knows at this point? So, the signatures you proposed need to have some nuance around what they’re asserting.
There is no original image to recover, since we can't capture and describe every photon, so it's not a "fake vs real" image signature... that would be a UI choice the image viewer client would make based on the pipeline data in the image.
But I suppose it ads friction so better than nothing.
Watermarking text without affecting it is an interesting seemingly weird idea. Does it work any better than (with knowledge of the model used to produce said text), just observing the perplexity is low because its "on policy" generated text.
...But it can be hard to tell the difference between content that’s been
AI-generated, and content created without AI.
What incentive do open models have to adopt this?
[1] Specifically, "...synthetic audio, image, video or text content, shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated", see https://artificialintelligenceact.eu/article/50/
Some previous discussion:
Which means short texts are basically useless. A 50-token reply has too little signal for the test to reach any confidence. The original SynthID text paper puts minimum viable detection at a few hundred tokens - so for most real-world cases (emails, short posts, one-liners) it just doesn't work.
The other thing: paraphrase attacks break it. Ask any other model to rewrite watermarked text and the watermark is gone, because you're now sampling from a different distribution. EU compliance built on top of this feels genuinely fragile for anything other than long-form content from controlled providers.
u1hcw9nx•1h ago
>The watermark doesn’t change the image or video quality. It’s added the moment content is created, and designed to stand up to modifications like cropping, adding filters, changing frame rates, or lossy compression.
But does it survive if you use another generative image model to replicate the image?
lxgr•1h ago
That's been a thing for a while: https://en.wikipedia.org/wiki/Digital_watermarking
nerdsniper•1h ago
elpocko•12m ago