Apple could easily not do this stuff and it may even be easier to not.
But they are until they are actually defeated. I would rather plan for failure. We are in a global climate where court rulings can be ignored.
> It also ensured pressure from governments and plaintiffs, including CSAM victims, who preferred Apple’s more interventionist approaches, which Apple had voluntarily demonstrated it was willing to do.
I feel that Apple open pandora's box with the client-side scanning. It proved that it was technically feasible, and was "privacy preserving". I use scare quotes there because I don't think that political or religious dissidents would find that the same or similar technology used to discover and persecute them is "privacy preserving". And that's really the problem with Apple here. They provided a model for scanning for any kind of message or material while purportedly maintaining privacy.
No, in the sense that your therapist is still required to report you to the police in various situations where you pose an immediate threat to yourself or others, etc.
And therapists are legally mandated to report you if you told them you viewed or possessed CSAM.
Like OP said, Apple isn't perfect nor will they ever be, but they do prioritize privacy better than most.
People understood this settlement to mean Apple was spying on their conversations and selling them to advertisers, when it seems to have more to do with people accidentally triggering Siri. But people don’t care about this kind of nuance or actually tallying up all the ways Apple is pro privacy against rare issues like this one. It’s all just tribalism at the end of the day.
Watchdog ponders why Apple doesn't apply its strict app tracking rules to itself (theregister.com)
161 points by Logans_Run on Feb 14, 2025 | 69 comments
https://news.ycombinator.com/item?id=43047952
Apple silently uploads your passwords and keeps them (lapcatsoftware.com)
170 points by ingve on Nov 1, 2024 | 127 comments
And whenever your privacy contradicts their control over "your" device, you are also out of luck, e.g., you can't have Ublock Origin on an iPhone. Relevant discussion: https://news.ycombinator.com/item?id=44804921
Except ublock, which can't do what it does the way it normally does, for the same reason you can't have any plugin inspecting realtime activity and doing scriptlet injection.
You can have ad blocking. You can't have plugins with that kind of low level access to your browser activity.
You can prefer something that allows dangerous behavior as a trade-off for greater capabilities, but you can't deny it's a safety trade-off where Apple picked what's safer.
At least on Desktop we have usable Linux, but on the phones there is literally nothing usable because thanks to Google efforts switching to GrapheneOS mean tons of apps either not working or break every few months.
Yes its possible to make Andoid spy on you a little less, but even for tech savvy person its damn inconvinient and Google making platform worse with every single release.
Thanks to Google "security" I can use my banking apps on 9 years old device with 6 years outdated firmware, but not on GrapheneOS.
They probably use E2EE just so they don't have to respond to court orders and such.
I've been using GrapheneOS for years and that hasn't been my experience. There are two financial apps that don't work for me, and that's it. Pretty much everything else I use is fine. But, to be fair, I'm very scrupulous about my apps and tend to avoid installing an app for every little thing that wants me to.
These companies are liars. I do not trust liars. It has served me well.
If Apple's interests sometimes align with ours then great. I'll take it. But don't attribute to this ~5 trillion dollar company some kind of altruism.
So once there’s a profit motive for violating your privacy, the justification for eroding your privacy will proceed. It’s really the inertia of Apple starting out as privacy-compatible that makes them hesitant to throw that away.
As sad as this is, end to end encryption means no CSAM scanning.
As an alternative Apple previously tried to do scanning on the phones locally but caught hell for that too.
This is one of those unfortunate tradeoffs but I see no alternative to privacy taking priority.
When arguing against surveillance, the arguments should be on its merits, not on whether the current proponents happen to have ulterior motives.
If you take a 1000 random people of the street now,how many of them are sharing CSAM via icloud?
If you take a 1000 random politicians, how many of them have corruption scandals? Why not start with them instead, a bodycam and an AI powered microphone that would detect corruption automatically... let them lead as an example, before they apply the laws onto "the rest of us".
The VAST majority of “CSAM” is consensually created and exchanged by teens. Their future selves and their parents form this pressure group attacking everyone’s liberty and privacy to try to undo the downsides of choices they made themselves with full knowledge of what could happen.
The criminal and disgusting tail end of this type of material deserves the worst of consequences for the perpetrators and all the support in the world for the victims, but these are mostly - you guessed it - poor and unprivileged children from far away places and they certainly can’t put this much pressure on apple
Our legal systems are not built to deal with that mess, and it may hang around your neck for the rest of your life. Unfortunately, the law is very explicit, leaving barely any avenue for the courts to drag us out of the mess, and politicians - even if they are actually interested in the topic in the first place - won't touch that area with a ten foot pole for fear of getting blamed a pedophile themselves.
[1] https://www.n-tv.de/panorama/KI-treibt-Jugendporno-Fallzahle...
While on statistics, I wonder, are there reliable statistics about child abuse of different types? Studying correlations with other social metrics, like sex education, liberal/conservative, policies regarding prostitution, and others can provide support for/against decisions.
Not that I hope these will impact people's and governments' choices, but I want to challenge my intuitions.
Same issue with Proton, MEGA, and any other e2ee app... it's only useful when the company decides not to mess with the data it could always decrypt locally. Also why people are hesitant to use javascript-based e2ee solutions where the site owner can modify the code at will to do what they want.
If the company is misleading, any encryption technology is irrelevant anyway.
Wasn’t Apple’s design to explicitly NOT scan in its cloud storage, but look at the file on-device at the moment you wish to upload it to iCloud? This method would make it compatible with Advanced Data Protection; so ADP could have always been in the pipeline rather than Apple u-turning. In fact, NeuralHash may have been proposed because Apple wanted to introduce ADP and saw a potential problem here/get concerns from government agencies about it and saw this as a means to an end(-to-end).
The system was designed pretty elegantly and offers far better privacy protections - including guardrails - than what Microsoft and Google do, but the communication from Apple about it was absolutely horrible and generated enormous backlash. (Not saying I agreed with implementing it, just saying the design was infinitely better than competitors.)
What world do you live in?
Most people don’t have computers, those who do, do not regularly backup their photos on them.
In both family and extended family many a cry would be avoided if people paid the 5 bucks it costs to backup their photos before your phone gets stolen or lost.
There is something ironic about US laws that attempt to prevent crime A by outlawing action B. For example:
* A: physical sexual abuse of children. B: possession or distribution of CSAM
* A: drug trafficking or tax evasion. B: structured cash withdrawals
It's my understanding that conviction of CSAM-related crimes do not require any physical act to have ever occurred to any real person: one can be convicted of CSAM-related crimes related to paintings/drawings/created_art of fictional people.
It's my understanding that one can be convicted of structured withdrawals that are not driven by, linked to, or in any way related to anything nefarious.
On top of that, while there are different types of child abusers, the worst ones almost invariantly collect CSAM to the point of hoarding. So it really isn’t that bad of a proxy.
The root comment is implying that legalizing or decriminalizing csam would somehow help with prosecution of child abuse? I’m kind of speechless. Csam IS child abuse. The fact that there are consumers encourages producers to, well, produce!
Honestly shocked that anyone would even say this, but even giving you the benefit of the doubt here -- the one case where I could imagine this might not happen would be if you're a police officer investigating such cases. But they also have their own therapists dedicated/trained in police-specific issues.
But also there are definitely ways to get accidentally exposed. That's an absolutely awful thing to call the cops over.
Didn't their paper disproved by reversing the perceptual hashes to reveal blurred version of the images being hashed, and Apple basically said "that's fair, it's not as robust as we wanted, let's visit this later"?
If not, I'll happily stand corrected, but please share sources.
Addenda:
- Apple's original paper: https://web.archive.org/web/20210807165030/https://www.apple...
- Paper breaking the hash: https://arxiv.org/abs/2111.06628
Skimmed your linked paper. It seems they were able to classify hashes up to ~8% top-1 accuracy and ~30% top-10. Not exactly a blurred version, or any images at all.
So for example, they can say that you probably have images of trees, or images of buildings, but without much other data & very low accuracy.
I'd still be a lot more concerned about them simply flagging political images rather than trying to get a broad understanding of what type of photos I have
Very different than trying to narc out users to the authorities.
Trying to avoid false positives by not firing until a threshold was hit (was it 20 images?) seemed insane from a PR position... rightly or wrongly, all it would take would be the wrong court case and you can see the headlines:
"Apple says users can have up to 20 CSAM images on their phone before they'll tell police"
You might not like pictures that way but honestly I think more important in procescuting CSAM is to go after the large sources of CSAM generation. Its trafficing in East Asia, and in Europe. I think weirdly America actually produces less CSAM in general because Americans are lot more off put by Sex than most other cultures. Abuse definitely happens in the US but making policy decisions like this produces bad policy.
Does iCloud rehost the photos to other people I don't really know because I use andriod tbh. If they are being rehosted (I assume to members of your contacts) that can be problematic but I think honestly the issue a lot more complex than just protect the children which the source of critic is a lot attacks against apples are coming from
I think it depends on your definition of e2ee and where the "end"s are.
If the locally running application can decrypt the data, it could always do whatever it wanted. Is that really how you define e2ee?
It's harder to do, but conceptually the same. So sad it's not being done. Very disturbing.
They could do it when people are not at home. There'd no problem, nobody would even notice.
A better one: what about rental property, like a business? Can the landlord randomly check for criminal behavior?
Absolutely not.
I think your conspiracy theory needs work, to be perfectly honest with you.
Just because we technically can make a privacy destroying drag net does not mean we should. Had phones existed 250 years ago, I have no doubt the founders would have thought it obvious that a cellphone’s contents were your personal papers which could not be freely searched.
An icloud is like a storage locker or a safety deposit box... the owner should go through all your stuff there, just in case you have some CSAM!
Metadata is just tracking info about who, where and with whom... every bartender should take your IDs and log when you came to the bar, who you sat with and how long you talked there.
EU Chat control is like general eavesdropping... every time you sit down and talk with someone, an EU bureaucrat should sit next to you and listen and write down your conversations, just in case.
etc.
Somehow people think that "it's ok if it's on the internet", even when it's stuff they'd never accept in real life.
Personally, I am on the side of privacy, just to be clear.
Not true. There is the option of scanning on the device.
The primary focus should always in preventing the creation of CSAM.
- Comprehensive Sex Ed starting young so kids can identify grooming and seek help from a trusted adult, even if abuse comes from a family member.
- Fixing schools in general so homeschooling isn't as attractive for parents. Keep a tab on home schooled children and identify social isolation.
- Bigger resources for actual honest to god on the ground investigations.
To be clear I'm not saying that homeschooling = child abuse, simply there's a lack of the mechanisms to detect it in homeschooling settings.
Why would some adults find kids sexually attractive? Is it abusive/aggressive behavior manifesting itself in sexuality? Or is it sexuality channeled in the wrong direction? If it's the second, is it out of desperation, and would happen less if the culture makes it easier for them to satisfy their needs with adults, or would it happen regardless? On the victim's side, are the shy and less social ones more in danger, or the socially active ones? From my social scientist friends I hear a lot that most child sexual abuse is domestic. What are measures that a society can take to prevent these, without turning the society into a surveillance state, which will ultimately harm everyone more, including the children? What can be done to make sure children speak up, so that such behavior is dealt with at the beginning (and maybe while the more terrible things have not happened yet), and not turn into a multi-year childhood trauma?
What are signs (and early signs) on the abuser's side and the child's side? How to deal with these signs?
The real question is what happens when a horny teenager sends another a nude. There definitely have been insane cases where they get stitched up for creating child porn. I don't know if that's the normal outcome today though.
I wrote to the rep and explained my concerns. I wholeheartedly agreed with the intent of the law, but the code was buggy. To my surprise, he wrote back in horror to say he hadn’t considered that and pulled the bill immediately. I’m proud of having done that.
I’m 100% pro yeeting child pornographers into the sun. I still don’t want to throw kids in prison or remove all traces of a right to privacy in our haste to sun-yeet them.
It was not a tool to identify private images as being underage. That’s an impossible task.
In theory, modern vision language models could classify human nudity and sexual activity very thoroughly. But every model I have tried is reluctant to clearly describe what is notable about sexualized/nude images. The models are deliberately under-exposed to nude and sexualized content during training and further RLHF'd away from generating straightforward descriptions of such images.
Models also occasionally hallucinate WTF captions for ordinary adult sexual activity. I recently ran a baseline test with frames extracted from adult videos and about 1/3000 of them was mis-captioned as involving a child according to Gemma 4 12b.
Just ask the dad who was investigated for taking pictures of his toddler for the doctor: https://www.koffellaw.com/blog/google-ai-technology-flags-da...
Yes, poor and unprivileged children can't really defend themselves here, but this is the system working to find some legal mechanism to do what it can, as a more powerful force. Protecting people from exploitation is a good use of government. If this was shot down for legal reasons, OK, the system is working and I hope there is a way to expand protections that fits into our system.
That's obviously only the case if they aren't also the sole providers of the "ends".
Sure, the NSA, GCHQ and Mossad have a way to get the encrypted data by a sidechannel but proprietary e2e is a good thing for most people IMO. Shifts the risk from "my messages are theoretically available to most law enforcement in the globe" to "YOU’RE STILL GONNA BE MOSSAD’ED UPON"[0]. This is specially good for me because I know the equivalent to the FBI where is live is too cheap to buy a Cellebrite [1] license.
[0] https://www.usenix.org/system/files/1401_08-12_mickens.pdf [1] https://arstechnica.com/gadgets/2025/10/leaker-reveals-which...
EDIT: I suppose someone could ask about Meta. The reason behind their support for scanning (and removing e2e in facebook msg) is simply regulatory capture. The zucc wishes to have a letter of marque to "protect" your children and remove the "unsafe" competitors.
If this were true then widespread availability of pornography on the internet would have resulted in a massive increase in rape of adult females. When in fact, assault numbers have been on a steady decline for decades.
As well as possession. I don't actually know if those are different for CSAM, but I would assume so because they are for drugs.
This isn't necessarily the case in the US, though I believe only for drawings. AI-generated CSAM probably wouldn't fly in a court of law.
Regardless, it's a naive conception of a system of law to think of it as a utilitarian system of restitution in contexts of "this individual harmed this individual". In fact, that would fall under the category of a "tort" rather than a "crime". The law is just as much about enforcing social mores and norms as it is about dealing with individuals harming each other. Hence why locales like Canada outlaw all forms CSAM, even fictional ones. The victim taken is to be society itself. The possession of this material, implicitly entailing enjoyment of it, is so gross a violation of society's norms and mores that it becomes elevated to a legal matter.
The reason why the Supreme Court upheld bans on possessing CSAM is not because it's obscene, but because it incentivizes abuse of children to produce it.
Sounds a lot like what an authoritarian society, driven e.g. by a conservative religion would look like. Something closer to the alternative you dismiss as naive sounds a lot more just to me.
This shouldn't be the case in a society that supposedly values liberty.
There have been a handful of convictions based on fictional content, but usually the defendants also possessed real CSAM so there wasn't much point in contesting the charges over fictional images.
jobs_throwaway•2h ago
hosteur•2h ago