Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Apple Addresses CSAM Detection Concerns, Will Consider Expanding System on Per-Country Basis
- Thread starter MacRumors
- Start date
- Sort by reaction score
No, on-device is more invasive because you have a program running on your device that is scanning your files. The current implementation will only work if you have iCloud Photos switched on. In the version for other countries such as Saudi Arabia or China, the folk in charge will simply tell Apple that the file scanner has to run and post to a server whether iCloud Photos is activated or not.
With on server scanning, the dissident in question simply doesn't upload to Google cloud; Apple adding on-device scanning gives governments to monitor what is actually on the population's iPhones, iPads and Macs whether they are connected to iCloud or not; all they have to do is tell Apple to do it, and Apple will do it if they want to continue to sell devices in that country (I don't seem them giving up on China any time soon).
But I think that we're all arguing at cross-purposes. Folk ask what is to stop this system being abused, and other folk reply that it can't be abused because the chances of a false positive is 1 trillion to one. That's not answering the question that's being asked.
What is to stop this system from being abused? If anyone knows then they should drop a note to Apple, because they've already admitted that it's open to abuse.
Just for reference once again:
You came to that conclusion because 1.4 trillion > 1 trillion?
Yes, that's what I was trying to say. That's exactly why I prefer server-side scanning.
Wow, and overnight, Android is the privacy platform. It's like waking up in an alternative universe.
What's stopping these governments for telling Apple and other to do this today without this new system?
China has the power to tell Apple to upload every photo taken or stored on an iPhone belonging to a Chinese user today. And it wouldn't be difficult for Apple to implement.
It's not a backdoor. A backdoor would require it to happen without either Apple or the user's knowledge.
You might argue it's a front door or maybe better a small side door.
The hash from the database is different on every iPhone. It's blinded. So if you get the hash from Apple, from the government database or from an iPhone it can't be used on any other iPhone.
Since the hash is different for every iPhone you would have to break the cryptography on every iPhone to transfer a cat picture to a matching hash.
Apple has said it is 1 in a trillion accounts, not pictures, with the threshold they have chosen. After that there are two manual reviews by humans before it maybe gets to law enforcement agencies.
Last edited:
My question is: what has changed compared to one week ago.
I’m sure dictators and agencies were already acutely aware Apple could do this potentially and actually already did all kind of AI processing to photo libraries.
People keep saying the cat is now out of the bag but I don’t see the cat. I just see a super specific use case so far, of a pretty mundane technique.
Apple can’t even stop scam apps in their App Store. So to answer your question, yes, I don’t trust them to be less incompetent in managing this new “feature” than the way they handle the App Store.
You can't. I was speaking more in general terms. There are 3rd party tools and services that allow you to encrypt files before uploading them to the cloud (e.g. Cryptomator and various E2E encrypted file syncing services). For example, you could use Cryptomator to upload pictures to iCloud Drive in encrypted form. But if Apple starts scanning the files on our devices, that undercuts any kind of E2E encryption. That's what alarms many security experts about Apple's proposal.
Apple says the scan happens right before you send the photo to iCloud.
Anyhow, it just a technical detail. Let's say in scans independent of uploading to iCloud. If a match is found a safety voucher is created with this information. But it's marked in such a way that you can't know, Apple can't know and anyone who gets hold of your iPhone can't either. So completely secret from everyone.
When these marks called safety vouchers are uploaded to iCloud Apple can't read the content of them. But can't they just count the numbers of them? Yes, but that doesn't indicate anything. Apple is creating false safety vouchers also, even for people with no matches. So the presence and number of safety voucher doesn't mean anything until the number of real safety voucher reaches a threshold.
Then Apple has enough data of a secret key to determine which safety voucher is real or fake and read which image was matched.
That is two different thing.
Now do you have directly law indicating privacy is a right, like right to bare arm, freedom of speech, freedom of movement?
Winning the lottery is very unlikely. And yet someone wins the lottery every week.
If you’re happy for Apple to falsely accuse an innocent person of being a paedophile every week then that’s on you.
Yeah comparing a yearly flow of millions of apps to review to the number of instances of frickin’ child p0rn escalated to human review is so relevant and edgy.
How are you so sure that the scanned pictures on the user's phone will generate the same hash as the scanned photo performed by the government?
If they used this system to catch scanned text it would probably leading to high number of false positives.
In fact, in iOS 15 Apple is introducing a way to convert text in images to text. Much better technology to misuse.
Which is partly why Apple decided it’s users should pay the cost of doing the checks.
For my own modest photo library, checking 30,000 photos against a database of 200,000 will require 6 billion checks.
And both numbers will only grow in the future.
Yeah thanks for totally ignoring the fact that Apple sucks at their gate keeping job. We haven’t gotten to how bad Apple is at fixing known security flaws.
Soon, billions, not millions, of comparisons will be made on user’s phones. What can go wrong?
I read the article; thanks for the recap.
It's a very BIG technical detail because there will be software on MY phone, not Apple's--my phone, that is scanning information from MY photos (not Apple's) to look for information (that I am not privy to and can't know) that they can send to someone to review after an unknowable number of photos in MY phone might end up resembling a known CSAM photo.
And right now it's CSAM (abhorrent and untenable, for sure), but in other countries (or here, even, in the next administration), what else might be considered abhorrent and untenable? Protesting? Adult pornography? Photos with same-sex relationships?
What else will Apple allow my phone to search for and use against me because the government doesn't like it?
Except Apple calls itself the beacon of users privacy. That's their entire business moto. If others are doing it, then Apple should be able too is hypocritical.
You can't create a set of hashes which covers "looks like protests in Hong Kong" even if you have thousands of pictures of a particular protest in Hong Kong to work from. Photos from the same event creates vastly different hashes.
But you can add hashes for media relating to the protests in HK that allow China to contextually identify the protestor in other ways. You can also do it in the Messaging System, as Apple explained. “Apple, monitor hashes that match with x symbol or feature that is a common denominator in the photos, I.e facial tracking and expression recognition (anger, happiness, sadness, all learned by the AI), or signposts or locational data based on specific building frames, etc”. There’s ALWAYS a way. And it’s probably already happening. If the AI can identify based on a database, it can also identify based on expressions that can be curated from other protests to “teach” the AI what anger and inciting information looks like. You know, like from the show Person Of Interest.
850 million iCloud users. For now Apple is only accusing its US customers of being perverts but let’s consider the inevitable application to worldwide users.
Let’s assume an average photo library of 10,000 photos and the current CSAM database is 200,000.
So an error rate of 1 in a trillion actually means Apple making 2 million false accusations.
Apple refuses to say how many accusations trigger a police call. Maybe 2. Maybe 200.
I hope anyone falsely accused sues Apple for all their worth.
In the past, America literally fought a war of independence over less abuse than what Tim Cook is inflicting on free people.
By having a threshold. One picture isn't enough. Maybe they have set the threshold to 50.
My impression is that many of the people having such pictures has thousands of them, even hundreds of thousands.
I read that WhatsApp reported 400 000 cases last year and someone here at MacRumours wrote that Facebook reported 20 million cases. If true, you can probably disregard those who just have a few photos.