I’m curious, can you provide an example of how it may be subverted? Give us a “real world” example as it applies to this specific technology being added to iOS.
Let me attempt to try provide a "real world" example that is personal to us. This is according to my understanding of this issue and might not be correct.
The issue here is not really the way the technology works, but the way Apple implements it breaks the trust we have with our phone and the trust we have with Apple. Privacy is a type of trust.
1. Your car (iPhone) and its speedometer (CSAM detector on your iPhone). And the road (iCloud)
So you own a car which is your personal vehicle you bought. In a normal car there is nothing from the car that makes you doubt anything. You trust it by nature. You drive it on the road which is a public property. Surely you make mistakes wether deliberately or accidentally sometimes by driving faster than the law on the highway. The highway camera takes a photo of your speeding car and fines you (this is the equivalent of scanning on the server side of iCloud).
Fine, you still trust that your car is
on your side. It never betrays you and you never have to doubt it. The only way that you can get fined is by the police placing cameras on the highway or whatever other methods they use. Your speedometer knows the car speed at all times and is like how the AI on your iPhone is analyzing all your photos in your photo library to identify faces and dogs. But your speedometer and car
do not report you to the police if it detects you're speeding. Same as the AI on your phone only analyzing the photos for you and only for your benefit.
So here is what will happen if Apple implements that feature. Your car will now record every time you speed. Whenever you want to go onto a highway (use iCloud photos) the car will be forced to report the police if you have speed more than 30 times on a local road (an example of subversion). Imagine that feeling. It feels like you can't really trust the car. You feel doubt and don't know when its going to betray you behind your back. Sure don't use the highway ever again, you have no secrets now because the car knows how your drive. We all make mistakes sometimes and what if that mistake is classified as a break of law one day (an example of subversion like in authoritarian governments "winnie the poo picture")? You cannot hide and fix before getting reported. You have no privacy, no room for error.
If given the choice, do you want to buy a car that only drives and shows you the speed. Or a car that drives, shows you the speed, and also reports to the police if you have sped 30 times and want to use the highway?
Some of my arguments here are not well thought out so it might change (like the highway and local road analogy, someone might say well don't use the highway then, etc).
2. Owning a self reporting iPhone feels like having a friend who likes to tell your mom or teacher every time you do something wrong (in this case its 30 mistakes).
You just don't feel the trust in that relationship. Do you want to spill your secrets or discuss important matters with this friend? No. If one day your mom or teacher (authorities, governments) ask your friend to tell them every time you talk about smoking (an example of subversion)?
I have friends where I trust they won't gossip behind my back no matter what I speak with them. That is trust and privacy.
Please let me know if these "real world" examples are convincing hahaha.
