Apple Says NeuralHash Tech Impacted by 'Hash Collisions' Is Not the Version Used for CSAM Detection

[chucker23n1]

While I am sure people may agree with this, it seems like one step away from doctors/dentists submitting DNA samples of *every* patient because it is in the public interest.

This is more comparable to:

1. police makes a database of DNA samples of convicts
2. they provide hashes of that database to Apple, so that Apple can verify if someone is a match to that database (but Apple cannot know who is on that database; it's one-way)
3. your Apple Watch checks your health metrics against those hashes
4. if there's about 30 matches, it reports you to Apple; otherwise, Apple never sees anything
5. if Apple manually verifies those matches to be correct, they report you to police; otherwise, they never see anything

(In practice, it's a lot more complex. Analogies are flawed like that.)

 

[jdb8167]

For starters, because this is a process that has access to at least: 1) your images, 2) the Neural Engine, 3) the local copy of the NCMEC hash database, 4) credentials to upload your photos Apple's servers. Probably more than that.

(It's possible they've sandboxed it further into separate processes, but I don't believe so.)

In terms of vectors, you could do worse.

But again, the iPhone tagging software has access to all that too except the hash database. I haven't investigated but I'm pretty sure that the iPhoto tagging is synced across devices via iCloud. What is dangerous about having access to the NCMEC hash database?

 

[B4U]

Between the iPhone 13 series with this iOS preinstalled vs a 12 series now possibly with a discount. Sounds like a no-brainer.

 

[AlexanderUK]

... Someone on Twitter mentioned what happens if someone airdrops you a bunch of illicit photos and they sync to iCloud in a matter of seconds? Boom you’re flagged. There’s 1,000,000 ways for this system to go wrong or be exploited or worse ruin innocent peoples lives. And if you do end up being one of those people, you will have exactly zero recourse to prove your innocence. It’s over for you. This entire thing is very stupid on Apple’s part.

I mentioned something similar on a previous thread except the threat I pointed out is what happens if a hacker were able to hijack your Apple ID. Remember a few years ago when there were the celebrity iCloud breaches? Never mind that individuals and account security don't really go hand in hand well (passwords and such).

Previously the worst case scenario was that you lost access to your account and purchases. Now you'll have individuals hijacking accounts and holding people for ransom with the threat of uploading kiddie porn via a VPN in their location in order to frame them which would set off the triggers and as you said... reported to the authorities, no recourse and you'd end up in jail with your life ruined. Doesn't even have to be a hacker, could be a revenge actor doing it, etc.

Apple can preach about the tech being sound till the cows come home, but the mechanism behind it stinks of bad actors, potential criminality, and interference.

 

[dguisinger]

For starters, because this is a process that has access to at least: 1) your images, 2) the Neural Engine, 3) the local copy of the NCMEC hash database, 4) credentials to upload your photos Apple's servers. Probably more than that.

(It's possible they've sandboxed it further into separate processes, but I don't believe so.)

In terms of vectors, you could do worse.

Imagine an exploit that comes in via messages (like pegasus), alters the hash database to include tank man, alters the reporting URL to a Chinese government server, and secretly re-enables this process even if it was turned off via turning off iCloud Photos.

Just because you bothered to turn it off, doesn't mean under the hood it is actually off if you got hit by malware like Pegasus. It doesn't have to be a direct attack vector, it could just be turned back on and re-used for different purposes by a different attack vector...... and you would never know.

If there is enough will, there is a way to take advantage of this. And for some governments, there is a lot of will.

Why on earth would we build a system into a phone that scans our images and reports out matches

 

[MuppetGate]

But again, the iPhone tagging software has access to all that too except the hash database. I haven't investigated but I'm pretty sure that the iPhoto tagging is synced across devices via iCloud. What is dangerous about having access to the NCMEC hash database?

Y’know, there’s something not right about putting child porn hashes on everybody‘s phone. Even if I can’t see them, knowing they‘re there is … yeeuck.

Who the hell signed off on this?

 

[dguisinger]

Y’know, there’s something not right about putting child porn hashes on everybody‘s phone. Even if I can’t see them, knowing they‘re there is … yeeuck.

Who the hell signed off on this?

Well, Tim Cook is a numbers guy and not a product guy.... I could see him saying this saves us so much money if we use phone CPU cycles instead of server CPU cycles, I like it

 

[jdb8167]

I mentioned something similar on a previous thread except the threat I pointed out is what happens if a hacker were able to hijack your Apple ID. Remember a few years ago when there were the celebrity iCloud breaches? Never mind that individuals and account security don't really go hand in hand well (passwords and such).

Previously the worst case scenario was that you lost access to your account and purchases. Now you'll have individuals hijacking accounts and holding people for ransom with the threat of uploading kiddie porn via a VPN in their location in order to frame them which would set off the triggers and as you said... reported to the authorities, no recourse and you'd end up in jail with your life ruined. Doesn't even have to be a hacker, could be a revenge actor doing it, etc.

Apple can preach about the tech being sound till the cows come home, but the mechanism behind it stinks of bad actors, potential criminality, and interference.

If someone has access to your Google or Facebook account, can't they already do this? I'm not discounting your scenario, it is a frightening prospect, but scanning on device or scanning in the cloud on the server makes no difference in this case.

 

[msackey]

5 days in and it's hacked. Looks really promising

The remark above seems to be coming from someone who has no idea what they’re talking about.

 

[scvrx]

I'm sure a lot of people here do understand what is going on but you can't deny that there are a lot of people also here who don't understand. In this small thread alone we have people asking if album covers will get them in trouble, denying that they authorized Apple's scanning, and those who don't believe that the scanning is disabled if you don't use iCloud.

People are trusting the experts. And in this case “the **** storm” is just starting. I ended 20 years expensive relationship with Apple after rational assessment of technical implementation and law perspective ( you can search the videos that I have posted). This situation has alerted a lot of people about the actual meaning of privacy (not some trillion dollar corporation redefinition) and that the only way to trust software is trough open source. Period. We all have slept enough about this, thanks to Apple God complex we are not anymore.

I will not buy another Apple product ever. They want every aspect of your life and data in their hands. Financial. Emotional. Health. Transportation. You name it. Can you imagine “what are they saving next, and how this will affect your life?”

Apple just added new revenue stream to the portfolio : Data brokers. Huge market. Huge profits. Lots of potential high-level customers, oppressive regimes, governments. And the funniest thing is that we are providing “the petrol” and paying for the “mining”.

Grow up people. Things are serious. Don’t drink the corporate propaganda. Think critically , not different.

 

[jdb8167]

People are trusting the experts. And in this case “the **** storm” is just starting. I ended 20 years expensive relationship with Apple after rational assessment of technical implementation and law perspective ( you can search the videos that I have posted). This situation has alerted a lot of people about the actual meaning of privacy (not some trillion dollar corporation redefinition) and that the only way to trust software is trough open source. Period. We all have slept enough about this, thanks to Apple God complex we are not anymore.

I will not buy another Apple product ever. They want every aspect of your life and data in their hands. Financial. Emotional. Health. Transportation. You name it. Can you imagine “what are they saving next, and how this will affect your life?”

Apple just added new revenue stream to the portfolio : Data brokers. Huge market. Huge profits. Lots of potential high-level customers, oppressive regimes, governments. And the funniest thing is that we are providing “the petrol” and paying for the “mining”.

Grow up people. Things are serious. Don’t drink the corporate propaganda. Think cryptically, not different.

What's the solution though? I'm thinking about switching to Linux but things will have to get a lot worse before I'm willing to do that. It's not like switching to Windows or a Chromebook is going to have any different consequences. You are trading off one trillion dollar company for another.

Ultimately, anyone who cares about privacy is probably going to have to think about switching to an open source solution. I'm pretty expert at IT and software development and switching to Linux as my primary workstation doesn't thrill me. I use Linux for server side stuff and that's fine but desktop is a lot more work than I'm willing to do right now. But if things get worse with Apple, Microsoft, Google, and others, then there might be no real choice.

 

[chucker23n1]

But again, the iPhone tagging software has access to all that too except the hash database.

Given how heavily Apple has been moving towards sandboxing, I wouldn't be surprised if all the tagging process has access to is photos, period — not even the file system, much less networking. (That's assuming it is a separate process, and should it ever get exploited, Apple will probably make it one.)

I haven't investigated but I'm pretty sure that the iPhoto tagging is synced across devices via iCloud.

Probably — but the process that writes the tags locally doesn't need to be the one that uploads them.

What is dangerous about having access to the NCMEC hash database?

Read access? Probably not much; it sounds like Apple even wants to give people explicit access to it. Write access, wellllllll.

 

[chucker23n1]

Y’know, there’s something not right about putting child porn hashes on everybody‘s phone. Even if I can’t see them, knowing they‘re there is … yeeuck.

Who the hell signed off on this?

They're hashes. Basically checksums. They don't really contain the data, or even a portion of the data; they're a fraction of the size of the data, and all they're good for is you can take the original data and verify a match (or not).

Apple's OSes also contain XProtect, which are hashes of malware — doesn't mean your devices have malware on them.

 

[JetLaw]

This whole mess makes it really hard to defend Apple. They are the ones who choose to build their business model by claiming to be privacy-first. At least with Google you know what you’re getting. Apple seems to be so proud of having created this CSAM scanning technology but if I’m going to be surveiled by my cloud provider it may as well be Google. At least with Google Photos the invasive scanning gives me the benefit of amazing search results in my image library. Meanwhile Apple can’t tell the difference between a photo of my cat and a photo of my car.

 

[TheWatchfulOne]

Will CSAM report my Nirvana Nevermind Album Cover in iTunes?

If you get in trouble for having that, would Apple get in trouble for selling it to you?

 

[JetLaw]

...and that's with "cryptographic" hashes that are designed to give different hashes from even slightly different sources with a low chance of collisions. This system is using perceptual hashes that are an image recognition technique to give the same hash for visually similar images (to detect images that have been cropped, scaled, changed in quality etc.) See https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf

- maybe the technique is extremely effective, but the cryptographic hash vs. perceptual hash distinction is rather important in principle.

I couldn’t agree more. If they were using a cryptographic hash function to search only for exact-matches then I would hate it less. The simple fact is that they are relying on layers of automated systems and humans to avoid inevitable hash collisions from trigging a false SWAT of peoples’ homes, and the idea that completely innocent people are going to get escalated in that system is reason enough for me to opt out.

 

[Mebsat]

This will lead to activists creating publicly downloadable collision images. Then we can all add hundreds of them to our iCloud libraries and waste thousands of hours of Apple's review time. And then this nonsense will be over.

 

[neuropsychguy]

When I first heard about this, my gut reaction was “good”. As I read more about it I switched to, “I don’t think Apple should do this.” As I’ve read more and more about it and taken the time to understand it, I’m back to “Good”.

I know many here disagree with Apple’s implementing this but I don’t. We don't know enough to know if this is a good move or not long-term. Saying it will be abused by bad actors (governments) is making a prediction and a slippery slope fallacy. This is not a backdoor into the OS. This is not even mandatory if you turn off iCloud sync for your photos. I respect the privacy of others but this seems like a balanced approach to combat CSAM.

 

[cyanite]

Wait - I’m confused by this. Does this mean that everyone’s iCloud is going to be scanned without user’s authorization in the name of child welfare??

You would be less confused, and make less incorrect statements, by reading the technical description as well as other documents: https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf

5 days in and it's hacked. Looks really promising

You apparently completely misunderstand how it works.

Matthew Green, who teaches cryptography at Johns Hopkins University

BAM! 👊 POW! 💥
But people in here will still side with Apple, unbelievable. 🤦‍♂️

Mr. Green is either misinformed about the specifics or talking about something else. It’s not a surprise that collisions can be found in a hash scheme that specifically seeks to maximize collisions for “similar looking” images. This is not, by design, a cryptographic hash.

yes. every photo on your iphone will converted to a "hash" by your phone before uploading to icloud. the hash will be sent along with the photo. the hashes (not the actual image) will be scanned to see if they match a know child abuse photo hash and if you have more than approximately 30 such images in your icloud collection a human being at Apple will be able to view just those images to verify they are indeed images that match known child abuse imagery. if so law enforcement will be notified.

like you said the fear is apple could use the same technology to search someday for other forms of imagery and could be coerced to do so by foreign governments. they promise they won't but the backdoor is there now.

This is pretty oversimplified and you should read the technical summary as well.

 

[smoking monkey]

Apple are going to have to pull this whole thing.

They have begrudgingly bowed to public pressure before. So who knows.

Someone on Twitter mentioned what happens if someone airdrops you a bunch of illicit photos and they sync to iCloud in a matter of seconds? Boom you’re flagged.

First problem with this is, "Someone on Twitter". Haha. Never trust anything you read of that cesspool. You have to accept airdrops and ALSO have to have it turned on to everybody for that to even be possible. Switch it to contacts only or only turn it on when needed.

Mass surveillance in the making, and you are paying for it.

That bird has already flown.

This is beginning to look really bad for Apple.

It certainly is on here and in the techworld. Wonder about normal folks world though? Do most people even know about this? Once MSM starts reporting on it, then it could be. But will they?

- maybe the technique is extremely effective, but the cryptographic hash vs. perceptual hash distinction is rather important in principle.

Thanks. Important to have the full information.

I would love to see this entire “feature” scrapped but I won’t hold my breath.

Apple could let this fade into the distance if there is enough hot air blown about it. Or they could change it to just be in the cloud.

Most pedos are android users to begin with lol

I don't know anybody in Hollywood who uses Android!


Moral Dilemma Question: How many Ps does this need to catch for it to be worth it?

It feels a bit like a rule change in your favorite sport. The league has good intentions as does the rule, but once it's implemented the coaches always find a way to exploit it.

I understand both sides of the argument. I have no answers.

 

[cyanite]

As Rene Ritchie says on MacBreak Weekly, Apple keeps talking down to us as if we don't understand, and our response is "You don't understand, we understand and do not like this"

I don’t know about you specfically, but people here definitely don’t understand in general, and it’s the same on Reddit. There is so much misinformation about this it’s crazy. So yeah the reason Apple keeps releasing more technical documentation really is because people don’t understand, as I see it.

 

[scvrx]

What's the solution though? I'm thinking about switching to Linux but things will have to get a lot worse before I'm willing to do that. It's not like switching to Windows or a Chromebook is going to have any different consequences. You are trading off one trillion dollar company for another.

Ultimately, anyone who cares about privacy is probably going to have to think about switching to an open source solution. I'm pretty expert at IT and software development and switching to Linux as my primary workstation doesn't thrill me. I use Linux for server side stuff and that's fine but desktop is a lot more work than I'm willing to do right now. But if things get worse with Apple, Microsoft, Google, and others, then there might be no real choice.

I have switched my entire business operation (web development/ design/ video) on Linux. For me this was time expensive and frustrating. We created entirely new workflows and worked overtime to avoid any mistakes for our clients.
If you cannot switch immediately, my suggestion is:
Stay on Catalina and don‘t update. Invest in multi-platform software.
We are running some design apps on Windows VMs.

Apple killed macOS long ago. The power of MacOS X on Intel was the software interoperability. With further closing of the os and constant unification of the Apple platforms, macOS as a professional tool is loosing “the aura”.

For me this was a tipping point. My computer is not a toy and my work and my clients files will not be ”scanned“ by third parties. Period.
I am obliged to protect my clients data by contracts and serious part of this is privacy and security, so this is no brainer.

The other motivating factor for moving to FOOS is my instinct.
Things will get wildly intrusive after this Apple “innovation”.

 

[cyanite]

They have begrudgingly bowed to public pressure before. So who knows.

Well, let’s hope not. Giving in to the mob is not the right step.

It certainly is on here and in the techworld. Wonder about normal folks world though? Do most people even know about this? Once MSM starts reporting on it, then it could be. But will they?


Thanks. Important to have the full information.

Yeah it is. And almost no one here bothers to get that. Read the source documents from Apple!

 

[Digital Dude]

I may switch to Google at this rate; at least they don’t masquerade around the issue.

 

[cwosigns]

Well Apple doesn't want your illegal child porn on their servers so if you choose to use their iCloud photo library they will make sure what you are sending them doesn't contain that kind of material.

Don't like it? Don't use iCloud.

Fine. Just roll over and give up your 4th amendment protections.