My main concern with the Messages portion of this (which uses on-device machine learning to determine if you've received or are trying to send a nude photo of a child) is that this could be co-opted by repressive regimes to detect whatever they want.
Russia for instance has made laws against "promoting" the Lesbian and Gay "ideology". Which essentially boils down to banning depictions of gay couples in media and making it illegal to advocate for their way of life, to ban the attempts to normalise homosexuality.
Combined with that Russia has forced Apple to allow preloading of apps on the iPhone from Russian companies. Did Apple stand up and say no this is our phone we would rather pull out of Russia than do that? No. They folded like a deck of cards.
What happens if Russia says we want you to expand this child exploitation technology in the Messages app to include homosexual imagery? like detecting pride flags, same-sex hand holding and so on? does Apple tell them no we're not doing that, we'll rather lose the Russian market and not sell iPhone there or do they fold again?
Apple made a great argument against governmental backdoors that once you make that door you've made the product less secure and it's ultimately just steps away from abuse.
I'm all for them scanning photos uploaded to their iCloud servers. That is their domain. But what happens on the phone or between two phone users in a private message chain should stay between those individuals. There is too much potential for abuse.
It's a lot easier to force a company to adjust something they've already made than it is to convince courts to press a company to create something from scratch which are the sorts of cases Apple has won in the past.
For instance imagine Apple didn't have an app store and didn't allow 3rd party apps on the iPhone at all then Russia has this law to pre-load Russian apps? That would be hard for a court to press Apple on because the apparatus/infrastructure to allow for that circumstance hasn't been created. But since it already exists forcing Apple to add a few apps is seen as easy-peasy.
Well the messages scanning thing has now been created, expect governments around the world to want to expand it to fight "terrorism", protecting the children and more. In-fact Russia's own LGBT laws have been put up to "save the children" so I'd expect them to be one of the first to want this system expanded beyond Apples remit.
