Apple and Google Hit With Demand to Pull AI 'Nudify' Apps [Updated]

[TruthAboveAllElse]

Deepfake nudity is illegal in most states by far. No matter what word you use for it. A federal law also passed two months ago.

It could be challenged as a violation of freedom of expression at some point.

 

[ChromeAce]

Let’s ban all image editing tools.

 

[VulchR]

I can't help but feel that these demands are actually coming from Adobe or some other tech or AI company that sells photo manipulation tools, looking to cut down on their competition.

Trying to call silly "face swap" apps as being malicious deep fake or nudify or involuntary porn apps is quite the stretch.

Interesting take on the situation. It is true that generic image manipulation apps can be used in this way, so where does one draw the line? For instance, generic image-editing apps are often used in to edit images in commercial pornography, so does that mean that Photoshop should be banned? It seems to me the line should be drawn at the output and not at the app. Drawing the line at apps by banning them is a bit like banning cars because they can be used as getaway vehicles for robberies. Unless there is some sort of specialised function in the apps that makes them suited only to one purpose, this sounds a little over the top. (FWIW I don't know because I have never used these apps).

 

[vantelimus]

It appears Google and Apple are already proactive in taking down apps that break California deep-fake laws, having deleted hundreds. The fact that they only missed 13 is amazing. It also appears that these apps were specifically trying to be sneaky to avoid detection. If the apps are breaking California deep-fake laws, then merely notifying Apple and Google is sufficient to get them to take them down. Chiu's allegations of aiding and abetting are demagoguery.

 

[Astuces iOS]

Deepfake nudity is illegal in most states by far. No matter what word you use for it. A federal law also passed two months ago.

Hmmm you should be alright if you ask the permission I think. This thing is like a gun

 

[Jumpthesnark]

It could be challenged as a violation of freedom of expression at some point.

States started passing these laws in 2019, people have had many opportunities to defend what they do. I don't think anyone has been interested in coming forward to say "I should have the right to make and share nonconsensual deepfake nudes, including of minors," because not a lot of juries are gonna side with that argument.

The point of the laws that have been passed is that this is not a victimless crime.

 

[TMRJIJ]

Tbh porn quality went so downhill that nowadays it's more embarrassing to watch it then to get aroused by it. And Softcore which was at least about some beauty as some kind of an erotic photography is almost dead as well. So now best people have is nudifying apps.. well crap.

I never knew porn to be considered ‘quality’. Who determines that level of quality? Are most OF models considered ‘low quality’ since they didn’t get into the major adult studios? Is there an adult version of the Oscars?

Hmmm you should be alright if you ask the permission I think. This thing is like a gun

What would be the purpose of these apps if you had permission? The person you’re asking could just send you real nudes then.

 

[DEMinSoCAL]

I’m fine with these restrictions.

Nudify apps are used to blackmail people.

Guns are used to kill people. But not most guns. I would bet the majority of the images created are for curious people to try and to post elsewhere, not for actual criminal extortion.

 

[Plutonius]

Are these apps illegal ?

 

[gaximus]

I’m fine with these restrictions.

Nudify apps are used to blackmail people.

And WhatsApp, Messages, RCS, Facebook messenger, etc were used for terrorist attacks. We should just abolish the AppStore completely if bad people can do bad things with Apps.

 

[I7guy]

But but i thought the 30% cut Apple charges is used to regulate the apps on the store so that all apps are safe and we are all protected? For sure Apple would already have noticed this and taken action by itself, right?

It’s true. You thought right. So we should ban any useful app where someone has found a way to somehow find a downside?

 

[ChrisA]

What if I got two paper photos and some scissors, and I cut off the head of one and swapped it with the other, then got some paint and painted over the cut marks? This is doing the same face swap but only with 100+ year-old technology.

Then we can use 30+ year-old photo technology called "Photoshop" to emulate the cut and paste.

What's different between the three methods? Only the amount of time and skill required. But logically, if you can do one method, you would have to ban them all.

So now we ban scissors and paint.

A better solution is to possibly have the app self-sensor and refuse to create a certain class of results.

 

[GumaRodak]

As usually, all new developed new tech is firstly used for military, porn and scam…

 

[Morod]

Glad I live under a rock. I never knew any of this stuff existed! (Yes I am an old fart…)

 

[tomtad]

Censorship at its finest. If this whole App Store has age restrictions why do we need this?

Na, you don’t have the right to do what the 🫪 you want

 

[TMRJIJ]

Guns are used to kill people. But not most guns. I would bet the majority of the images created are for curious people to try and to post elsewhere, not for actual criminal extortion.

This is a bad analogy. The ‘sole purpose’ of guns are to harm or kill another living thing.

I’m fine with these restrictions.

Nudify apps are used to blackmail people.

These apps most likely were not specifically ‘Nudify’ apps. They were face-swapping apps that bad actors are using for malicious purposes. Any app for that matter could be misused in such a way.

 

[turbineseaplane]

It’d be nice if Apple and Google were more aggressively policing this on their own.

Apple, in particular, is supposed to be doing more of that as a benefit of their App Store model.

 

[turbineseaplane]

But but i thought the 30% cut Apple charges is used to regulate the apps on the store so that all apps are safe and we are all protected? For sure Apple would already have noticed this and taken action by itself, right?

Apple mostly stops listening after the part where they take a 30% cut…

 

[TruthAboveAllElse]

States started passing these laws in 2019, people have had many opportunities to defend what they do. I don't think anyone has been interested in coming forward to say "I should have the right to make and share nonconsensual deepfake nudes, including of minors," because not a lot of juries are gonna side with that argument.

The point of the laws that have been passed is that this is not a victimless crime.

A law being challenged as being unconstitutional would not be decided by jurors. And I wasn’t talking about when in involves minors. There’s still lots of time for it to end up in court.

 

[Astuces iOS]

I never knew porn to be considered ‘quality’. Who determines that level of quality? Are most OF models considered ‘low quality’ since they didn’t get into the major adult studios? Is there an adult version of the Oscars?


What would be the purpose of these apps if you had permission? The person you’re asking could just send you real nudes then.

Because some are real pictures, other are not, could be defended if somehow it goes public

 

[arkhanjel]

Yes, Apple and Google should be policing the App Store for stuff like this. But ultimately this falls on the developer of each of these apps. If they legitimately wanted their apps to do innocent AI face swaps they could just as easily use that same AI to detect when people are trying to use a nude or inappropriate photo in the swap. The fact that they don't do that kinda means that's what they knew, maybe even wanted to happen.

 

[Jumpthesnark]

A law being challenged as being unconstitutional would not be decided by jurors. And I wasn’t talking about when in involves minors. There’s still lots of time for it to end up in court.

You can handwave away your comment by saying you weren't talking about minors, but these laws are about nonconsensual use, including minors (such images are already recognized as obscene) and adults. Best of luck waiting for that First Amendment challenge.

 

[TracerAnalog]

I’m fine with these restrictions.

Nudify apps are used to blackmail people.

This is not about nudify apps. This is about face swaps, like Snapchat had or has.

 

[mwhite]

If somebody where pasting my head on someone else's hot, nude, well endowed body, I'd be like, "Go for it! Disemminate the images far and wide!"

 

[TruthAboveAllElse]

You can handwave away your comment by saying you weren't talking about minors, but these laws are about nonconsensual use, including minors (such images are already recognized as obscene) and adults. Best of luck waiting for that First Amendment challenge.

There’s no way it would stand that it could be used on minors that way. There is a chance it could for adults. Do people need to consent to being painted?