Apple and Google Hit With Demand to Pull AI 'Nudify' Apps

[MacRumors]

A San Francisco City Attorney has sent cease-and-desist letters to Apple and Google demanding the removal of 13 AI "face-swap" apps that can generate nonconsensual nude images, according to WIRED.

The letters, sent on Thursday, target eight apps on the App Store and five on the Play Store that market themselves as face-swapping tools but are used to "undress" photos of real people. The attorney's office wants both companies to cut ties with the developers and stop taking a cut of in-app payments, arguing they are effectively "aiding and abetting" the sale of explicit deepfake images. The attorney called the practice "illegal, harmful, and completely unacceptable" and believes Apple and Google have collected millions in fees from the apps.

Both companies have developer rules against pornography and have removed batches of nudify apps in the past after being flagged by researchers. A Google spokesperson told WIRED that the company has deleted "hundreds" of apps with nudifying features for policy violations, including the five named in the letters. In June, it emerged that Apple had already tightened its App Store guideline language on developer responsibility for pornographic content.

Note: Due to the political or social nature of the discussion regarding this topic, the discussion thread is located in our Political News forum. All forum members and site visitors are welcome to read and follow the thread, but posting is limited to forum members with at least 100 posts.

Article Link: Apple and Google Hit With Demand to Pull AI 'Nudify' Apps

 

[ArtOfWarfare]

I can't help but feel that these demands are actually coming from Adobe or some other tech or AI company that sells photo manipulation tools, looking to cut down on their competition.

Trying to call silly "face swap" apps as being malicious deep fake or nudify or involuntary porn apps is quite the stretch.

 

[k1121j]

Censorship at its finest. If this whole App Store has age restrictions why do we need this?

 

[cjsuk]

Censorship at its finest. If this whole App Store has age restrictions why do we need this?

I’m fine with these restrictions.

Nudify apps are used to blackmail people.

 

[Mac Fly (film)]

Porn websites have turned into total brainwashing websites, I'm convinced. Also, no matter what you try to search certain things are hoisted on you, on repeat. Unsure who are building these website but they are a strange bunch with an odd agenda. Also, who's pushing all the violence on porn websites, and where did all of that even come from? It is understandable how lonely men can develop porn addictions, but be careful out there you are a lab rat in an experiment and you've no idea who's running it.

 

[I7guy]

This smacks of censorship, it’s not that these apps invented a way to harass people. Like other things tech has made it easier to be a bad actor. It’s like sending a ceased and desist letter to the gun manufacturers.

 

[Macalway]

A losing battle.

 

[groove-agent]

I agree these apps shouldn't be allowed. I think using apps to generate images/ videos of real people is not only an infringement on their rights but also could be used for nefarious purposes (blackmail etc).

For example, all these Star Wars AI videos using the original actors shouldn't be allowed. The actors should be compensated for their use.

 

[maruyama]

There is a huge difference between an app that is specifically intended to take an image and make the subject of it appear nude (which would already be against the App Store terms, so if those were getting through... wow, Apple) and an app intended to swap faces, which can be used for anything.

If face-swapping apps should be banned because they can be used for "nudifying", then messaging apps should be banned because they can be used to send fake nudes.

 

[sw1tcher]

Trying to call silly "face swap" apps as being malicious deep fake or nudify or involuntary porn apps is quite the stretch.

Censorship at its finest. If this whole App Store has age restrictions why do we need this?

You might feel differently if someone was using such apps to harass your teen

Teen girls are being victimized by deepfake nudes. One family is pushing for more protections


Some have committed suicide.

Girl killed herself when bullies shared fake nudes of her

A teen died after being blackmailed with A.I.-generated nudes. His family is fighting for change

 

[I7guy]

You might feel differently if someone was using such apps to harass your teen

Teen girls are being victimized by deepfake nudes. One family is pushing for more protections


Some have committed suicide.

Girl killed herself when bullies shared fake nudes of her

A teen died after being blackmailed with A.I.-generated nudes. His family is fighting for change

Or accept that tech use has downsides and that tech can be appropriated for not so nice uses.

I wouldn’t deny that’s horrible but bad actors always find a way to abuse systems, processes and tech. Maybe we should ban AI that would really resolve the issue.

 

[SpotOnT]

Is animated nudity even considered pornography?

If you draw someone punching someone else, is that considered assault?

 

[East India Company]

But but i thought the 30% cut Apple charges is used to regulate the apps on the store so that all apps are safe and we are all protected? For sure Apple would already have noticed this and taken action by itself, right?

 

[HouseLannister]

Sounds like App Review is doing their usual stellar job /s

 

[Saturn1217]

I am anti censorship but this is not censorship. An app designed to create a photo that misrepresents a real person is just set up for abuse. Nudes is one thing. But there are plenty of other reality bending abuses this type of app can do. The whole point of these apps is to put a real person’s face in a situation they were not really a part of and make it look real. That entire concept is too dangerous to put in an easy to use packaged app for anyone to toy around with. It just is. The same way there are legit uses for guns and explosives but you probably shouldn’t make it easy for everyone to have them.

There is evidence for real harm being done by these apps. And importantly the risk is not opt in. Just because I choose not to install the app does not protect me from what it can do. That’s the difference. It is my responsibility as a parent to monitor the content my child consumes. I don’t need Apple or anyone else censoring that content. That should be my job. But I can’t be a good enough parent to stop my kid from being a victim of someone else’s abuse of these apps.

 

[jeroenvip]

Don't give apple any idea's please

 

[Jumpthesnark]

I learned a new word today: "nudify."

Developers who create apps that allow people to make nonconsensual nudes of others are disgusting in my opinion. Sadly the devs make a lot of money from people who want to make these images, so the incentive is there for them to continue making them.

According to the Wired story, Google and Apple have removed many apps from their stores that do this, hopefully they'll take down the 13 cited in this suit, and hopefully they will continue to remove these apps, which are a violation of California law.

 

[Jason2000]

It is understandable how lonely men can develop porn addictions, but be careful out there you are a lab rat in an experiment and you've no idea who's running it.

Its not just men anymore.

 

[Ja50n]

Setting aside the majority of this article… why is the C&D coming from a San Francisco City Attorney? Wouldn’t this normally come from a state or federal level?

 

[KENESS]

Is animated nudity even considered pornography?

If you draw someone punching someone else, is that considered assault?

Umm, yes, to the first question.

I'm confused by your confusion on the matter...

Also, no, to the second. That's ridiculous. Why? Because a PICTURE of a punch, real or drawn, is not actually a punch.

Whereas a picture of a nude person, real or drawn, is, in fact, a picture of a nude person.

It's not... complicated.

 

[AppleMega]

If they comply, then twitterX needs to be removed too. Not just nudes via grok.

 

[Jumpthesnark]

.

 

[Jumpthesnark]

Is animated nudity even considered pornography?

If you draw someone punching someone else, is that considered assault?

Deepfake nudity is illegal in most states by far. No matter what word you use for it. A federal law also passed two months ago.

 

[Atog]

Porn websites have turned into total brainwashing websites, I'm convinced. Also, no matter what you try to search certain things are hoisted on you, on repeat. Unsure who are building these website but they are a strange bunch with an odd agenda. Also, who's pushing all the violence on porn websites, and where did all of that even come from? It is understandable how lonely men can develop porn addictions, but be careful out there you are a lab rat in an experiment and you've no idea who's running it.

Tbh porn quality went so downhill that nowadays it's more embarrassing to watch it then to get aroused by it. And Softcore which was at least about some beauty as some kind of an erotic photography is almost dead as well. So now best people have is nudifying apps.. well crap.

 

[SpotOnT]

Deepfake nudity is illegal in most states by far. No matter what word you use for it. A federal law also passed two months ago.

Ah ok - that is helpful.