Apple Threatened to Pull Grok From App Store Over Sexualized Images

[MacRumors]

Apple privately warned Elon Musk's xAI company in January that it would remove the Grok app from the App Store unless the company put a stop to the chatbot's nude and sexualized deepfakes, according to a letter Apple sent to U.S. senators and obtained by NBC News ($).

Earlier this year, Grok's AI capabilities came under scrutiny after X users shared nonconsensual sexualized images of women and children created by the app, many of which were based on photos of real people.

What followed was a confusing rollout of moderation changes to Grok, some of which could be easily bypassed. Publicly, Apple did not comment on the controversy at the time, but it did respond, and was in fact the instigator of the changes. Internally, the company had found both X and Grok in violation of its App Store guidelines and demanded its developers submit a content moderation plan, the letter reveals.

According to the letter, Apple rejected an initial fix from xAI as insufficient, saying the "changes didn't go far enough," and Apple warned it that additional alterations were required or Grok would be removed. After further back-and-forth, however, Apple eventually concluded that a later submission of the app had improved enough for it to be approved.

The disclosure was apparently prompted by a January letter from Senators Ron Wyden, Ben Ray Luján, and Edward Markey, who urged Apple and Google to pull both apps, arguing the imagery violated App Store rules barring offensive, sexual, and exploitative content.

The senators also said that Apple's response would test its own arguments, since the company has long defended its curated App Store by claiming its review process keeps users safer. Letting Grok continue to generate this kind of imagery, they argued, would undermine that case in the eyes of the public and in a court of law.

After NBC News published its report, X posted the following statement on its platform:

"We strictly prohibit users from generating non-consensual explicit deepfakes and from using our tools to undress real people. xAI has extensive safeguards in place to prevent such misuse, such as continuous monitoring of public usage, analysis of evasion attempts in real time, frequent model updates, prompt filters, and additional safeguards."

While the amount of sexualized deepfakes created by Grok and posted to X appears to have decreased significantly, NBC News found that Grok is still able to generate similar imagery, with some users apparently having simply updated their prompt tactics to get around the safeguards. You can read that report in its entirety by following this link.

Note: Due to the political or social nature of the discussion regarding this topic, the discussion thread is located in our Political News forum. All forum members and site visitors are welcome to read and follow the thread, but posting is limited to forum members with at least 100 posts.

Article Link: Apple Threatened to Pull Grok From App Store Over Sexualized Images

 

[ghostface147]

Should’ve followed through as you can still make these types of pics.

 

[EugW]

Yeah, just remove Grok. And remove X while you’re at it.

 

[BabyBoii]

@Grok, put macrumors in a bikini

 

[Eminemdrdre00]

"Threatened"

 

[johannnn]

Imagine the crybaby Elon would become if Grok disappeared from the App Store

 

[Zenman12]

Barring the limited answers, I found Grok to be the most useful in answers. Then there’s Gemini. Chat GPT is the worst.

I do think Apple is doing the right thing here. Give them a deadline, then kick them off. You could get sued too with Elon and Grok. If Elon and Grok want to be stupid and play with fire, having AI generated CP, don’t get roped in with something obviously stupid.

 

[sw1tcher]

Internally, the company had found both X and Grok in violation of its App Store guidelines and demanded its developers submit a content moderation plan, the letter reveals.

According to the letter, Apple rejected an initial fix from xAI as insufficient, saying the "changes didn't go far enough," and Apple warned it that additional alterations were required or Grok would be removed. After further back-and-forth, however, Apple eventually concluded that a later submission of the app had improved enough for it to be approved.

Apple/Tim Cook giving X special treatment.

If this was a small app developer, the app would have been pulled first, requiring a fix for the app to appear on the App Store.

Tim Cook: "We treat every developer the same. We have open and transparent rules, it's a rigorous process. Because we care so deeply about privacy and security and quality we do look at every app before it goes on. But those rules apply evenly to everyone."

 

[cateye]

For those keeping score:

Have your silly live service game link sneakily to an outside payment method? IMMEDIATE BANISHMENT AND YEARS OF LITIGATION; THIS INJUSTICE WILL NOT STAND.

Update your app to make sexualized images of teens and children against their will? Strongly worded memo. Maybe a finger wag.

🎶 The grabbing hands, grab all they can, all for themselves... after all, it's a competitive world... Everything counts in large amounts 🎶

 

[nt5672]

Stupid and political. You can do the same thing with Safari, are they gonna pull Safari off the App Store also?

 

[rjp1]

This is the reason people want to install apps outside of the App Store. Ultimately, nobody wants someone else’s morality forced on them. In this case, many will cheer this on, because they don’t like Elon. That is shortsighted, because what if it is an app that you support?

 

[Zenman12]

This is the reason people want to install apps outside of the App Store. Ultimately, nobody wants someone else’s morality forced on them. In this case, many will cheer this on, because they don’t like Elon. That is shortsighted, because what if it is an app that you support?

Bro if you want to take the side of allowing CP on an app otherwise that is “someone else’s morality forced upon them” then be explicit about it. You can die on that hill.

 

[cateye]

Stupid and political. You can do the same thing with Safari, are they gonna pull Safari off the App Store also?

Strawman argument. Apple professes no control or curation of the web.

Apple does, however, strenuously defends its gatekeeper status on the App Store under the guideline that it specifically curates and rejects "objectionable, harmful, and illegal material." Let's say I run a porn web site. People are free to access that porn website with Safari. The moment I submit an app called "Free Porn" Apple will reject it. There is zero conflict in this.

If Apple does not want to be put in a position to make these judgements, it is free not to. It can then stop arguing under oath that this is the primary reason why it wants to maintain a closed App Store and that it has nothing at all to do with money or influence.

 

[turbineseaplane]

Yet they did not remove it, and Grok remains quite capable of this gross behavior even today.

So I'm not really sure where this situation actually "is" in Apple's mind at this point.

Does it just have to be "a little bit harder to make sexualized nudes" and it's "ok for the App Store!"?

 

[sw1tcher]

This is the reason people want to install apps outside of the App Store. Ultimately, nobody wants someone else’s morality forced on them. In this case, many will cheer this on, because they don’t like Elon. That is shortsighted, because what if it is an app that you support?

Aside from sickos, who supports an app to create child porn?

EDIT.... Hmm... one thumbs down. Should I take the thumbs down to mean you support child porn? How else am I suppose to read that? SMH

 

[HouseLannister]

Apple did not comment on the controversy at the time, but it did respond, and was in fact the instigator of the changes.

Pretty sure the broad negative press surrounding the whole incident was the instigator. Journalism trumps strongly worded letters from one lawyer to another lawyer.

 

[Dr McKay]

I look forward to the well thought out and completely reasonable response to this from Elon.

 

[kagharaht]

Why is that a problem? Apple supports an Adjudicated RAPIST & Convicted Felon by 34 counts. I mean Timmy actually visited the Perp and gifted him... this is a double standard. Hypocritical of Apple to do this.

 

[Hajj.david]

Yet they did not do remove it, and Grok remains quite capable of this gross behavior even today.

So I'm not really sure where this situation actually "is" in Apple's mind at this point.

Does it just have to be "a little bit harder to make sexualized nudes" and it's "ok for the App Store!"?

You could just not prompt it to give you fake CP, unless thats your thing. Its like complaining that the hammer from the hardware store killed my neighbor, I could just NOT use it as a weapon and be ok.

 

[diamond3]

This is a very difficult line to establish. Safari - access all the crude content in the world, but it's okay. Telegram/Whatsapp/etc - people can chat with underage minors on there so we should ban it. Grok, people can abuse the platform inappropriately, so ban it. Why is it the outlier?

Now I feel it's okay if Apple wants to push them to improve the limitations of creating that type of content, but threatening them with removal see problematic.

Aside from sickos, who supports an app to create child porn?

EDIT.... Hmm... one thumbs down. Should I take the thumbs down to mean you support child porn? How else am I suppose to read that? SMH

If you want to view it that way, why do you support an internet browser or messaging apps? Just because something can be used for bad doesn't make the app bad.

 

[BaldiMac]

You could just not prompt it to give you fake CP, unless thats your thing. Its like complaining that the hammer from the hardware store killed my neighbor, I could just NOT use it as a weapon and be ok.

No, it’s like if you asked a guy at the hardware store to use the hammer to kill your neighbor, and he did.

Musk’s company isn’t simply distributing a tool. They are the ones creating and distributing the CSAM and non consensual sexual material. It’s their app that they control running on their servers. “But someone asked me to make it” shouldn’t be a defense.

 

[HouseLannister]

Grok, people can abuse the platform inappropriately, so ban it. Why is it the outlier?

Because X was unprompted surfacing nonconsensual images generated by Grok to users regardless of their searches or followed accounts. They should have removed X for that reason, but I guess Grok was the lower-hanging fruit(?)

 

[rjp1]

Bro if you want to take the side of allowing CP on an app otherwise that is “someone else’s morality forced upon them” then be explicit about it. You can die on that hill.

Aside from sickos, who supports an app to create child porn?

EDIT.... Hmm... one thumbs down. Should I take the thumbs down to mean you support child porn? How else am I suppose to read that? SMH

This is more than child porn.

If you want to limit it to child porn, then why remove just this app? You can open Notes and draw child porn. You can store child porn pictures in Photos. You could take pictures of it with the Camera app. You could record audio of it with Voice Memos.

I'm sure you have a well reasoned argument of why Apple's apps aren't normally used for that and they are fine to stay. Apply that logic to this app. It is no different. That is why Apple shouldn't be deciding what app is "good" or "bad" - that is morality issue.

 

[BaldiMac]

This is more than child porn.

If you want to limit it to child porn, then why remove just this app? You can open Notes and draw child porn. You can store child porn pictures in Photos. You could take pictures of it with the Camera app. You could record audio of it with Voice Memos.

I'm sure you have a well reasoned argument of why Apple's apps aren't normally used for that and they are fine to stay. Apply that logic to this app. It is no different. That is why Apple shouldn't be deciding what app is "good" or "bad" - that is morality issue.

Because there is a difference between buying a camera and taking illegal pictures and asking a company to create and distribute illegal pictures.

 

[Zenman12]

This is more than child porn.

If you want to limit it to child porn, then why remove just this app? You can open Notes and draw child porn. You can store child porn pictures in Photos. You could take pictures of it with the Camera app. You could record audio of it with Voice Memos.

I'm sure you have a well reasoned argument of why Apple's apps aren't normally used for that and they are fine to stay. Apply that logic to this app. It is no different. That is why Apple shouldn't be deciding what app is "good" or "bad" - that is morality issue.

You’re right that it’s more than that. I didn’t read as thoroughly. However, I do know CP is the issue spearheading this whole controversy (and unauthorized sexual images of others without their consent follows it [which is illegal now in many countries]).

If you are fine with CP and other sexualized images of people without their consent so you can have your fun app, then just own it, rather than this roundabout game of whataboutism regarding other apps.

It is not the same. Drawing with your finger in the Notes app is not the same as a realistic computer generated image. And downloading (already illegal) in photos and is again a different thing entirely to something being created and generated. Creating false equivalencies will not help you here.

I am fine banning companies off of platforms who will not clamp down on sexual images of children. Period. Done.