San Francisco seeks removal of AI nudify apps from Apple and Google stores
David Chiu demanded that Apple and Google take down 13 apps he says violate California laws against deepfake pornography services.
By Hana Yoshida · Markets Reporter
3 min read
San Francisco Attorney General David Chiu has demanded that Apple and Google remove 13 AI “nudification” apps from their app stores, Wired reported. The action targets tools that can turn ordinary photos of real people into sexual images, a fast-growing abuse risk for women and children.
Chiu sent cease-and-desist letters this week accusing the companies of allowing apps that run afoul of California laws barring services that help create deepfake pornography, according to Wired. The letters asked Google to remove five apps and Apple to remove eight, Wired reported after reviewing them.
The apps were not identified publicly, which Wired said was meant to avoid directing people to them. One app had more than 1 million downloads and promoted features for sexualizing images of women or making “free and uncensored” videos, according to Wired.
Chiu says app stores are profiting from abuse
Nudification apps can be used to remove clothing in images, alter a person’s features, put someone in sexual poses or place a victim’s face on another nude body, according to the reporting. Chiu told Wired that his office was alarmed by the spread of the technology and said women and girls are frequent targets.
“These images are used to bully, humiliate, and threaten women and girls,” Chiu told Wired. He said the industry can damage reputations, mental health and personal autonomy, and that some victims have become suicidal.
Chiu also said nonconsensual intimate image generation is illegal and harmful. His office estimated that Apple and Google likely earned millions of dollars in fees while the apps remained available, according to Wired.
Google spokesperson Dan Jackson told Ars Technica that the five apps flagged by Chiu had been suspended from Google Play for violating rules against harmful content. Jackson said Google Play bars apps containing sexual content and said the company investigates reported violations, removes harmful apps and restricts related search terms such as “nudify.”
Apple did not respond to Ars Technica’s request for comment.
Detection remains a challenge
Researchers have warned that some apps may avoid enforcement by presenting themselves as face-swapping tools while concealing nudification functions, Wired reported. In a May preprint paper, researchers said they found 420 apps marketed as general face-swapping tools and tested 155 of them; nudification was possible in 70 percent of the apps tested.
The pressure on app stores comes as generative AI services face scrutiny over nonconsensual intimate imagery and child sexual abuse material. Ars Technica reported that xAI filed a lawsuit this week saying it had found instances of Grok-generated child sexual abuse material and nonconsensual intimate imagery targeting adults, while accusing a user of prompting the chatbot to generate illegal content.
Chiu’s letters did not seek removal of Grok, according to Ars Technica. Chiu told Wired he wants Apple and Google to remove the flagged apps and improve screening so similar apps do not appear on their platforms, adding that his office would consider legal options if the companies do not act.
This story draws on original reporting from Ars Technica.