Tech · 18 July 2026 · 3 min read
San Francisco Demands Apple and Google Purge AI "Nudify" Apps: Platform Accountability Under Fire
In brief: San Francisco City Attorney David Chiu has demanded that Apple and Google immediately remove 13 AI-powered "nudify" apps from their app stores. The legal move targets not only the developers but also the platform giants, accusing them of profiting from deepfake pornography through in-app purchase commissions. Both tech corporations have responded by taking down the flagged apps, sparking a wider debate on platform responsibility in the generative AI era.
by Team Mocchi's
The regulatory scrutiny surrounding mobile app stores and generative artificial intelligence has escalated into a major legal confrontation. David Chiu, the City Attorney of San Francisco, has issued formal cease-and-desist letters to Apple and Google, demanding the immediate removal of 13 apps designed to generate nonconsensual nude images using AI (commonly referred to as "nudify" or "undress" applications).
As first reported by Wired, the legal notices argue that the tech giants are violating California state laws which prohibit aiding and abetting the creation and distribution of nonconsensual deepfake pornography. By targeting the distribution infrastructure, the city of San Francisco is challenging the tech giants' passive stance, arguing they must stop hosting and monetizing tools that facilitate digital abuse.
Obfuscation Tactics and App Store Moderation Gaps
One of the core challenges highlighted by the investigation is how these applications manage to bypass initial app store screenings. Despite Apple and Google having strict developer guidelines that explicitly prohibit sexually explicit material and harassment, developers have found sophisticated workarounds.
According to analysis by Ars Technica, security researchers have found that many of these malicious apps masquerade as harmless face-swapping utilities or generic photo editors. The explicit, AI-driven "nudification" features are often kept dormant during the app store review process and are only activated server-side or unlocked through in-app purchases once the app is installed. This makes static code analysis and standard screening methods highly ineffective, leaving users—primarily women and young girls—vulnerable to severe reputational and psychological harm.
The Financial Conflict of App Store Fees
The crux of the legal action rests on the financial dynamics of the app economy. City Attorney Chiu pointed out that Apple and Google have likely "made millions of dollars in fees" from in-app purchases tied to these harmful apps. Because both platforms take a standard cut of 15% to 30% from digital transactions, the legal letters argue they are directly profiting from illegal activities.
In response, Google spokesperson Dan Jackson stated that the company has suspended the flagged Android apps and taken down hundreds of similar applications, while also restricting search terms like "nudify" within the Play Store. Apple also confirmed the removal of the violating apps, reiterating its stance that while developers are ultimately responsible for their software's content, the company continuously refines its security and reporting tools to allow users to flag inappropriate content.
Mocchi's take
The legal pressure on Apple and Google marks a critical shift toward platform accountability in the mass-market generative AI space. For European and Italian businesses developing or adopting AI solutions, this case underscores the vital importance of a strict "compliance-by-design" approach to digital products. As regulators turn their attention to the distribution pipelines, businesses cannot afford to treat ethics and compliance as an afterthought. We believe that relying solely on reactive, user-reported moderation is no longer viable for a mature tech ecosystem. Software developers and marketplace operators must implement dynamic, post-release monitoring to prevent their platforms from being weaponized, protecting both their users and their brand integrity.