San Francisco is turning up the legal heat on the two companies that run the world’s dominant app stores. City Attorney David Chiu has sent letters to Apple and Google demanding they purge so-called “nudify” apps — tools that use AI to generate non-consensual intimate images — from the App Store and Google Play, warning that continued hosting could bring civil penalties (TechCrunch). The move escalates a years-long campaign by advocates who argue the platforms have profited from abusive software while promising to keep it off their shelves.
The letters, reviewed by TechCrunch, argue the companies have “been on notice” for their role in “processing payments for illegal purchases for almost a year.” California law criminalizes activity that “knowingly facilitates” or “recklessly aids or abets” the creation of non-consensual deepfake pornography, and a 2025 state law lets victims pursue civil actions against third-party facilitators. Chiu’s office frames the app stores not as passive distributors but as active enablers of a paid economy built on images of real people. “Apple and Google are profiting off apps that exploit women and girls by generating nonconsensual intimate deepfakes,” Chiu said in a statement to TechCrunch.
The pressure did not appear from nowhere. The Tech Transparency Project issued reports in January and April identifying “dozens of apps” inside both stores that “sold deepfake NCII [non-consensual intimate images] in exchange for payments” processed by the firms. Its April report went further, accusing Google and Apple of steering users toward such apps and calling both “key participants in the spread of AI tools that can turn real people into sexualized images.” Chiu told Wired the companies had likely made “millions of dollars in fees” from the apps in question. Those findings now form the backbone of the city’s demand that the platforms act within 28 days or face consequences.
Both companies responded with removal pledges. An Apple spokesperson told TechCrunch that nudify apps are forbidden and that the company “has removed three of the apps in question and are in the process of terminating their developer accounts from our program,” while contacting four others to fix violations or risk removal. Google said all five Play apps named in Chiu’s letter had been suspended, adding that “when violations are reported to us, we investigate and take swift action, which in the case of these apps has included suspending hundreds of violating apps and restricting related search terms like ‘nudify’ on our store.” The statements suggest the apps were already partially scrubbed — which is part of the city’s point: if they were removable, why were they still earning money for months?
The dispute sits at the center of a larger unresolved question about app-store responsibility. Apple and Google market their stores as curated, safe environments, a contrast they pitch against sideloading and open platforms. That curation is also the legal basis on which they collect commissions and control what reaches billions of devices. Critics argue that the same gatekeeping that justifies those fees should make abusive paid apps impossible to find in the first place. The nudify category is especially pointed because the harm is built into the product: unlike a generic image editor, these apps exist to create non-consensual imagery of specific people from a single public photo.
For users, the practical risk is personal and asymmetric. Deepfake pornography has long targeted female celebrities, but “nudify” apps lower the barrier so far that anyone with a social-media photo can be victimized. The technology does not require skill, consent, or even the subject’s knowledge, which is why regulators and lawmakers have treated it as a distinct category of abuse rather than ordinary edited content. Removing the apps from official stores does not eliminate the underlying models — web-based versions persist — but it cuts off the most convenient, payment-integrated distribution channel.
What happens next depends on whether Chiu’s 28-day window produces more than promises. The letters warn of civil penalties under California law, and the city has positioned itself as willing to litigate where other jurisdictions have only issued reports. Apple and Google have historically preferred quiet removal and policy updates to public confrontations with regulators, and both signaled they are already moving. The test will be enforcement at scale: hundreds of suspended apps, in Google’s telling, implies a cat-and-mouse problem where new listings reappear as fast as old ones are taken down.
The episode is a marker of how app-store moderation has become a frontline for AI policy. As generative tools spread, the cheapest and most damaging use cases tend to cluster around non-consensual imagery, and the stores that profit from app sales are increasingly expected to police them proactively. San Francisco’s letter is a reminder that “we ban that content” is no longer a sufficient answer when the banned content has been generating fees on your platform for nearly a year.
