RESOLVING THE LEDGER
This is one outlet's own report from Wired — the article as it was filed. Other outlets are covering the same event; open the full story to compare every source side by side.
See the full story · 1 sources
The tool, which is called Kromix, bills itself as an “AI image styler.” A voice-over for one of the video advertisements, viewed by WIRED, promotes the tool as having “no restrictions,” saying “all the characters are real people” and describing it as “the AI that men actually use.” In one ad, a woman closely resembling a prominent female US politician is seen in front of a US flag and the flag of the president of the US above the caption “What if she moved?” She then winks, before the ad cuts to a scene of a woman with the same face performing in a pornographic video.
The Kromix app, which was hosted by Apple’s App Store prior to a request for comment from WIRED, invites paid users to upload photos so that it can use them in scenarios including “bedroom rape” and “Disney love.” Videos on the app also feature AI-generated pornographic videos of women wearing Spider-Man costumes. Requests for comment sent to a contact listed in the app went unanswered.
Apple has struggled to keep similar tools, which violate its policies against nonconsensual intimate imagery, nudifying, and sexually explicit content, off the App Store. San Francisco’s city attorney sent the company a cease-and-desist letter earlier this summer demanding the removal of 13 specific apps and that it stop “aiding and abetting” the sale of explicit deepfake images.
The ads were, according to Meta’s ad library, exclusively targeted to male users with the tagline “Unleash Next-gen AI Magic” and were removed after WIRED asked about them. In all, the account tied to the campaign ran 32 ads, according to Meta, which ran for between five and 46 hours. Most received 10 or fewer impressions.
Katie Paul, director of the Tech Transparency Project, which discovered the ads, says, “All of the ads that we've seen so far broadly on the nudify issue are exclusively sexualizing and nudifying and putting women into AI-generated porn. And this raises a lot of questions about how Meta is going to deal with this kind of content, not just broadly but when it comes to female politicians, when the company is actively running ads containing this content on their platform.”
Politicians and elected officials, overwhelmingly women, have been a target of nonconsensual deepfake images and videos for years. In January, as Grok was used to create around 3 million sexualized images of women and girls, officials in Sweden, England, and Northern Ireland were targeted by people using the technology, often using official images as a starting point.
“As the production and distribution of AI-generated content have democratized, formerly benign media—official portraits, video from a press conference—have become raw material for low- and no-code tools that anyone with an internet connection can use,” says Benjamin Shultz, the lead researcher at the American Sunlight Project, who is investigating how the ecosystem of harmful deepfake technology is used to target politicians.
According to the company’s public documentation, before an ad is published on Meta’s platforms, which includes Instagram and Facebook, each ad is “reviewed” against its policies that detail what advertisers can and can’t post. “Our ad review system relies primarily on automated tools to check ads and business assets against our policies,” the company’s advertising standards say, pointing out that the review starts before ads begin running. The page running the advertisements was created on August 3, not long before the ads began running, and had no followers—two things that might have indicated to Meta that the account was suspicious.
The company specifically has a policy that says ads cannot contain sexual imagery—including nudity, sexual activity, or sexually suggestive content. Meta’s broader policies on adult sexual exploitation say that nonconsensual intimate imagery is not permitted and the promotion of “nudify” or “AI kissing” apps are not allowed.
However, ads for nudify or undressing apps have been appearing on Meta’s platforms frequently over the past few years, with researchers finding thousands the company had failed to detect—by the company’s own count, it also removed 344,000 ads on Facebook and Instagram that were promoting the services between November last year and January and has filed a lawsuit against at least one firm promoting nudify services.
"Sexual exploitation, including promoting nudify apps, is horrific, and we work aggressively to fight it,” says Cindy Southworth, head of women’s safety at Meta. “This is a highly adversarial space where bad actors constantly evolve their tactics, which is why we're continually strengthening our defenses: removing over 340,000 violating ads for nudify apps late last year alone, launching new AI technology to better detect and block new violating ads, and taking legal action against nudify app developers.” Southworth says that Meta shares the information of apps and developers that it removes from its platform with other tech companies as well.
Most of the video ads had a cover photo of something like a beach, mountains, or a waterfall, disguising the fact that the rest of the content was pornographic. “This is an example of an adversarial tactic these advertisers use to try and evade detection. We’re aware of this tactic and block and remove many violating ads despite them trying to obscure violating content with a benign image,” says Southworth. “That said, no system is perfect, and we’re continuing to investigate why these ads were able to bypass our systems and how we can improve.”
An Apple spokesperson told WIRED that the company has removed Kromix—which, they said, appeared to have added prohibited content and features after initial review—from the App Store. “We have always strictly prohibited apps designed to generate, distribute, or consume pornography,” they said. “‘Nudification’ apps are against our App Review Guidelines and we aggressively work to identify and remove apps—as well as the developer accounts behi...
AIPROPX is an independent multi-source news index — we track, compare, and connect coverage from across the web into one place you won't find anywhere else.