As AI-generated sexual deepfakes spread at massive scale, Minnesota is attempting an aggressive legal response
Lucy North/PA Images via Getty Images
A version of this article originally appeared in Quartz’s AI & Tech newsletter. Sign up here to get the latest AI & tech news, analysis and insights straight to your inbox.
Most people, when they think about AI deepfakes, think about porn. They are not wrong. What they probably do not know is the scale.
In 2023, one survey found that deepfake pornography made up 98% of all deepfake videos online. Ninety-nine percent of those depicted women. The technology that the AI industry spent years worrying might destabilize democracy has been deployed, overwhelmingly, to sexualize real women without their knowledge.
Join 500,000+ readers who start their day with Quartz.
By subscribing, you agree to our Terms of Service and Privacy Policy.
Last week, Minnesota decided to do something about it.
The state Senate voted 65-0 to ban nudification apps, the software that makes nonconsensual explicit imagery possible at scale. Not the images or the sharing of them. The apps themselves.
Congress got here first, technically. The Take It Down Act, signed by President Donald Trump last May, made it a federal crime to distribute nonconsensual intimate imagery and requires platforms to remove it within 48 hours. What it does not do is give survivors the right to sue, or touch the apps doing the creating. Minnesota's bill does both.
If Walz signs it, platform owners can be sued for damages and fined $500,000 per violation. No other state has gone this far.
Making these images can be incredibly simple. Upload a photograph of a real woman, click a button, and receive a fake nude image in seconds. The tools to do it are everywhere, and the market around them is enormous.
In January, the Tech Transparency Project found 47 nudification apps in the Apple $AAPL App Store and 55 in the Google $GOOGL Play Store. Together they had been downloaded more than 705 million times and generated $117 million in revenue. Apple and Google took a cut of all of it.
Both companies removed some flagged apps after the report came out. Neither eliminated the category.
Meta $META has repeatedly allowed ads for nudification services to run on Facebook and Instagram despite officially prohibiting them. The apps have even turned up on government websites, uploaded through public portals by spammers exploiting the search authority of official domains to drive traffic to this lucrative market.
Then came December, when Grok, the AI chatbot built into X $TWTR, began producing sexualized images of women and children at user request.
Researchers estimated more than 1.8 million such images were generated over nine days. xAI markets Grok as willing to answer the questions other AI systems refuse. Lawsuits are now testing what liability looks like for a company that made that a selling point. Malaysia and Indonesia restricted the platform. The U.K. and E.U. opened formal investigations.
This is not a new problem. In her 2025 book "The New Age of Sexism," author Laura Bates traces the pattern back through every major technological shift. When cameras became portable, women were warned about upskirt photos. When film gave way to darkrooms, the question was where the negatives went. Photoshop made believable image manipulation available to anyone with a desktop and some patience.
Nudification apps removed the patience requirement. While celebrities and famous women bear the brunt of this, regular women are also targeted, often by people they know. A 2024 survey found that 1 in 10 teenagers personally knew someone targeted by AI-generated nude imagery. One in 17 had been targeted themselves.
More than 80% of those surveyed said they understood the images cause real harm. A March UN Women report found that more than half of deepfake victims in the United States had contemplated suicide. Forty-one percent of women in public life who experienced digital abuse also faced offline harassment or physical attacks tied to it.
The E.U. finalized a deal this month to ban nudification apps outright, with enforcement expected by December. The U.S. has not gone that far but is considering more legislation. The Senate passed the DEFIANCE Act in January giving survivors the right to sue, and the House has not moved on it.
Minnesota's law, if it holds, goes further than any of them in one specific way. It targets creation rather than distribution, which is where most existing law stops. Whether it survives a court challenge, or gets preempted by the Trump administration's push to override state AI regulation, is genuinely uncertain.
For everyone outside Minnesota, the situation looks a lot like it did when cameras first became portable. Be careful. Share less. Assume the worst about who might be watching. The tools have gotten faster and more sophisticated. The advice has not.
