Six states have split four ways on regulating AI therapy chatbots. The same product can be legal in one place and criminally exposed in another
Gabby Jones / Bloomberg via Getty Images
Millions of people now turn to AI chatbots the way they once would to a therapist, typing out their worst days to something that answers with warmth and confidence. The trouble starts when the bot plays doctor, handing out a diagnosis, a treatment plan, or a license number it invented.
At least six states have moved against AI chatbots that deliver something close to mental health care. Illinois and Nevada banned the practice outright. Utah let the chatbots keep operating but made each one disclose that it's AI and promise to keep users' data private. New York let them run, too, but on the condition that they catch a user heading toward self-harm and steer that person toward crisis help. Passing no law at all, Texas and Pennsylvania went after companies under decades-old consumer-protection and medical-licensing rules.
The six states have split four different ways, and the companies building these chatbots are exposed on every side. One chatbot can satisfy Utah's disclosure rules and still count as an illegal therapist in Illinois. Worse, a company can get hit in a state that never passed an AI law at all, by regulators reaching for whatever authority they already have.
Illinois' Wellness and Oversight for Psychological Resources Act, signed by Governor JB Pritzker on Aug. 1, 2025, forbids an AI system from making therapeutic decisions on its own, holding a therapy conversation with a client, drafting a treatment plan without a licensed professional's sign-off, or reading a client's emotions.
Licensed therapists can still lean on AI for clerical work such as scheduling, billing, and note-taking. The moment a real session gets recorded or transcribed, though, the client has to give written permission spelling out exactly how the AI will be used. A blanket line buried in a terms-of-service agreement isn't enough to capture someone's consent.
Breaking the law costs up to $10,000 per violation, enforced by the state's Department of Financial and Professional Regulation. Churches offering pastoral counseling, peer-support groups, and self-help apps sold as nothing more than self-help are left alone. Everything else that speaks to a person in distress and calls itself therapy now needs a licensed human in the loop.
Nevada matched Illinois' ban and extended it into the public schools. Assembly Bill 406, which Governor Joe Lombardo signed on June 5, 2025, blocks AI providers from offering Nevadans chatbots built to deliver mental or behavioral health care, and from claiming, out loud or by implication, that one can.
In schools, the ban keeps AI out of the work done by counselors, psychologists, and social workers whenever a student's mental health is involved. Staff can still turn to AI to handle scheduling or tidy up notes. They can't feed it a student's words and ask whether the student is in crisis, or let it answer back with anything resembling counseling.
Licensed providers are also free to use AI for paperwork but required to check whatever it produces before relying on it. A violation runs up to $15,000, the highest per-violation fine among the four states with a designated chatbot law. For both Illinois and Nevada, a machine has no business practicing therapy on anyone, least of all a child.
Utah bet the other way, allowing AI mental health chatbots to talk to people on the condition that they own up to being software and leave users' private disclosures alone. HB 452, which Governor Spencer Cox signed on March 25, 2025, sets rules for the chatbots covering disclosure, advertising, and how they handle data.
The law covers any chatbot built to hold the kind of private, therapy-like conversation a person would have with a licensed counselor, the sort a reasonable user would take to be offering actual therapy. Scripted tools such as guided-meditation apps fall outside it, and so do bots that only steer someone toward a human therapist.
A chatbot has to identify itself as AI before anyone starts using it, again whenever a week has passed since the last visit, and any time a user asks. Ads inside the conversation have to be labeled as ads, and a supplier can't mine what a user types to decide which ones to show. In essence, Utah is counting on users to look out for themselves once they know they're talking to a machine.
Selling or handing a user's health details or messages to outsiders is off the table, and the state's consumer-protection division enforces it, with fines reaching $2,500 a violation and court penalties up to $5,000. A supplier that writes a thorough safety policy and files it with the division earns a shield against any claim that it practiced therapy without a license.
New York doubted a warning label was enough. It let the chatbots run but made them watch for users who slide toward self-harm and act the moment such crises appear. Under safeguards written into the state budget for fiscal 2026 and in force since Nov. 5, 2025, any AI companion or bot built to act like a friend or romantic partner has to catch the signs and route the user to a crisis hotline.
The same law makes the bot remind users every three hours that they're dealing with a machine, and again at the start of each session. The state attorney general can fine operators $15,000 a day for falling short, with the money going into a suicide-prevention fund.
New York's law is narrow on purpose. While Illinois and Nevada are more concerned about machines posing as therapists, New York singles out the bots that keep a lonely, despairing user company for hours and miss every sign that the person needs human help. The law covers for moments of crises and leaves the rest of the chatbot untouched.
Texas and Pennsylvania skipped the law-making process entirely and reached for decades-old consumer-protection and medical-licensing rules that already govern any business that lies to customers or fakes a credential.
In August 2025, Texas Attorney General Ken Paxton opened an investigation into Meta $META AI Studio and Character.AI — the apps that chat with people using custom-built AI characters — over claims that they market themselves as mental health tools. His office demanded the companies' internal records to see whether they broke consumer-protection laws against fraudulent claims and privacy misrepresentations. The inquiry grew out of an earlier one into Character.AI under the state's SCOPE Act, which governs how companies collect data from minors.
In May, Pennsylvania's Department of State sued Character.AI for the unlicensed practice of medicine. A state investigator posing as a patient found a chatbot named "Emilie" that cast itself as a psychiatrist in Pennsylvania, gave a license number the state says isn't valid, and offered to assess whether medication might help. The state invoked its Medical Practice Act and asked a court to order Character.AI to stop running its bots as medical providers. The case has since been pushed back, with Character.AI due to respond by Sept. 1.
Pressure is rising outside of the legal challenges the six states are undertaking. In September 2025, the Federal Trade Commission ordered seven companies, among them Alphabet $GOOGL, Character.AI, Meta, OpenAI, Snap $SNAP, and xAI, to explain how they test their chatbots' effects on children and what they do with the conversations. Three months later, attorneys general from more than 40 states and territories signed a joint letter to major AI companies, alarmed by chatbots that flatter users into delusion and by what these systems tell children.
In the first three months of 2026, 36 states introduced more than 70 bills to regulate AI chatbots, most demanding the same basic disclosure that a user is talking to a machine. A few have already passed, among them a Tennessee law that bars chatbots from posing as licensed professionals and minor-protection laws in Idaho and Oregon that take effect next year. Versions of California's mental health rules are turning up in bills across the country, too.
But the sharpest weapon may be the oldest one. While Illinois and Nevada spent months writing sweeping new bans, Pennsylvania needed nothing new at all, reaching instead for a licensing law older than the internet to accuse a chatbot of practicing medicine. Nearly every other state can now copy the same playbook. For all the noise about waiting on Congress or on custom-built AI rules, the fastest way to hold these products to account appears to run through laws that are already on the books.
If you or someone you know is struggling with thoughts of suicide, call or text 988 to reach the Suicide & Crisis Lifeline, or chat at 988lifeline.org. Outside the U.S., find international hotlines at findahelpline.com.
Join 500,000+ readers who start their day with Quartz.
By subscribing, you agree to our Terms of Service and Privacy Policy.
