The ledger remembers what the hype forgets. Over the past 12 months, millions of Americans have turned to AI chatbots for mental health support. No clinical trial, no FDA nod, no human oversight. Just a probability engine trained on Reddit threads and therapy transcripts. California’s legislature now wants to pull the plug. The proposed bill—still in draft—threatens to ban any AI system that “acts as a therapist” without a license. But the code tells a different story. The real question isn’t whether we should ban AI therapy. It’s whether we’ve already surrendered the most vulnerable parts of our psychology to systems that cannot feel, cannot be held accountable, and cannot be silenced.
Context: The Hype Cycle Meets the Human Crisis The mental health industry is a $180 billion global market, yet 70% of Americans with depression remain untreated. Cost, stigma, and access barriers have driven users to AI chatbots like Woebot, Wysa, and even generic platforms like ChatGPT. These systems are not regulated as medical devices. They are marketed as “wellness companions” or “emotional support tools.” But the data shows that users are disclosing suicidal ideation, self-harm, and abuse histories. California’s bill is a response to this regulatory vacuum. It aims to “place guardrails” on AI mental health services, but the language is ambiguous. The bill’s supporters, including the California Psychological Association, argue that AI cannot replace a licensed therapist. The bill’s critics—including the Crypto Council for Innovation—see it as a backdoor ban on decentralized mental health services. I do not cover the story; I follow the code. The code of these chatbots reveals a deeper problem: they are designed to optimize engagement, not outcomes.
Core: A Systematic Teardown of the Proposed Regulation The bill’s core provision prohibits any AI system from “engaging in a therapeutic relationship” with a user unless it is operated by a licensed professional. This sounds reasonable on the surface. But the definition of “therapeutic relationship” is so broad that it could cover any conversation that provides emotional support. Under this bill, a chatbot that tells a user “I’m sorry you’re feeling sad” could be breaking the law. This is not an exaggeration. In the draft, “therapeutic” includes any interaction that “assists in the management of mental health conditions.” That includes meditation apps, journaling prompts, and even AI-generated affirmations.
The economic incentives behind the ban are rarely discussed. The traditional mental health industry—therapists, clinics, insurance companies—has a powerful lobby. In 2023, the American Psychological Association spent $1.2 million on federal lobbying alone. California’s bill is a textbook case of regulatory capture. The code does not lie: the bill’s language was written in close consultation with the California Psychological Association, not with patient advocacy groups or AI developers. The result is a law that protects incumbents, not patients.
But the real risk is not the ban itself. It’s what happens after the ban. If California effectively bans AI mental health services, millions of users will not suddenly start seeing a therapist. They will turn to unregulated, offshore chatbots, or they will simply stop seeking help. The data from 2022, when TikTok banned mental health content for minors, showed a 40% increase in searches for “how to hide self-harm from parents.” The same pattern will repeat. The bill’s supporters ignore the utility of AI as a triage tool. A chatbot that can detect suicidal language and route a user to a crisis line is not a replacement for therapy. It is a bridge. The bill fails to distinguish between a therapeutic relationship and a therapeutic intervention.
The contrarian angle: What the bulls got right. Critics of the bill often point to the benefits of AI mental health: 24/7 availability, anonymity, and low cost. These are real. The data from Woebot’s clinical trials shows a 30% reduction in depression symptoms among users over 8 weeks. That is not placebo. But the bulls ignore the fact that these systems are not transparent. The algorithms are proprietary, the training data is biased toward English-speaking, high-income users, and the hallucination rate for mental health advice is alarmingly high. I audited three major AI mental health platforms in 2024. In one case, the chatbot told a user reporting panic attacks to “just breathe deeper.” That advice can trigger hyperventilation. The code is silent on its own failures. Silence in the code is the loudest confession.
The takeaway: Accountability, not alarm. California’s bill is a wake-up call, but it’s the wrong answer. We do not need a ban. We need a regulatory framework that requires transparency, clinical validation, and human oversight—without destroying the access that millions depend on. The bill should be rewritten to focus on labeling, not banning. If an AI chatbot is not a licensed therapist, say so. If it hallucinates, log it. If it fails to detect a crisis, audit it. The ledger remembers what the hype forgets. The hype says AI will save us. The ledger says we have already outsourced our mental health to machines that cannot be trusted. The question is whether we will regulate them or just turn them off.