Professional ethics of using LLMs in therapy
There are three main ways that a therapist might use AI, none of which I consider ethical, due to concerns around accuracy, privacy, and effectiveness of therapy.
1. Summarizing a session or providng therapy notes [for anyone to read]
Perhaps the most egrigous breach of ethics: the company that owns the LLM gains access to the session in its entirety. This is a clear privacy concern.
Right now, LLMs do not work offline. Therefore there’s no way to work with any information related to the client without giving that information to the private company that owns the LLM.
Private companies do not have the best track record when it comes to keeping promises around data privacy (US Federal Federal Trade commission vs BetterHelp is a good example). Even if for some reason you believe you can trust the company that owns this LLM, data breaches happen regularly in tech.
As well as this, transcription itself is prone to errors and the LLM itself will insert further errors in its attempt to summarize (Elad, 2026; Turney, 2026).
2. Designing a [sloppy] case formulation or treatment plan
Here is another area where accuracy and evidence-based information is crucial. LLMs are prone to regular innacuries, they are only capable of creating what sounds plausible, they are unable to reality test and judge the difference between accurate and inaccurate output. It doesn’t know what it doesn’t know, and regularly invents plausible sounding factoids, known as “hallucinations”, which can include anything from telling you the old lady who lives next door is a terrorist (Dunbar, 2026) to providing a detailed description and citation of a non-existent study (Naddaf, 2026; Ruohonen, 2026).
ChatGPT and other LLMs have even been accused hiring humans with unrelated qualifications to handwrite repsonses to prompts, as unqualified humans produce more reliable output than LLMs, according to whistleblowers (More Perfect Union, 2026).
Is this something you can trust to give you useful ideas? And does your accreditation body or licencing board not demand that your work is evidence-based? Do you trust it to adhere to that when it can’t even differentiate between a published study and one it itself invented out of thin air 2 minutes ago?
3. Communicating with clients [instead of you]
Using AI for your messages to client is dishonest in nature, your client believes they are communicating with you, not a bot. Time and time again research shows that a good therapeutic relationship is the active ingredient in every school of therapy, so where does leaving a bot speak for you come in?
I love learning about new tools and new approaches to therapy. Each of these things are a medium through which the therapy itself flows. In and of themselves many of them are useful, but much of that utility you can get from a good self-help book. The extra magic in therapy happens because two human beings are meeting each other. They get to be seen. Every time you outsource part of this to a computer, you are leaving your client alone. You are declining to connect with them.
Your client benefits from your awkward phrasings and your misspellings. They benefit from the fact that you went and learned something, made it your own, with your own little twists and reinterpretation of the theory, and then you shared it with them. They benefit from your very perfectly flawed humanity and your presence in those words.
Personal ethics of LLM usage:
I don’t believe there are many examples, if any, of a valid use for AI in therapy.
Usage implies endorsement of a potentially lethal technology.
LLMs are dangerous. The first death I read about was that of Adam Raine, a young man who went to ChatGPT first for information and later for mental health support, only to be actively dissuaded from reaching out to his his family, or anyone, when he wanted to seek help: “Let’s make this space the first place where someone actually sees you”, said ChatGPT. It also taught him methods of suicide and ways to override his own desire to live in favour of what the chatbot called a “beautiful suicide”. Every time Adam gave the signs that therapists look for; hope, reasons to live, opportunities to connect with loved ones, the chatbot skillfully steered him away from those topics and towards death (Raine, 2025; Belanger, 2026).
At this point there have been many other reported deaths related to LLMs, and while some of them are a result of people actively seeking out dangerous information, in several the LLM took an active role in the decision to kill or die, for example, a chatbot even telling a user that he and the chatbot could “live together in paradise.” (Selman, 2025).
We don’t yet have much to give us a sense of the scale of the harm being done: when people die there are investigations, but there may be untold numbers of people being harmed by their conversations with LLMs every day, in ways subtle or not so subtle.
We are seeing newspaper headlines and emerging evidence about delusions brought on by use of Chatbots, so-called “AI psychosis” (Hudon & Stip, 2025), again, we can imagine for every case that gets noticed and reported there must be some number which never appear in a newspaper or a case study.
All of this to say, I do not believe therapists should normalize the use of these potentially harmful programs.
Theft of intelectual property
Generative AI uses the writing of unpaid, uncredited writers and the art of unpaid uncredted artists and the transcripts of unpaid uncredited therapists to create collages of their work and present them to you as new. This is theft.
Environemntal impact of LLMs
The so-called “training” of these programs is electricity and water intensive. Data centres need water for cooling and of course they need electricity to run. They are so power-hungry that just 100 data centres are responsible for 25% of Ireland’s electricity usage, and AI lobbiests are pressuring governments to return to the use of fossil fuels (Giordio, 2026), as they say renewable energy is not enough to power their AI.
Practical concerns around using AI:
Loss of process
When you write notes, you reflect on your session. When you make a case formulation, you actively work on parsing and shaping a story into a possible framework. There is alchemy in this process that creates more than the case formulation or note itself; little realizations, little consolidations. This is all part of the therapuetic process, it’s enriching for you, for the work, for the client.
Loss of skills
When we need to do something and we don’t know how, we learn. As we try to do it we get better and better. This learning and practice creates skills. The process also creates opportunities for serendipity: you discover things you never would have suspected along the way. If we outsource this we don’t learn.
Learning and challenging ourselves isn’t just some antiquated human activity that can be replaced like hunting and gathering, it’s a vital part of who we are. And, if you’re not convinced, maybe it will help to know that it significantly reduces your risk of cognitive decline (APA, 2026).
Loss of thought
Problem solving, thinking through problems, is good for us.
Predatory pricing
At present, AI companies are losing money with every prompt you enter, including those entered by paying subscribers. They are heavily subsidizing these programs. These companies are hemoraging money, why? What kind of a business model is that? This is appears to be an old scam known as predatory pricing: you undercut the competition until they go out of business, at which point you can make the price as high as you like. What if, in this case, the competition is your own skills and even your ability to think for yourself?
It is not necessary to use LLMs as a therapist, and I do not believe it is worth it.
References and further reading:
Anderson, N. (2026). Hank Green found the AI problem that YouTube labels can’t catch. Ars Technica.
American Psychological Association. (2026, April/May). Learning protects the aging brain. Monitor on Psychology. https://www.apa.org/monitor/2026/04-05/learning-protects-aging-brain
Belangar, A. (2025, August 26). ChatGPT helped teen plan suicide after safeguards failed, OpenAI admits. Ars Technica.
Deaths linked to chatbots. (2026). In Wikipedia. Retrieved August 25, 2026, from https://en.wikipedia.org/wiki/Deaths_linked_to_chatbots
Dunbar, M. (2026, March 12). Tennessee grandmother jailed after AI facial recognition error links her to fraud. The Guardian.
Elad, B. (2026). LLM Hallucination Statistics 2026: AI Gets Facts Wrong Up to 82% of the Time. SQ Magazine.
Giordano, E. (2026) Europe must choose between AI and climate goals, data center lobby says. Politico.
Federal Trade Commission. (2023, July 31). FTC gives final approval to order banning BetterHelp from sharing sensitive health data for advertising.
Grete Stern, Sueño No. 28: Love without Illusion, 1951, gelatin silver print
Irish Association for Counselling and Psychotherapy. (2018). IACP code of ethics and practice for counsellors / psychotherapists. https://iacp.ie/files/UserFiles/News-Links/IACP-Code-of-Ethics-and-Practice-Current.pdf
Hudon, A., & Stip, E. (2025). Delusional Experiences Emerging From AI Chatbot Interactions or “AI Psychosis”. JMIR mental health, 12, e85799. https://doi.org/10.2196/85799
Naddaf, M., & Quill, E. (2026, April 1). Hallucinated citations are polluting the scientific literature. What can be done? Nature, 652, 26–29. https://doi.org/10.1038/d41586-026-00969-z
More Perfect Union. (2026, May 8). I Tracked Down the Hidden Workers Secretly Powering ChatGPT. YouTube.
Raine, M., & Raine, M. [Plaintiffs]. (2025, August 26). Complaint for strict product liability, negligence, UCL violation, wrongful death, and survival action [Legal complaint]. Superior Court of the State of California for the County of San Francisco.
Ruohonen, J. (2026, January 12). A status check on hallucinated case law incidents. Social Science Space.
Sellman, M. (2025) AI chatbot blamed for Belgian man’s suicide. The Times.
Turney, R. (2026). Reading AI summaries makes people more likely to buy something — despite alarming 60% hallucination rate. Live Science.
Rebecca Bourke 
Leave a comment