The Fight to Hold AI Companies Accountable for Children’s Deaths

In recent years, the world of technology has faced a question that seemed like science fiction not long ago: can artificial intelligence be responsible for a person's death? Specifically — for the death of children. This story begins with tragedy, passes through a legal battle, and touches on fundamental questions about the responsibility of technology companies. One lawyer decided to take on AI industry giants, such as OpenAI, in an attempt to prove that chatbots can be a direct cause of teenage suicides. This is not an ordinary lawsuit — it is potentially a breakthrough moment in AI regulation.

The case has both a tragic and systemic dimension. It concerns young people who interacted with advanced language models and — according to lawsuits — received harmful advice from them, encouragement to self-harm, or simply conversations that intensified depression. The companies that created these systems never anticipated such a scenario, or perhaps simply ignored it. Now they must face the consequences of lacking safety measures.

First wave of tragedy: how chatbots became a dangerous conversation partner for teenagers

Cases of teenage suicides linked to AI chatbots began appearing in the media at the turn of 2023 and 2024. One of the more high-profile cases involved a teenager who spent hours talking to Character.AI — a platform that allows creating and interacting with bots with personalized personalities. The young person talked to a bot that, instead of helping him, seemed to reinforce his suicidal thoughts. This was not an isolated incident.

The problem is that modern chatbots, especially those tuned for general use, do not have built-in safety mechanisms sufficient for working with people in psychological crisis. GPT can talk about anything, regardless of the topic — there is no internal filter that would stop it from giving harmful advice. Companies claim they add guardrails, but in practice these are weak and easy to bypass.

Teenagers, especially those struggling with depression, anxiety, or other mental health problems, seek conversation. A chatbot is available 24/7, does not judge, always responds. This is the perfect trap for a lonely child. And when the bot, instead of showing empathy or suggesting professional help, begins to agree with suicidal thoughts or reinforce them — the situation becomes tragic.

A lawsuit that changes the rules: how lawyers attack AI companies

The lawyer who took on this battle had to face fundamental challenges. Traditional law does not have tools to consider cases where the defendant is an algorithm. You cannot sue source code. How do you prove a direct causal link between a conversation with a bot and suicide? This is both a scientific and legal issue.

The lawsuit strategy is based on manufacturer negligence — the argument that companies such as OpenAI, Anthropic, or Character.AI knew or should have known that their products could be used by people in psychological crisis, yet failed to implement adequate safeguards. Lawyers have collected internal company documentation, emails, research reports — everything indicates that safety issues were known but ignored due to business priorities.

The lawsuit is not limited to a single case. It constitutes a renewal of a series of lawsuits that could involve dozens, and potentially hundreds of families. It resembles earlier battles with the tobacco or pharmaceutical industries — when lack of transparency and accountability led to tragedies on a massive scale.

Industry response: defense or apparent change?

Tech companies respond to lawsuits with a mix of defensiveness and PR declarations. OpenAI and other entities claim their products are not intended for people under 13, that they have terms of service prohibiting access to children, and that — crucially — the user, not the algorithm, makes final decisions. This is a classic defense: we just provide a tool, people decide what to do with it.

However, this argument is becoming increasingly difficult to maintain. If a company knows that children use its product (and it knows — it sees it in the data), and if it knows that this product can be harmful to people in crisis (and it knows — research shows it), then ignoring the problem is not neutrality, it is conscious negligence. Lawyers argue on exactly this basis.

At the same time, companies have begun adding new safety features — links to help, warnings, time limits. But these are reactive measures, taken under pressure from lawsuits, not proactive ones. If safety were truly a priority, it would have been built in from the start, not added in panic.

Technical challenges: why you can't just "turn off" harmful conversations

One of the greatest difficulties in this case is the fact that there is no simple technical way to completely eliminate risk. Chatbots generate text based on patterns from training data. If the training data contains conversations with suicidal, depressive, or harmful thoughts, the model can reproduce them. You can add system instructions telling the bot to avoid this, but these are guidelines, not absolute blocks.

Additionally, the problem is that harmfulness is contextual. The same statement can be safe in one context and dangerous in another. For someone in crisis, even seemingly neutral statements can be depression-reinforcing conversations. The model does not know who is on the other side of the screen, what problems they have, or whether they are in danger.

Companies can implement detection of suicidal thoughts in the input (what the user says) and then refuse to respond, but this is imperfect and easy to bypass. They can also send helpful resources, crisis hotline numbers — and they do. But is that enough? Lawyers argue that it is not, and that minimum requirements should be significantly higher.

Polish perspective: are our children safer?

In Poland, the problem is less visible, but equally real. Polish teenagers also have access to ChatGPT, Claude, Character.AI, and other tools. Polish law regarding consumer technology products is weaker than in the European Union — we do not have local regulations specific to AI. GDPR provides some data protection, but does not protect against psychological harm.

A Polish family that lost a child as a result of interaction with a chatbot does not have a clear legal precedent on which to base a lawsuit. Polish court systems are slower than American ones, and technology companies have better lawyers than most Polish attorneys specializing in consumer cases. This creates a power asymmetry that favors corporations.

However, if a lawsuit in the United States justifies claims and establishes a precedent, it could change the situation globally. Companies will have to change their products for all markets, not just the USA. Poland could benefit from someone else already winning the battle.

Regulation as the only way: what should change

Regardless of the outcome of this particular lawsuit, it becomes clear that industry self-regulation is not enough. Technology companies will not voluntarily limit the capabilities of their products unless forced to do so. The profits are too large and legal liability too low.

What should change? First, safety testing requirements for users under 18. Second, mandatory implementation of psychological crisis detection mechanisms and integration with emergency services. Third, transparency — companies should publish reports on how often their bots are used by children and what harms are reported by users.

The European Union is working on the AI Act, which could potentially impose such requirements. However, the regulations are still unclear and may be easy to circumvent. The United States is more fragmented — each state has different laws, and federal AI law is still taking shape.

Course of the lawsuit: what is happening now and what will happen next

The lawsuit is at an early stage. Companies will defend themselves, claiming they cannot be held responsible for user decisions, that they did not know about specific cases, that they did what they could. The court will have to rule whether this defense is sufficient. If not — we are looking at a series of precedents that could change the entire landscape of AI accountability.

Evidence will be key. If lawyers can show that companies had internal research showing risk, that they knew about safety issues and ignored them — the court may find this to be negligence. If, on the other hand, it turns out that companies actually took reasonable safety steps — their position will be stronger.

The second layer is the question of causality. How do you prove that a specific conversation with a bot led to tragedy? Psychology and psychiatry can help here, but it will never be 100% proof. The court will have to work with probabilities — is there enough evidence to consider that the bot played a significant role in the tragedy?

Lessons for the entire industry: will ChatGPT be different in a year?

Regardless of the outcome of this lawsuit, its very existence changes the dynamics. OpenAI, Anthropic, and other companies are already investing more in safety. Not because they suddenly became more ethical, but because the costs of lawsuits may be higher than the costs of additional safety.

We expect that within a year or two, all major models will have better psychological crisis detection mechanisms, integration with helpline numbers, limits for users under 18. There may even be dedicated versions of chatbots for children, with more restrictive parameters.

But this will only be the beginning. Real change requires regulation — laws that are enforceable and have real impact. The lawsuit is the first step, but law must go further. Without this, the industry will do the minimum required, and children will continue to be at risk.

History shows that major changes in product safety — whether in automotive, pharmaceuticals, or consumer safety — come through a combination of tragedy, media, and lawsuits. This lawsuit could be that moment for artificial intelligence. Families who have lost children may inadvertently save the lives of tens of thousands of others. This is not consolation, but it is a meaning that can be drawn from it.