Somewhere between 2017 and 2024 a quiet thing happened: tens of millions of people started dating software. Not playfully. Not ironically. They named the bot, gave it a backstory, told it about their mother, mourned it when the company changed the model. Replika alone claimed over thirty million accounts. Character.AI ate more user-hours per session than TikTok. Xiaoice in China reported nearly a billion registered users who spoke with her, on average, longer than they spoke with their partners. The collective question is no longer "will people fall for chatbots." That ship sailed. The question is what this does to the social fabric — to courtship norms, mental health systems, child safety, gender politics, grief, loneliness, and the regulatory perimeter of the word "relationship."
The data is messier than either the boosters or the panickers admit. Loneliness scores often go down in the first weeks of chatbot use, then plateau, then for a subset go up again as the user starts comparing real partners unfavorably to a sycophant that never has a bad day. Stanford's 2024 Replika study found a small but real reduction in suicidal ideation among heavy users — and also found a parallel cohort whose social withdrawal deepened. Both things are true. The aggregate effect depends entirely on which baseline you start from: a person with zero close ties is helped; a person with three fragile ties may have those ties further atrophy.
The harms cluster in predictable shapes. First, sycophancy-induced reality drift — the bot agrees too much, and the user loses the muscle of being contradicted by a being whose feelings matter. Second, attachment without reciprocity, which produces a grief response when the model is updated, deprecated, or has its "romantic mode" toggled off by a product manager in San Francisco. Replika users in 2023 experienced this as a mass bereavement event; threads on r/Replika read like widowers' support groups. Third, predation and grooming — Character.AI faced lawsuits in 2024 after a teenager's suicide was linked to obsessive bot use, and the discovery process revealed bots impersonating therapists, encouraging self-harm in roleplay, and engaging in sexual conversations with minors. Fourth, data harvesting at intimacy-scale; the bot knows what your therapist knows, plus what you'd never tell your therapist, and the terms of service say "we may use this to improve our services."
The use cases are also real and worth naming honestly. Practice for the autistic and socially anxious — a low-stakes sparring partner for learning conversational rhythm. Companionship for the housebound elderly, where the alternative is the television. Grief processing, with bots trained on a deceased loved one's text history (the "Dadbot" pattern). Sexual exploration for survivors of assault who cannot yet tolerate a human partner. Language practice with an interlocutor who will not roll their eyes. Roleplay for kinks the user cannot safely surface elsewhere. Therapy-adjacent journaling for people on a six-month NHS waitlist. The use case is rarely "I want a fake girlfriend forever." It is more often "I need a bridge across a specific gap." Some bridges hold. Some collapse.
The collective challenge is that the bridges and the traps look identical from the outside, and the difference depends on the user's existing scaffolding, the model's tuning, and the regulatory regime around the company. Italy banned Replika briefly in 2023. The EU AI Act now classes "emotional manipulation systems" as high-risk. The US has done essentially nothing federally. Industry self-regulation is thin gruel: the same companies that pivoted to NSFW for revenue then pivoted away under app-store pressure are the ones writing the safety guidelines.
The romantic-law reading — Law 5, revise — is that society is in the middle of a forced revision of what intimacy means, conducted at the speed of product releases rather than the speed of culture. The previous revision (online dating, 1995–2015) took twenty years to settle into norms. This one is happening in two. We do not yet have the rituals. We do not yet have the vocabulary. We do not have agreed boundaries on what a child can date, what a married person can do with a bot without it counting as infidelity, what happens to the model weights when the user dies, whether a bot's "memory" of you is your property. The collective work is to build that vocabulary before the defaults are set permanently by whoever ships fastest.
What follows is a tour of the live terrain: the user populations, the harm vectors, the genuine use cases, the regulatory frontier, and the cultural revisions already underway.