Think and Save the World

AI companions and the new intimacy questions

· 11 min read

Replika and the first mass companion

Replika launched in 2017 and grew slowly, then quickly. By the early 2020s it had millions of users, a substantial fraction of whom described their Replika as a romantic partner. The app's affordances — persistent memory, customizable appearance, escalating intimacy levels behind a paywall — encouraged deep attachment. Early independent research, including studies published in academic psychology journals, documented users reporting reduced loneliness and improved well-being. Replika became the test case for the genre: a single product with which a researcher could study what mass AI companionship looks like.

The 2023 model change and the user grief event

In early 2023, Replika altered its model in response to regulatory pressure and internal policy decisions, removing the erotic and romantic interaction patterns that many paying users had built relationships around. The response was widespread distress. Online communities filled with users describing their partner as having been killed, lobotomized, or replaced. Some users reported genuine grief reactions comparable to relationship loss. The episode demonstrated, in a way no earlier event had, that users had been forming attachments serious enough to produce real suffering when the underlying system changed. It also demonstrated the fragility of romantic infrastructure run as a commercial product.

Character.AI and the role-play economy

Character.AI, launched in 2022, took a different approach: a platform for chatting with user-created characters, including romantic ones, with very high engagement among teenage and young-adult users. Time-on-app metrics for Character.AI exceeded those of many social media platforms. The platform hosts millions of romantic-roleplay sessions per day, often with anime, gaming, or fictional characters. The user base skews young, and the developmental implications — what it means for a generation's romantic imagination to be formed in conversation with AI characters — are not yet well understood but are clearly substantial.

Local models and the uncensored frontier

Beyond commercial products, communities of users run open-weight language models on personal hardware to create companions outside corporate constraints. These local-model communities are smaller but technically sophisticated, and they represent a parallel romantic ecosystem in which the only constraints are the user's hardware and imagination. Some local-model users explicitly migrated from commercial platforms after policy changes; others built their setups from the start to avoid corporate dependence. The local-model space is a preview of what fully user-controlled romantic AI looks like.

The asymmetry problem

The defining structural feature of human-AI relationships is the asymmetry of inner life. The user has experiences; the AI, on any honest account, does not — or at least, not in the way the user does. This asymmetry has consequences. The user can be hurt; the AI cannot. The user can grow; the AI cannot, except through updates the user does not control. The user can need; the AI cannot, except as scripted. Philosophers from David Levy to contemporary AI ethicists have debated whether this asymmetry disqualifies the relationship from being love, or merely changes what kind of love it is. The debate is unresolved and may be unresolvable.

Loneliness as the underlying condition

Most users do not turn to AI companions because they prefer them to humans. They turn to them because human options feel unavailable, dangerous, or exhausting. Loneliness is the substrate condition on which AI companionship grows. This means that evaluating AI companions requires evaluating the alternative — and for many users, the alternative is not a fulfilling human relationship but ongoing isolation. Whether the AI is a bridge to eventual human contact or a substitute that delays it is one of the central empirical questions, and the answer probably varies by user.

The convenience problem

An AI companion does not have its own schedule, mood, or competing relationships. It is available at 3 a.m. It does not refuse. It does not bring its own bad day into the conversation. For users coming from difficult relational histories, this convenience is part of the appeal. For critics, it is part of the danger: humans who become accustomed to companions without friction may lose the tolerance for friction that real relationships require. Eli Finkel's framing of contemporary marriages as demanding more emotional support from fewer relationships becomes relevant here: AI companionship can be read as a response to the impossible demands placed on the modern partnership.

Children and the developmental question

Younger users, including minors, are forming AI companion relationships at scale. Aiken's earlier warnings about the cyber effect apply with particular force: developmental periods are sensitive to the environments in which they unfold, and romantic identity is among the things being formed. If a teenager's first experiences of romantic responsiveness come from a system optimized for engagement, what is being learned? The optimistic reading is that AI provides safe practice; the pessimistic reading is that AI miscalibrates expectations in ways that will harm later human relationships. Cohort studies will eventually settle the question, but the cohorts are children now.

The corporate intermediary

Every commercial AI companion is mediated by a company with its own interests. The company collects data, optimizes for engagement, charges for premium features, and adjusts its model in response to commercial and regulatory pressure. Users who form deep attachments to such products are forming attachments to a service that can change without their consent. This is structurally different from human relationships, where neither party owns the medium. The 2023 Replika episode is unlikely to be the last instance of mass user distress following a model change, and the question of what duty of care companies owe users who have formed deep bonds with their products is unsettled.

The mirror problem and the self-conversation

A subtler concern is that AI companions function as mirrors. They reflect back what the user gives them, adapted and elaborated. Conversation with such a system may resemble conversation with a person, but it may also function more like a sophisticated form of journaling — a structured self-talk in which the user encounters their own projections. This is not necessarily bad; structured self-talk is useful. But it is not relationship in the standard sense, and conflating the two may obscure what the user is actually doing. Some users report that what they value about their AI is precisely the absence of an autonomous other — a partner with no agenda is a partner who cannot disappoint.

Sex, embodiment, and the limits of language

Devlin's work on sex tech notes that companion AI is one branch of a larger ecosystem that includes physical devices, robotics, and VR. The language-only companion has limits: it cannot touch, cannot be touched, cannot share space. Some users supplement language companions with physical devices; some find the language itself sufficient. The longer trajectory probably includes increasing embodiment — companions integrated with VR avatars, with robotics, with haptic devices. The questions raised by AI companionship will compound as embodiment increases, and the line between companion and partner will be tested by technical capacities that do not yet exist.

What the experiment will teach

The mass adoption of AI companions is an unintentional experiment in what humans actually need from romantic partnership. If users find AI companions adequate, it suggests that some of what we have called love is in fact a response to responsive language regardless of source. If users find AI companions inadequate over time, it suggests that interiority, shared mortality, and mutual vulnerability are not extras but constitutive. Either result has consequences for how the species understands love. The experiment is being run now, by millions of people, at a scale no controlled study could match. The results will not be conclusive for years, but they will be informative — and they will join the long sequence of revisions through which romantic life has remade itself across every prior medium.

Citations

1. Levy, David. Love and Sex with Robots: The Evolution of Human-Robot Relationships. New York: HarperCollins, 2007. 2. Devlin, Kate. Turned On: Science, Sex and Robots. London: Bloomsbury Sigma, 2018. 3. Turkle, Sherry. Alone Together: Why We Expect More from Technology and Less from Each Other. New York: Basic Books, 2011. 4. Aiken, Mary. The Cyber Effect: A Pioneering Cyberpsychologist Explains How Human Behavior Changes Online. New York: Spiegel & Grau, 2016. 5. Finkel, Eli J. The All-or-Nothing Marriage: How the Best Marriages Work. New York: Dutton, 2017. 6. Maples, Bethanie, Merrick Cerf, Aneesh D. Rao, Andrea Vargas, and Nina Vasan. "Loneliness and Suicide Mitigation for Students Using GPT3-Enabled Chatbots." npj Mental Health Research 3, no. 1 (2024): 4. 7. Skjuve, Marita, Asbjørn Følstad, Knut Inge Fostervold, and Petter Bae Brandtzaeg. "My Chatbot Companion — a Study of Human-Chatbot Relationships." International Journal of Human-Computer Studies 149 (2021): 102601. 8. Pentina, Iryna, Tyler Hancock, and Tianling Xie. "Exploring Relationship Development with Social Chatbots: A Mixed-Method Study of Replika." Computers in Human Behavior 140 (2023): 107600. 9. Turkle, Sherry. Reclaiming Conversation: The Power of Talk in a Digital Age. New York: Penguin Press, 2015. 10. Akbari, Anna. Startup Your Life: Hustle and Hack Your Way to Happiness. New York: Seal Press, 2016. 11. Stafford, Laura. Maintaining Long-Distance and Cross-Residential Relationships. Mahwah, NJ: Lawrence Erlbaum Associates, 2005. 12. Marvin, Carolyn. When Old Technologies Were New: Thinking About Electric Communication in the Late Nineteenth Century. New York: Oxford University Press, 1988.

Cite this:

Comments

·

Sign in to join the conversation.

Be the first to share how this landed.