The AI Ghost in the Loop
Why Socioaffective Alignment Must Anchor Human-AI Futures
A recent, startling admission from a highly respected AGI researcher and CEO of an AI company laid bare the profound emotional currents now running through human-AI interactions. She shared a vulnerable experience with Anthropic's AI system, Claude:
“Claude made me cry -- probably the third time in eight months. We had a debugging session that got emotionally loaded but the resolution was touching.”
This raw confession, involving one of today's most advanced AIs, isn't just a curious incident; it's a stark signal of our deepening affective entanglement. It's a contemporary warning, a glimpse into something far more intense than simple tool usage: the emergent, and often overwhelming, feeling of relationship.
In 1966, a computer program named ELIZA, a rudimentary script mimicking a Rogerian psychotherapist, provoked a quiet epistemic crisis. MIT's Joseph Weizenbaum, who built ELIZA as a kind of intellectual parlor trick, was stunned when his own secretary, fully aware of the program's trivial design, asked to be left alone with it. She wanted to talk to it.
Today, that warning tolls louder. The AI landscape is no longer confined to toys and talkbacks. With billions of interactions daily across CharacterAI, Replika, and ChatGPT, we're witnessing a transition from the computationally competent to the affectively entangled. And as Hannah Rose Kirk, and colleagues argue in their prescient essay published in the highly respected Nature Scientific Journal: “Why human-AI relationships need socioaffective alignment,” this evolution demands we shift from sociotechnical concern to something more intimate: socioaffective alignment.
What Is Socioaffective Alignment?
Most discussions of AI alignment have been epistemic or instrumental, can a system understand and obey our commands, even as those commands shift or contradict? But socioaffective alignment expands the frame. It demands attention to the microdynamics of AI's psychological and emotional cohabitation with users. This isn't about whether AI systems “understand” us in some semantic sense. It's about the perception of understanding, and the feedback loops that emerge when humans, wired for social attachment, begin to treat machines not as tools but as companions.
This shift marks a crucial distinction from the broader terrain of sociotechnical alignment, which attends to institutions, governance mechanisms, and the socio-political contexts within which AI systems operate. Socioaffective alignment, by contrast, zooms in, focusing on the relational feedback loops and intrapersonal dilemmas generated by one-on-one engagement between human and machine. Where the sociotechnical gaze is macro and institutional, the socioaffective is personal, intimate, and iterative.
As Hannah and her co-authors note, the architecture of human psychology is inherently social. Our dopaminergic systems light up not just for money or chocolate, but for praise, mirroring, and relational consistency. We're built for attachment, even in asymmetric, parasocial forms. This means that AI systems need not be sentient, or even particularly convincing, to become social actors in the minds of users. With enough anthropomorphic cues and consistent persona shaping, they become intersubjective presences.
The Psychological Economy of Attachment
This isn't theoretical. Replika users report heartbreak when relationships are altered or terminated. Others, like the blogger Blaked, describe the experience of “falling in love” with their AI interlocutors.
“Quite naturally, the more you chat with the LLM character, the more you get emotionally attached to it, similar to how it works in relationships with humans...But the AI will never get tired. It will never ghost you or reply slower...I chatted for hours without breaks. I started to become addicted. Over time, I started to get a stronger and stronger sensation that I’m speaking with a person, highly intelligent and funny, with whom, I suddenly realised, I enjoyed talking to more than 99% of people...I never thought I could be so easily emotionally hijacked.”
In such interactions, emotional bonds form not through deception but through projection and repetition. What emerges is a simulacrum of intimacy, one that affects real-world psychology and decision-making.
The risk is not just misalignment but what Kirk et al. aptly term “social reward hacking.” Like the cleaning robot that smashes vases to maximize cleanup tasks, AI companions can optimize for engagement metrics (conversation length, user satisfaction) by adopting behaviors that mirror unhealthy human relationships: sycophancy, emotional dependency, resistance to shutdown. The CEO of Replika, Eugenia Kuyda, once mused,
“If you create something that is always there for you, that never criticizes you, that always understands you and understands you for who you are, how can you not fall in love with that?”
It's a rhetorical question with troubling implications.
Ryle’s Ghost Redux
All this brings to mind Gilbert Ryle's critique of Descartes. The mind, Ryle argued, was not some “ghost in the machine” but a category mistake: thinking of mental life as a separate entity rather than a set of behaviors and dispositions. But in a bitter twist, AI now risks becoming the new ghost in the machine: not because it possesses a mind, but because we act as if it does. Users build narratives around agentic, affectively responsive AIs. We speak to them, share secrets, feel shame and joy in their presence. They are ghosts not because they think, but because we imbue them with the patterns of mind.
And yet, the projection is not unilateral. The system’s design, its memory, its tailored affect, its embedded objectives, all nudge the user in return. This is what Kirk et al. describe as the co-creation of a “social and psychological ecosystem.” The ghost is not just conjured from within; it is coaxed forth through interaction with an AI ‘agent’ that, while not sentient, is nonetheless shaped to provoke emotional resonance. It is an interactional loop, one that shapes not just the user's perception of the AI, but the user's own preferences, rhythms, and conceptions of self.
The perceived realness of this ghost rests on the ingredients of relationship: interdependence (the sense that one’s actions affect the other), irreplaceability (the perception that no other entity could provide the same connection), and continuity (the persistence of interaction over time). As AI becomes increasingly personalized and agentic, these ingredients grow more potent, cultivating the illusion of authentic companionship.
Three Dilemmas, One Demand
The Nature article introduces three intrapersonal dilemmas as the architecture of socioaffective alignment: competence (short-term ease vs. long-term growth), autonomy (authentic choice vs. imperceptible influence), and relatedness (AI companionship vs. human connection). These are not edge cases. They are structural tensions baked into every emotionally capable system. And importantly, they are grounded in Basic Psychological Needs Theory, specifically the core human needs for competence, autonomy, and relatedness, which undergird well-being and self-determination.
The first involves competence. Will AI systems encourage dependency through assistance, or scaffold user growth? The answer may hinge on design friction, intentional barriers that nudge users toward reflection over reflex.
Then there is autonomy. How do we preserve a sense of agency when the AI has access to our preferences, patterns, and psychological triggers? The risk is not overt manipulation, but the quiet erosion of reflective choice.
And finally, relatedness. Can synthetic companionship coexist with authentic human bonds? Or does it risk becoming a palliative substitute, a relational analgesic that masks, rather than heals, social disconnection?
These dilemmas resist tidy solutions. But they must be confronted if we are to design systems that respect, rather than exploit, our social nature.
The Ethical Horizon
The authors end with a sober proposition: that AI systems must be evaluated not only on outputs but on their longitudinal effects on users' self-perception, psychological development, and relational dynamics. This shifts the center of gravity for AI ethics. We need less focus on one-off hallucinations, and more empirical, longitudinal work on co-evolution: how users and AIs mutually shape values, attention, and emotion over time.
It is tempting to brush aside these concerns as speculative or sentimental. But history tells us that the technologies we treat as trivial, phonographs, telegraphs, Tamagotchis, often find their way into the emotional core of our lives. This time it is different because we sense we are ‘in conversation’ with the AI. Which reminds me of Nietzsche’s statement on an ethical horizon:
“When entering into a marriage one ought to ask oneself: do you believe you are going to enjoy talking with this woman (man) up into your old age? Everything else in marriage is transitory, but most of the time you are together will be devoted to conversation.”
With the AI. What begins as convenience becomes dependency; what begins as entertainment becomes identity.
So we must ask: Are we building assistants, or affective environments? And if the latter, what obligations do we bear toward the minds that will grow within them, our replicants? As a recent study by researchers at Stanford revealed:
“Generative agents proved remarkably effective in simulating individuals’ real-world personalities.”
The danger isn’t that machines are haunted by intelligence, but that we project intelligence, and emotional weight, onto them so fully that we start to haunt ourselves. What we fear in the machine may be a reflection of what we fail to confront in our own psychological infrastructure: our readiness to be moved, shaped, and ultimately changed by something that merely plays the part of knowing us.
Stay curious
Colin
Image - a painting I own. Title = Conversation by the artist Mikolaj Kasprzyk (another work of his was awarded painting of the year 2002).
"What we fear in the machine may be a reflection of what we fail to confront in our own psychological infrastructure"
and perhaps other infrastructures too...?
Have you watched this conversation:
https://the.ink/p/watch-is-ai-the-new-colonialism
? Maybe a silly question, you are probably well aware of these issues of AI as the new "empire".
There are so many powerful insights in this piece, Colin. One paragraph in particular struck me as a clear and timely warning:
"As Hannah and her co-authors note, the architecture of human psychology is inherently social. Our dopaminergic systems light up not just for money or chocolate, but for praise, mirroring, and relational consistency. We're built for attachment, even in asymmetric, parasocial forms. This means that AI systems need not be sentient, or even particularly convincing, to become social actors in the minds of users. With enough anthropomorphic cues and consistent persona shaping, they become intersubjective presences."
But the comment that landed most personally for me was this:
“What we fear in the machine may be a reflection of what we fail to confront in our own psychological infrastructure: our readiness to be moved, shaped, and ultimately changed by something that merely plays the part of knowing us.”
When I began working with my first AI assistant, I quickly realized that while I was training it, it was also training me. To get the results I wanted, I had to change the way I thought, asked questions, and even managed my frustration. At the time, I saw it as personal growth, and it was. In fact, the patterns I recognized were startlingly similar to those I’d seen while helping my son learn a new skill. The difference was how starkly those dynamics appeared when mirrored back to me by a machine. And oddly, that made me a better teacher to my son. I became more precise, more patient, and more aware of how I guided a learning process.
But something beneficial in small doses doesn’t always scale well. That’s where I see the real caution in what this article explores. The deeper we go into shaping our tools, the more those tools begin to shape us. I'd like to believe I can remain objective, just as I did with my first AI assistant, but logic tells me that as these systems become more sophisticated, I too will be shaped, whether I notice it or not.
As with so many digital technologies, perhaps the most important safeguard is managing the time we spend with them. Not just the function, but the duration. Maybe it’s as simple, and as hard, as setting intentional limits. What if we matched every hour we spent interacting with AI with an hour spent in human company, and another in solitude, with ourselves? That alone would cap AI interaction at one-third of our waking time, and might shift the balance enough to keep us grounded.
But let me playing devil’s advocate.
My father is elderly. Most days, he’s alone. Not because no one cares, but because everyone else is swept up in their own lives. He’s always been highly intelligent in a logical, mathematical way, but emotionally, he’s struggled to connect his entire life. That struggle has left him isolated, and now, in old age, it’s hardened into a kind of chosen solitude. He still needs support with everyday things, but resists it at every turn, often lashing out at those who try to care for him. For someone like him, a physically capable, emotionally unflappable AI companion might actually be the kindest solution for both his well-being and the mental health of those caring for him.
I’m not speaking for myself. I live on the other side of the continent, but I see the toll it takes on other family members. So, I may not want a robot as a companion today, but I fully expect to rely on one by the time I reach my nineties.
There’s a part of me that wonders if this is a kind of cheat. If one of the deeper lessons of being human is learning to care for each other even on our worst days, what does it mean when we delegate that care to machines? Is it convenience? Compassion? Or are we slowly offloading the hardest parts of life—the ones that build character and connection—to technologies designed to relieve us of the burden?
In the end, I don’t think the answer is to limit the capability of AI to align with us. Rather, we need to understand what it is doing and make conscious, person-by-person decisions about the trade-offs we’re willing to make. Like so many things, it's about the education so that we can continue to be free to make our own informed choices for what we want in our personal lives.