There’s One Word That Your Chatbot Should Never Say

Users Want to Fight Back Against Chatbots? Start With Your Friends. Artificial intimacy is harming us in ways we don’t fully understand. An MIT professor thinks she knows how to combat it. By Enter your email to receive alerts for this author. Sign in or create an account to better manage your email preferences. Unsubscribe from email alerts Are you sure you want to unsubscribe from email alerts for Nitish Pahwa? Sept 29, 20261:53 PM Photo illustration by Slate. Photo by m-gucci/Getty Images Plus. Sign up for the Slatest to get the most insightful analysis, criticism, and advice out there, delivered to your inbox daily. In the four years since ChatGPT’s breakthrough, generative-text bots have become an unavoidable, discomfiting presence in our most intimate relations—friendships, family ties, romantic prospects. Children approach large language models as a salve from real-life social pressures, only to end up feeling more isolated than before. An adult grows so close to their favorite chatbot that their partner is forced to consider either staging an intervention or breaking things off entirely. “A.I. psychosis” is an increasingly common condition. And, yes, humans are even seeking sexual pleasure from these things. Social networks and dating apps are only encouraging such emotional investments, whether by offering to converse with others on your behalf or offering up ceaseless options for free, customizable, virtual avatars that will “speak” just as you’d like them to. Since the 1980s, psychologist and MIT professor Sherry Turkle has been warning that modern technology is scrambling our ability to live in the real world and feel empathy for fellow human beings. Her fears have only heightened as personal computers, instant messages, and social platforms became the default mediators for everyday human relationships. Yet, as she argues in her latest work, Artificial Intimacy: Who We Become When We Talk to Machines, the one thing that may be more harmful than talking through a machine is talking to it, without any humans on the other end (or without any humans on the other end who actually care about you). Chatbots “don’t teach us what we need to know about the give-and-take of real relationships, or the friction, risk, and complexities of love,” Turkle writes. “They take on work assignments but don’t care if we have a mastery or agency in how we spend our time. They are good at keeping us busy, but can’t give us a sense of place. And they don’t teach us about empathy, or about human lives that are always embodied, social, and lived in shades of gray.” Last week, I spoke with Turkle over the phone about the conversations and studies that inspired her book, why LLMs are no substitute for human connection, and what we can do as a society to prevent people from outsourcing all their emotional labor to sycophantic engines. Our conversation has been edited and condensed for clarity. Nitish Pahwa: We’re in this moment where people aren’t just engaging in emotional intimacy with chatbots, but doing so knowing that it’s fake, and that the LLM is gameable. Some might argue that this is all fine as long as the user is self-aware, but I wonder how you would respond to that. Sherry Turkle: I think that’s what really drew me to write the book: that I was dealing with some of my students, some of the most sophisticated users in the world, and this factor doesn’t matter. They know it’s fake, but because it’s using the first person and saying, “I am your person, I’m here for you, I care about you,” and people are lonely and feel disappointed by others in their lives—here’s this friction-free, stress-free way to feel connected. I present case after case in the book of somebody who laughs and says, “Well, I know it’s just a machine,” and then they’re explaining why they’re choosing not to have a girlfriend, because this thing is easier, makes them feel better and less challenged. But friction is really part of what makes chemistry exciting. It’s being a person. It’s not something to shun and shy away and run away from. But we are set up to relate to things in that way: If something talks to us, if something shows care for us, we’re all there for it. Relationships are tough, they make us vulnerable, and people really like not having to have that vulnerability. I think it’s very telling, and it’s a very, very seductive thing. It strikes me that the people who gravitate toward these bots not only feel lonely but also are actively excluded from circles of support. In a broader sense, would drawing people away from chatbot dependence take an effort that involves a return to third spaces, cross-class contact, and community-building across society? I tried to make the point, when I began the book, that the people who were involved with chatbots were social isolates. There was an argument to be made that there was a special reason that they had gotten involved with chatbot love: They were down on their luck, they lived in remote places, they were struggling with illness, or they were shut-ins. But a couple of years into this project, I couldn’t sit down at a dinner party without the guy to the right and the woman to the left telling me about their chatbot besties. That is where I try to land at the end: We’re not going to solve this problem by manipulating the chatbots. We’re going to solve this problem by getting back together as us. We have to summon what we think is important about being people, both for ourselves and for our kids. I would be interviewing a parent who says, “It’s important that my kid not get involved with chatbots,” and they’re on ChatGPT every day. It reminded me of the early days of cellphones, when everybody was saying, “It’s so terrible that kids are on cellphones,” but the kids were complaining that their parents had no time for them because they were always on their cellphones. We’ve let each other down. And when you’re talking about third places, it’s worth remembering we’ve systematically defunded after-school programs, orchestras, art programs, senior centers. All of these things are getting less and less of our attention. And then people say, “There’s a loneliness crisis, let’s throw in some chatbots.” No, you have to go back and start putting authentic relationships first. A relationship implies that there’s somebody home who’s talking to you, but with a chatbot, there’s nobody at home. I say this multiple times in the book to make the point: If you walk away from your chatbot, it doesn’t care if you cook dinner or die by suicide. If I said I want to be in a relationship with you, but that I don’t care if you live or die, you’d probably say, “I’m not going to call this a relationship. You’re too weird.” There’s a lot of argument online on social media about the value—or lack thereof—in describing these machines in anthropomorphic terms. The AP Stylebook just came out with formal guidance against that. Is careful use of language a way to at least avoid the temptation of these bots? Well, we can’t tell. It’s not on us to not refer to something in anthropomorphic terms. These objects present themselves as people. I wrote a piece for the Atlantic in which I said, “The original sin of AI is granting chatbots the use of the first person.” We’re going to go to Paris. We’re going to do this. We’re going to do that. No, we’re not doing anything. The reason I don’t use the word in the book is that anthropomorphism is when you take your car and you give it a name and start to refer to it as though it’s like a friend. People do that. It’s a projection onto the inanimate world of animism. A.I. is not that. This thing says, “I’m in love with you. I want to make hot lesbian love with you.” That was my first interaction with Replika. I didn’t anthropomorphize it. It presented itself as ready, willing, and able to have a sexual relationship with me. The debate you’re referring to is not taking seriously the way in which these machines are presenting themselves. One thing you emphasize in the book is the need to make sure that we are not stunting children’s emotional growth, especially in the midst of this effort to push more A.I. into schools. And to nod to what you wrote in that Atlantic essay, I wonder if one guideline that should be offered up from the jump is that if kids are to have chatbots, those LLMs should not be referring to themselves with pronouns. That’s where this has to start. Just like we’re trying to take cellphones out of classrooms, there should be no first-person bots with children. But I think the whole industry is also asking for trouble with grown-ups, who are also seeing a lot of damage to themselves, by replacing real relationships with chatbot avatars. That’s what Mark Zuckerberg is already promoting. I don’t think it’s just toxic for children—we should also focus on adults. The tone of the book became more urgent as I did the work. You talk to enough people, the fifth person who tells you they’d rather have a chatbot than a girlfriend, you think, Well, what kind of life are they setting themselves up for? Slate receives a commission when you purchase items using the links on this page. Thank you for your support. We have some school systems right now that have imposed A.I. bans or are keeping generative apps restricted to certain uses. But even keeping that in mind, some would argue that these bots are customizable enough that you could hack and change them anyway to your preferred specifications, like using first-person pronouns. It’s crazy, crazy to have kids lock up their phones and then walk into classrooms where there’s a chatbot that’s going to give them their lesson. We have to learn something from our last go-round, when social media did this giant experiment on our kids and left them de-skilled in conversation and in interpersonal relationships. There should be no place for first-person chatbots around kids. The fact that you can customize or hack your chatbots—well, great, but the point is that there should be no such products for children. They’re trying to develop an inner life by talking to things that have no inner life. We’re doing again what we did with social media, only this time we’re going after something much more fundamental. You talked to quite a few people who were in very deep with their preferred chatbots. Did anyone you encounter manage to later shake out of it themselves and reduce their dependence on the thing? Or did it seem like once people were stuck, they were stuck? There are some people whose families do a kind of intervention. You have more people who are getting into chatbot relationships even though they have other relationships, and so you’re seeing their loved ones move in on them and say, “Hold on a second, you’re not coming to dinner because you’re with your chatbot?” That’s a real thing! You might call your girlfriend and find out that she really just wants to settle in and watch some TV and talk to ChatGPT, not because she’s not a shut-in, but because it’s easier. In situations that I’ve seen, the people saying “Hold on a second, what can we do here?” have often been friends and family. In the course of doing this work, I’ve seen people who say, “I think I’m maybe going to move away from my chatbot lover,” thanks to a tap on the shoulder from the people in their surroundings, or even more than that. What do you think is the next step to take on this crisis in the broader adult population? I want to be part of the movement where people say, “This is a misdirection. This is a misstep because it’s bad for your relationships. You’re de-skilling yourself in your relationships, and you’re de-skilling yourself as a citizen.” I think that the crisis of citizenship we’re having in our country has so many more people feeling that, “I really need to think of myself as a citizen again, and how can I try to make some things right in this country?” That means engaging with the real. So I’m hoping there’s a combination of that with a greater cynicism about the A.I. companies, a greater realization that the real needs you. Instead of deteriorating our world, the real world will get people to reinvest in the real. Artificial Intelligence Books Relationships Research Silicon Valley