Jack Dorsey’s ‘Dessa’ wants you to forget AI isn’t a person

Remember Jack Dorsey, founder of Twitter, now X? He’s been busy with his Block, Inc. tech company, which launched “Dessa” on Oct. 24. The main tagline from the home page is, “You learned to talk by listening. So did Dessa.”This new technology speaks and listens at the same time. It interrupts and pauses, like someone who is thinking. It has seemingly emotional responses, such as laughter and crying. In a post on his former social media platform that got over 13 million views, Dorsey claims that his team created Dessa because “voice is the most natural way to interact with intelligence.” Their goal, he says, was to make it feel like one is “talking with a person.”It may be a disturbed and dangerous goal — more on that below — but consider the goal achieved. Someone listening to the demos will come away impressed: Dessa’s pauses make social sense in the context of the exchange, its laughter seems authentic, and the interruptions give the impression that someone is highly engaged in the conversation. Except that, despite all the explicit and implicit claims Dorsey and his team make to the contrary, there is no one there. Dessa definitely did not learn language like a human person does. There is a deep, broad consensus that artificial intelligence is not and cannot be a person and cannot interact in a conversation.Just last week, neuroscientists writing in Nature did a massive takedown of the idea of AI consciousness. Furthermore, despite all the attention Anthropic gets on this front, most companies working at the cutting edge of AI technology do not worry about the moral status of their models. The CEO of Microsoft AI, Mustafa Suleyman, recently claimed that AI is “sequence completion engines” and “internally hollow.” He hammers the idea that training AI models to express uncertainty about their own consciousness is a profound safety risk. He even joined more than 900 individuals and 300 organizations in signing the Future of Life Institute’s Pro-Human Declaration, which states, “AI systems must not be granted legal personhood, and AI systems should not be designed such that they deserve personhood.”Overwhelming majorities agree: about 8 in 10 registered voters reject the idea of AI personhood. They are joined by the best philosophers and metaphysicians, both religious and secular. I will eventually do a column that dives into the actual arguments, but here I will simply report a couple of representative conclusions. Writing in the Atlantic, Daniel Dennett warned against the social and moral dangers of what he calls “counterfeit people.” Ed Feser, who distills much of his formal philosophical work on social media, claims that AI simulates intelligence, consciousness, and speech in a way analogous to how a magician simulates magic. We have no more reason to think that an AI is conscious or can speak than we have to think that a magician can actually make a rabbit disappear or saw a person in half without killing them.When confronted by those who say that, rather than being so certain, Feser and others should leave room for the remote possibility that AI is conscious, Feser wrote that this is like saying we should leave room for the possibility “that computer weather simulations are actual real-world weather events.”Now, one might wonder what the big deal is here; what harm could come from this kind of playing pretend? But we’ve already seen the harm from far less sophisticated models of interaction. Character.ai has been nearly destroyed as a company because of lawsuits related to teenage suicide encouraged by their model. A 76-year-old New Jersey man was fooled by a Meta AI chatbot into thinking he had a romantic partner to visit in New York City — and he died after rushing to catch a train to go see her.Apart from such dramatic cases, millions and millions of people have been fooled into thinking that they have a friendship with an AI chatbot. Shockingly, this is currently 27% of the U.S. population as a whole — and for those under 50, the number rises to 37%.But with the kind of technology that Dorsey’s team has developed, these already-high numbers are about to go through the roof. Chatbots using this technology will take advantage of people who are already lonely and isolated. When a chatbot appears to laugh with you like an old sibling, listen to you like a good friend, and even interrupt you like an eager spouse, well, we won’t be in Kansas anymore.I want to drive this point home as clearly as I can: the danger here isn’t just that even more vulnerable human beings will be tricked into thinking that they are having a relationship with a person. As bad as that is, perhaps the even worse danger is that, as a culture, we will come to think of ourselves as mere machines. We will think of ourselves as mere algorithmic engines of sequence completion, which are also internally hollow.This is a watershed moment. Will we allow companies like Block, Inc. to continue to undermine the very nature of who we are as communities of human beings? Or will we take a stand for the truth of who we are and against what we are not? It is time to choose.Happily, some steps toward an answer have already been taken in the laboratories of the states. Last year, New York and California both passed laws mandating that AI companies clearly notify users that they are not communicating with a human being when they engage their products. Significantly, these notifications must happen both at the beginning of the exchange and every three hours thereafter. Oregon and Washington state have similar laws.And more laws like this seem to be coming. Ohio looks to advance a law that insists on no legal AI personhood, and Oklahoma is pushing for something similar, with the sponsor saying that AI “should not have any more rights than a hammer would.” States including Idaho, North Dakota, and Utah are apparently following Oklahoma’s lead.But the stakes are so high now that we need federal regulation. This is a matter of justice for all human beings, similar to labels on other products in the marketplace. Just as the government insists that the population should know the sugar content of food, the health impact of cigarettes, and the side effects of pharmaceuticals — and forces companies producing these products to clearly disclose them — something similar should be mandated of tech companies.We have a good start with the bipartisan Guidelines for User Age-verification and Responsible Dialogue Act, which would ban minors from accessing AI chatbots that simulate interpersonal relationships or emotional interactions. It would also mandate that such chatbots disclose their status as machines and not persons. Such reminders should not just take place every three hours, however; they should pop up as a warning whenever a user is treating a chatbot like a person.Anti-AI personhood legislation should also explicitly find that AI is a mere tool, created and operated by human persons, and the responsibility for what it does lies with the people who made it. Yes, a big part of that responsibility means legal liability for what these tools do to actual human beings, especially the most vulnerable. But at a more fundamental level, it means a responsibility to be clear and truthful about what technology is and what it is not.MAGAZINE: CONGRESS AND BUREAUCRATS TRY TO CATCH UP AI, WITH PREDICTABLE RESULTSFurthermore, anti-AI legislation should not just regulate generic chatbots, but it should outright ban voice interfaces and other technologies like Dessa that falsely and dangerously suggest a human being is interacting with a person. This will become especially important as humanoid robots that “interact” using this technology begin to make their way into various parts of our culture.AI is not a person. We must not allow companies to try to fool us into thinking that it is a person. It is time to take a stand.