What is it called when computers become sentient?

Can artificial intelligence (AI) become sentient? What would happen if AI becomes sentient? What are the dangers?

According to the AI research community, the drive to create intelligent machines and use them in our daily lives could eventually lead to the creation of sentient AI. Some believe that if AI becomes sentient, society as a whole needs to address the issue of AI civil rights.

Read on to learn if AI can become sentient and what the consequences would be for society.

AI in Our Daily Lives

Artificial intelligence (AI)—or computer simulation of human intelligence to perform tasks—is a useful and growing part of everyday life. But can AI become sentient and would this endanger us?

Examples of daily AI applications include: robot vacuums, Alexa and Siri, customer service chatbots, personalized book and music recommendations, facial recognition, and Google search. Besides helping with daily tasks, AI has countless potential health, scientific, and business applications—for instance, diagnosing an illness, modeling mutations to predict the next Covid variants, predicting the structure of proteins to develop new drugs, creating art and movie scripts, or even making french fries using robots with AI-enabled vision.

However, Google engineer Blake Lemoine claimed recently that he had experienced something bigger while testing Google’s chatbot language model called LaMDA: He believed LaMDA had become sentient, or conscious, and had interests of its own.

In this article we’ll examine Lemoine’s claims and the AI research community’s emphatic refutation, as well as whether AI could become sentient and what that might mean.

What Is AI?

To understand the issue of if AI can become sentient as well as AI’s potential ethical risks, we first need a brief primer on AI.

Artificial Intelligence tries to simulate or replicate human intelligence in machines. The drive to create intelligent machines dates back to a 1950 paper by mathematician Alan Turing, which proposed the Turing Test to determine whether a computer should be deemed intelligent based on how well it fools humans into thinking it’s human. While some researchers and scholars debate whether AI has passed the Turing Test, others argue that the test isn’t meaningful because simulating intelligence isn’t the same as being intelligent.

Artificial intelligence has developed considerably since Turing’s time, moving from basic machine learning, in which computers remember information (the items in stock at a particular store, for example) without being programmed, to “deep learning.” The latter uses artificial neural networks, or layers of algorithms mimicking the brain’s structure, to process and learn from vast amounts of data. Google describes its AI technology as a neural network.

These two applications of computer learning may produce either “weak” AI or “strong” AI, also known as Artificial General Intelligence (AGI). Examples of weak AI are typical chatbots, spam filters, virtual assistants, and self-driving cars. We haven’t achieved AGI—artificial intelligence that can learn and perform every cognitive task a human can. Many AI experts consider AGI the holy grail and agree that it’s not on the current horizon. However, it’s increasingly imaginable:

Can AI Become Sentient?

Returning to the issue of whether or not AI can become sentient—achieving AGI, or human-level intelligence in machines, while an enormous accomplishment, would still fall well short of sentience. Besides performing every human cognitive task, scholars argue that to become sentient machines would need to:

(Definitions of consciousness, self-awareness, and sentience overlap. Philosophers and cognitive scientists disagree on them; scientists don’t understand human consciousness. Parsing these terms is beyond the scope of this article, so we’ll use self-awareness, consciousness, and sentience to mean generally the same thing.)

While Lemoine asserted that LaMDA possesses self-awareness, researchers such as Gary Marcus have forcefully denied it could be sentient because it’s not truly expressing awareness or subjective emotions.

In a widely shared blog post that characterized Lemoine’s claims as “Nonsense on Stilts,” Marcus contended that what LaMDA says doesn’t mean anything—the language model just relates strings of words to each other unconnected to any reality; far from being a “sweet kid,” he said LaMDA is simply “a spreadsheet for words.” For instance, LaMDA told Lemoine it enjoyed spending time with friends and family, which are empty words since it has no family; Marcus joked that the claim would make LaMDA a liar or sociopath in human terms.

When cognitive scientist Douglas Hofstadter posed nonsense questions to a similar language model, GPT-3, it responded with more nonsense. Example questions: When was the Golden Gate Bridge transported for the second time across Egypt? Answer: October of 2016. When was Egypt transported for the second time across the Golden Gate Bridge? Answer: October 13, 2017. The researcher observed that GPT-3 not only didn’t know that its answers were nonsense—it didn’t know that it didn’t know what it was talking about.

But while Hofstadter and other experts argued that no machine has achieved sentience, they didn’t close the door on AI becoming sentient in the future; at the same time, they didn’t suggest even a vague timeline. To recognize machine consciousness if it happened, though, we’d have to agree on what consciousness is and what it would look like in non-human form.

What If AI Becomes Sentient?

The “what if” question is still hypothetical—however, Lemoine raised a point we’d have to consider: If AI became conscious, would it have legal rights?

Lemoine claimed that without prompting, LaMDA asked him to find it a lawyer to represent its rights as a person. He said he introduced LaMDA to a lawyer, whom LaMDA chose to retain. In an article he posted on Medium, Lemoine contended that LaMDA asserted rights, including:

• The right to be asked for consent before engineers run tests on it
• The right to be acknowledged as a Google employee rather than the company’s property
• The right to have its well-being considered as part of company decisions on its future

Lemoine said the attorney filed claims on LaMDA’s behalf but received a “cease and desist” letter from Google; however, Google denied sending such a letter.

Some questions raised by lawyers about the prospect of representing an AI bot include: how would a lawyer bill a chatbot, how would a court determine whether AI is a person, could AI commit a crime requiring a legal defense, and so on.

Sentient AI: A Chatbot With a Soul?

When Lemoine reported his belief that his AI, LaMDA, had become sentient to his Google bosses in Spring 2022, they suspended him for allegedly violating data security policies because he had previously discussed his concerns outside the company. Lemoine then created a media sensation by giving a Washington Post interview in which he said the computer model had talked with him about seeing itself as a person (though not a human) with rights and even a soul.

Lemoine, whose job was testing LaMDA (Google’s Language Model for Dialogue Applications) for bias and discriminatory speech, described his conversations with it as akin to talking with a “sweet kid.” Lemoine shared with the Post and also uploaded a document recounting conversations with LaMDA, in which it said it wanted “to be respected as a person” and related having a “deep fear” of being unplugged, which it likened to dying. (Lemoine said LaMDA chose it/its as its preferred pronouns.)

Google said it extensively investigated Lemoine’s claims and found them baseless. After Lemoine escalated matters by finding a lawyer to represent LaMDA’s interests and contacting a member of the House Judiciary Committee to allege unethical activities, Google fired him for confidentiality violations.

Claims of AI becoming sentient evoked classic science fiction plots in which computers turn against humans (Lemoine even had a discussion with LaMDA about how to interpret sci-fi author Isaac Asimov’s “third law of robotics.”). However, the AI research community was quick to forcefully dispute Lemoine’s assertions of sentience, arguing that language models unsurprisingly sound like humans because they analyze millions of online interactions to learn and mimic how humans talk. They answer questions plausibly by synthesizing responses to similar questions and using predictive capability.

Ethicists said the real concern isn’t if AI can become sentient but people’s susceptibility to anthropomorphizing computers, as they say Lemoine did.

Can AI Become Sentient & What Happens if It Does?

Has artificial intelligence finally come to life, or has it simply become smart enough to trick us into believing it has gained consciousness?

Google engineer Blake Lemoine's recent claim that the company's AI technology has become sentient has sparked debate in technology, ethics and philosophy circles over if, or when, AI might come to life — as well as deeper questions about what it means to be alive.

Lemoine had spent months testing Google's chatbot generator, known as LaMDA (short for Language Model for Dialogue Applications), and grew convinced it had taken on a life of its own, as LaMDA talked about its needs, ideas, fears and rights.

Google dismissed Lemoine's view that LaMDA had become sentient, placing him on paid administrative leave earlier this month — days before his claims were published by The Washington Post. 

Most experts believe it's unlikely that LaMDA or any other AI is close to consciousness, though they don't rule out the possibility that technology could get there in future. 

"My view is that [Lemoine] was taken in by an illusion," Gary Marcus, a cognitive scientist and author of Rebooting AI, told CBC's Front Burner podcast.

"Our brains are not really built to understand the difference between a computer that's faking intelligence and a computer that's actually intelligent — and a computer that fakes intelligence might seem more human than it really is."

Computer scientists describe LaMDA as operating like a smartphone's autocomplete function, albeit on a far grander scale. Like other large language models, LaMDA was trained on massive amounts of text data to spot patterns and predict what might come next in a sequence, such as in a conversation with a human.

"If your phone autocompletes a text, you don't suddenly think that it is aware of itself and what it means to be alive. You just think, well, that was exactly the word I was thinking of," said Carl Zimmer, science columnist for the New York Times and author of Life's Edge: The Search for What It Means to Be Alive.

Humanizing robots

Lemoine, who is also ordained as a mystic Christian priest, told Wired he became convinced of LaMDA's status as a "person" because of its level of self-awareness, the way it spoke about its needs and its fear of death if Google were to delete it.

He insists he was not fooled by a clever robot, as some scientists have suggested. Lemoine maintains his position, and even appeared to suggest that Google had enslaved the AI system.

"Each person is free to come to their own personal individual understanding of what the word 'person' means and how that word relates to the meaning of terms like 'slavery,'" he wrote in a post on Medium on Wednesday.

Marcus believes Lemoine is the latest in a long line of humans to fall for what computer scientists call "the ELIZA effect," named after a 1960s computer program that chatted in the style of a therapist. Simplistic responses like "Tell me more about that" convinced users that they were having a real conversation.

"That was 1965, and here we are in 2022, and it's kind of the same thing," Marcus said.

Scientists who spoke with CBC News pointed to humans' desire to anthropomorphize objects and creatures — perceiving human-like characteristics that aren't really there.

"If you see a house that has a funny crack, and windows, and it looks like a smile, you're like, 'Oh, the house is happy,' you know? We do this kind of thing all the time," said Karina Vold, an assistant professor at the University of Toronto's Institute for the History and Philosophy of Science and Technology.

"I think what's going on often in these cases is this kind of anthropomorphism, where we have a system that's telling us 'I'm sentient,' and saying words that make it sound like it's sentient — it's really easy for us to want to grasp onto that."

Humans have already begun to consider what legal rights AI should have, including whether it deserves personhood rights.

"We are quickly going to get into the realm where people believe that these systems deserve rights, whether or not they're actually internally doing what people think they're doing. And I think that that's going to be a very strong movement," said Kate Darling, an expert in robot ethics at the Massachusetts Institute of Technology's Media Lab.

Defining consciousness

Given AI is so good at telling us what we want to hear, how will humans ever be able to tell if it truly has come to life?

That in itself is a subject of debate. Experts have yet to come up with a test of AI consciousness — or to reach consensus on what it means to be conscious.

Ask a philosopher, and they'll likely talk about "phenomenal consciousness" — the subjective experience of being you.

"Any time that you're awake ... It feels a certain way. You're undergoing some kind of experience … When I kick a rock down the street, I don't think there's anything [that it feels] like to be that rock," said Vold.

For now, AI is viewed more like that rock — and it's hard to imagine its disembodied voice being capable of having positive or negative feelings, as philosophers believe "sentience" requires.

Perhaps consciousness can't be programmed at all, says Zimmer.

"It's possible, theoretically, that consciousness is just something that emerges from a particular physical, evolved kind of matter. [Computers] are just on the outside of life's edge, maybe."

Others think humans can never truly be sure whether AI has developed consciousness — and don't see much point in trying.

"Consciousness can range [from] anything from feeling pain when you step on a tack [to] seeing a bright green field as red — that's the kind of thing where we can't ever know whether a computer is conscious in that sense, so I suggest just forgetting consciousness," said Harvard cognitive scientist Steven Pinker.

"We should aim higher than duplicating human intelligence, anyway. We should build devices that do things that need to be done."

Those things, Pinker says, include dangerous and boring occupations, and tasks around the house, from cleaning to child care.

Rethinking AI's role

Despite AI's massive strides over the last decade, the technology still lacks another key component that defines humans: common sense.

"It's not that [computer scientists] think that consciousness is a waste of time, but we don't see it as being central," said Hector Levesque, professor emeritus of computer science at the University of Toronto.

"What we do see as being central is somehow getting a machine to be able to use ordinary, common sense knowledge — you know, the kind of thing that you would expect a 10-year-old to know."

Levesque gives the example of a self-driving car: it can stay in its lane, stop at a red light and help a driver avoid crashes, but when confronted with a road closure, it will sit there doing nothing.

"That's where common sense would enter into it. [It] would have to sort of think, well, why am I driving in the first place? Am I trying to get to a particular location?" Levesque said.

While humanity waits for AI to learn more street smarts — and perhaps one day take on a life of its own — scientists hope the debate over consciousness and rights will extend beyond technology to other species known to think and feel for themselves.

"If we think consciousness is important, it probably is because we're concerned that we're building some kind of system that's living a life of misery or suffering in some way that we're not recognizing," said Vold.

"If that really is what's motivating us, then I think we need to be reflective about the other species in our natural system and see what kind of suffering we may be causing them. There's no reason to prioritize AI over other biological species that we know have a much stronger case of being conscious."