Scientists Claim: ChatGPT May Already Have Consciousness, AI Could Be Immortal
-
In real life, Sutskever is not fond of socializing and rarely appears in the media. The only thing that excites him is artificial intelligence.
Recently, Ilya Sutskever was interviewed by Will Douglas Heaven from MIT Technology Review. In the interview, he discussed OpenAI's early entrepreneurial history, the possibility of achieving AGI, and OpenAI's future plans for controlling 'superintelligence.' He hopes that future superintelligence will view humans as parents view their children.
The following is the main text:
Ilya Sutskever lowers his head in thought. His arms are spread out, fingers resting on the table, like a pianist about to play the first note at a concert. We sit in silence.
I came to meet with Sutskever, co-founder and chief scientist of OpenAI, at his company's unmarked office on an inconspicuous street in San Francisco's Mission District. I wanted to hear about the next steps for the world-disrupting technology he helped create. I also wanted to know his next moves, especially why building the next flagship generative model for his company is no longer his focus.
Sutskever told me that his new focus is not on creating the next GPT or image generator DALL·E, but on researching how to prevent AGI (which he considers a hypothetical future technology) from spiraling out of control.
Sutskever also told me many other things. He believes ChatGPT might have consciousness (if you squint hard enough). He thinks the world needs to wake up to the true power of the technology his company and others are striving to create. He also believes that one day humans will choose to merge with machines.
Much of what Sutskever says sounds crazy, but not as crazy as it would have sounded a year or two ago. As he himself told me, ChatGPT has already rewritten many people's expectations of the future, turning 'never going to happen' into 'will happen faster than you think.'
He said:
"It's important to discuss the direction of all this."
When predicting the future of AGI (Artificial General Intelligence, AI as smart as humans), he spoke with the confidence as if it were just another iPhone:
"One day, AGI will be achieved. Maybe by OpenAI. Maybe by another company."
Since the release of the wildly popular ChatGPT last November, discussions surrounding OpenAI have been impressive, even in an industry known for hype. No one can help but be curious about this $80 billion startup. World leaders seek (and receive) private meetings with CEO Sam Altman. The awkward product name ChatGPT frequently pops up in casual conversations.
This summer, OpenAI's CEO Sam Altman spent most of his time on a weeks-long outreach tour, engaging in friendly talks with politicians and delivering speeches to packed venues worldwide. But Sutskever is not as much of a public figure as he is, nor does he often give interviews.
He speaks thoughtfully and methodically. He pauses for long periods, carefully considering what he wants to say and how to say it, turning questions over like puzzles. He seems uninterested in talking about himself.
He said:
"My life is simple. I go to work, then go home. I don't do anything else. A person can have a lot of social interactions, attend many events, but I don't."
But when we talk about artificial intelligence and the epoch-making risks and rewards he foresees, his eyes light up:
"AI will be immortal and shake the world to its core. Its birth is like the creation of heaven and earth."
In a world without OpenAI, Sutskever would still be etched into the annals of AI history. As an Israeli-Canadian born in the former Soviet Union, he grew up in Jerusalem from the age of five (he still speaks Russian, Hebrew, and English fluently). Later, he moved to Canada, studying under AI pioneer Geoffrey Hinton at the University of Toronto. (Sutskever declined to comment on Hinton's statements, but his focus on the risks of superintelligence suggests they share common ground.)
Hinton later shared the Turing Award with Yann LeCun and Yoshua Bengio for their work on neural networks. But when Sutskever joined his team in the early 2000s, most AI researchers considered neural networks a dead end. Hinton was the exception.
Sutskever says:
"This was the beginning of generative AI. It was really cool, just not good enough yet."
Sutskever is fascinated by the brain: how it learns, and how to recreate or at least mimic this process in machines. Like Hinton, he saw the potential of neural networks and the trial-and-error techniques Hinton used to train them—deep learning. "It kept getting better and better," Sutskever recalls.
In 2012, Sutskever, Hinton, and another of Hinton's graduate students, Alex Krizhevsky, built a neural network called AlexNet. Trained to recognize objects in photos, it far surpassed other software at the time. This was the big bang moment for deep learning.
After years of failures, they finally proved the astonishing effectiveness of neural networks in pattern recognition. All you need is enough data (they used one million images from the ImageNet dataset, maintained by Princeton researcher Fei-Fei Li since 2006) and computing power that is off the charts.
The computing power came from a new type of chip called a Graphics Processing Unit (GPU), produced by Nvidia. GPUs were originally designed to render fast-moving video game visuals onto screens at lightning speed. However, the computations GPUs excel at—multiplying large numerical grids—are strikingly similar to those required for training neural networks.
Nvidia is now a trillion-dollar company. At the time, it was eager to find applications for its niche new hardware.
"When you invent a new technology, you have to embrace crazy ideas," said Nvidia CEO Jensen Huang. "My mindset is always looking for something outlandish, and the idea that neural networks would transform computer science was a very outlandish one."
Huang said that when the Toronto team was developing AlexNet, Nvidia sent them a few GPUs to try out. But what they wanted was the latest version, a chip called the GTX580, which sold out quickly in stores. According to Huang, Sutskever drove from Toronto to New York to buy the GPUs.
"People were lining up around the block," Huang said. "I don’t know how he did it—I’m pretty sure each person could only buy one; we had a strict policy of one GPU per gamer. But he apparently filled a trunk with them. A trunk full of GTX580s changed the world."
It’s a great story, though it might not be entirely true. Sutskever insists his first GPUs were bought online. But in this bustling industry, such myths are commonplace.
Sutskever himself was more modest, saying:
"I thought that if I could make even a tiny bit of real progress, I would consider it a success. The real-world impact felt too distant because computers were so weak back then."
After the success of AlexNet, Google came knocking. They acquired Hinton's company DNNresearch and hired Sutskever. At Google, Sutskever demonstrated that deep learning's pattern recognition capabilities could be applied to data sequences like words and sentences, not just images. Jeff Dean, Sutskever's former colleague and current Chief Scientist at Google, said: "Ilya has always been interested in language, and we've had great discussions over the years. Ilya has a strong intuition about where things are headed."
But Sutskever didn't stay at Google for long. In 2014, he was recruited as a co-founder of OpenAI. This new company had $1 billion in funding (from CEO Altman, Musk, Peter Thiel, Microsoft, Y Combinator, and others) and harbored Silicon Valley-style ambitions to develop AGI from the outset - a prospect few took seriously at the time.
Sutskever was the driving force behind the company, and his ambition was understandable. By then, he had already achieved increasing success with neural networks. Dalton Caldwell, Managing Director of Investments at Y Combinator, said Sutskever was already renowned and was a key factor in OpenAI's appeal.
Caldwell recalled: "I remember Sam (Altman) saying Ilya was one of the most respected researchers in the world. He believed Ilya could attract many top AI talents. He even mentioned that Yoshua Bengio, a world-leading AI expert, thought it would be impossible to find anyone more suitable than Ilya to be OpenAI's Chief Scientist."
However, OpenAI struggled initially.
Sutskever said, "When we started OpenAI, there was a period when I wasn't sure how we would continue to make progress. But I had a very clear belief that you couldn't bet against deep learning. Somehow, every time we hit a roadblock, researchers would find a way around it within six months or a year."
His belief paid off. In 2016, OpenAI's first large language model, GPT (which stands for "Generative Pre-trained Transformer"), was released. This was followed by GPT-2 and GPT-3. Then came the eye-catching image-generation model DALL·E. At the time, no one had created anything quite like it. With each release, OpenAI expanded the boundaries of what was considered possible.
In November of last year, OpenAI launched a free-to-use chatbot, repackaging some of its existing technology. It reset the agenda for the entire industry. At the time, OpenAI had no idea how popular its product would become.
The company's expectations couldn't have been lower. Sutskever admitted, "I confess, I was a bit embarrassed—I don't know if I should admit this, but whatever, it's the truth—when we created ChatGPT, I didn't know if it was any good. When you asked it a factual question, it would give you a wrong answer. I thought it would be underwhelming, and people would say, 'Why did you make this? It's so boring!'"
Sutskever noted that the appeal lay in its convenience. The large language model powering ChatGPT had already existed for months. But packaging it in an easily accessible interface and offering it for free allowed billions of people to experience, for the first time, what OpenAI and other companies were building.
Sutskever said, "That first-time experience captivated people. The first time you used it, I think it was almost a spiritual experience. You'd think, 'Oh my God, the computer seems to understand what I'm saying.'"
OpenAI amassed 100 million users in less than two months, with many captivated by this astonishing new tool. Aaron Levie, CEO of storage company Box, summarized the post-launch atmosphere on Twitter: 'ChatGPT is one of those rare moments in technology where you get a glimpse of how everything is about to change.'
When ChatGPT spouts nonsense, that sense of wonder quickly collapses. But by then, it doesn't matter. Sutskever says, 'That glimpse was enough. ChatGPT changed people's perceptions.'
'In the field of machine learning, AGI is no longer a dirty word,' he said. 'This is a huge shift. The historical attitude was: AI doesn't work, AI doesn't work, every step is incredibly hard, you have to fight for every tiny improvement. When people hyped AI, researchers would say, "What are you talking about? This doesn't work, that doesn't work. There are too many problems." But with ChatGPT, the feeling started to change.'
Did this shift only begin a year ago? 'It was because of ChatGPT,' he said. 'ChatGPT made machine learning researchers dream.'
OpenAI's scientists have been evangelists from the start, sparking these dreams through blog posts and speaking tours.
It's working: 'Now we have people talking about how far AI will go, people discussing AGI or superintelligence. Not just researchers. Governments are talking about it too—it's crazy.'
Sutskever insists all this discussion about technology that doesn't yet exist (and may never exist) is good because it makes more people aware of the future he already takes for granted.
He said: "You can do many amazing things with AGI, incredible things: automate healthcare, make healthcare costs a thousand times lower, healthcare outcomes a thousand times better, cure many diseases, truly solve global warming. But many people are also worried, oh my, can AI companies successfully manage this enormous technology?"
AGI sounds more like a wish-granting genie than a technology that can appear in the real world. Few would refuse to save lives and solve climate change. But the problem with a non-existent technology is that you can say anything you want about it.
When Sutskever talks about AGI, what exactly is he referring to?
He said: "AGI is not a scientific term. It's just a useful threshold, a reference point. It's an idea." He began, then paused. "It refers to the level of intelligence in AI where if a human can accomplish a task, AI can too. Then, you can say AGI has been achieved."
AGI remains one of the most controversial ideas in the AI field. Few believe the arrival of AGI is inevitable. Many researchers think significant conceptual breakthroughs are still needed before we see anything like what Sutskever envisions, while some believe we will never see it.
Yet, this has been his vision from the start. Sutskever said: "I have always been inspired and motivated by this idea. It wasn't called AGI back then, but you know, like making neural networks do everything. I didn't always believe they could do it. But it's a mountain to climb."
He drew an analogy between neural networks and how the brain works. Both receive data, aggregate signals from the data, and then decide whether to propagate these signals based on simple processes (mathematics in neural networks, chemistry and bioelectricity in the brain). It's a simplified metaphor, but the principles are similar.
Sutskever said:
"If you believe this, if you allow yourself to believe this, then many interesting implications arise. If you have a very large artificial neural network, it should be capable of doing many things. In particular, if the human brain can do something, then a large artificial neural network should be able to do something similar."
"If you take this realization seriously enough, everything will fall into place," he said. "Most of my work can be explained by this."
While discussing the brain, I asked Sutskever about a post he made on X (Twitter). His posts read like a scroll of aphorisms: "If you value intelligence above all other human qualities, you'll have a bad time"; "Empathy in life and business is underrated"; "Perfection ruins many perfectly good things."
In February 2022, he posted, "Perhaps today's large neural networks are slightly conscious" (to which Murray Shanahan, Chief Scientist at Google DeepMind and scientific advisor for the film Ex Machina, replied: "...just as a large wheat field might be slightly spaghetti").
When I brought this up, Sutskever laughed. Was he joking? He wasn't. "Are you familiar with the concept of Boltzmann brains?" he asked.
He was referring to a (tongue-in-cheek) thought experiment in quantum mechanics named after 19th-century physicist Ludwig Boltzmann, where one imagines random thermodynamic fluctuations in the universe causing brains to suddenly appear or disappear.
"I think these language models are somewhat like Boltzmann brains," Sutskever said. "You start talking to it, you speak for a while; then when you're done, the brain just..." He made a gesture of disappearance with his hand. Poof—goodbye, brain.
I asked him, are you saying that when the neural network is active, when it's firing, there's something there?
He replied:
"I think it might be. I'm not sure, but it's a possibility that's hard to refute. But who knows what will happen, right?"
While others are still grappling with the idea of machines matching human intelligence, Sutskever is preparing for machines to surpass us. He calls it artificial superintelligence: "They will see more clearly. They will see things we cannot."
I still find it hard to understand what this really means. Human intelligence is our benchmark for intelligence. What does Sutskever mean by intelligence that is smarter than humans?
He said we've already seen a limited example of superintelligence in AlphaGo. In 2016, DeepMind's AI Go program defeated one of the world's top Go players, Lee Sedol, with a score of 4:1 in a match.
Sutskever said:
"It discovered a way to play Go that was different from the methods humans had developed over thousands of years. It introduced new ideas," said Sutskever, referring to AlphaGo's enigmatic 37th move in its second match against Lee Sedol. This move, which stunned commentators and was initially thought to be a mistake, turned out to be a brilliant, unprecedented strategy in Go history, dubbed the "AlphaGo style" by enthusiasts. "Imagine AlphaGo's insight being so profound and comprehensive," Sutskever remarked.
This line of thinking led Sutskever to make the biggest shift in his career. Together with OpenAI scientist Jan Leike, he formed a team focused on what they term "superalignment." Alignment, in AI jargon, refers to ensuring AI models do exactly what they are intended to do. Superalignment is OpenAI's term for addressing the alignment challenges posed by superintelligent AI.
The goal of superalignment is to establish foolproof procedures for building and controlling this future technology. OpenAI has committed to allocating one-fifth of its vast computational resources to solve this problem within four years.
"Existing alignment methods won't work for models smarter than humans because they fundamentally assume humans can reliably evaluate what AI systems are doing," Leike explained. "As AI systems become more capable, they will take on more complex tasks. The idea is that humans will find it increasingly difficult to evaluate them. In forming the superalignment team with Ilya, we are tackling these future alignment challenges head-on."
Jeff Dean, Google's Chief Scientist, added: "It's crucial to focus not only on the potential opportunities of large language models but also on their risks and drawbacks."
OpenAI announced this project with great fanfare in July. But for some, it's nothing more than a pipe dream. The blog post OpenAI published on Twitter drew ridicule from prominent critics in the tech world, including Abeba Birhane, who works on AI accountability at Mozilla ("So many grandiose yet hollow phrases in one blog post"); Timnit Gebru, co-founder of the Distributed Artificial Intelligence Research Institute ("Imagine ChatGPT being even more 'superaligned' with OpenAI's engineers. Chilling"); and Margaret Mitchell, chief ethics scientist at AI company HuggingFace ("My alliance is bigger than yours"). Admittedly, these are familiar dissenting voices. But they serve as a powerful reminder that while some see OpenAI as a leader at the forefront, others view it as a leader on the fringe.
For Sutskever, however, alignment is the inevitable next step. "This is an unsolved problem," he says. He believes that core machine learning researchers like himself aren't focusing enough on this issue. "I'm doing this for my own sake. Obviously, it's important that whoever builds superintelligence doesn't betray humanity."
Work on superintelligence has only just begun. Sutskever says it will require extensive reforms at research institutions. Still, he already has a model in mind for the safeguards he hopes to design: an AI that views humans the way parents view their children. "To me, this is the gold standard," he says. "After all, people genuinely care about their children. Does AI have children? No, but I hope it can think that way."
As my conversation with Sutskever was coming to an end, I thought we had finished. But he had another thought—one I hadn't anticipated:
"Once you solve the challenge of runaway AI, what then? In a world with more intelligent AI, is there still room for humans?"
"There's a possibility—one that might seem crazy by today's standards but less so by future standards—that many people will choose to become part of AI. This could be humanity's way of keeping up with the times. At first, only the boldest and most adventurous would try it. Maybe others would follow, or maybe not."
Wait, what? He was getting ready to leave. Would you do it? I asked him. Would you be among the first? The first? I don't know, he said. But it's something I've considered. The real answer is: maybe.
With that, he stood up and walked out of the room. 'Nice to see you again,' he said as he left.