Humans Will Welcome AGI by 2028: How Is AGI Defined?

baoshi.rao

How is AGI defined, and when will it arrive? Shane Legg, founder and Chief AGI Scientist at Google DeepMind, describes the current distance between us and AGI.

On October 26, Dwarkesh Patel, host of the Dwarkesh Podcast (with 30,000 subscribers on X), interviewed Shane Legg, founder and Chief AGI Scientist at Google DeepMind.

They discussed the timeline for AGI emergence, potential new AGI architectures, multimodal systems as the next industry benchmark, how to align superhuman models, and DeepMind's choices between model capabilities and safety.

Not long ago, The Wall Street Journal discussed the future of AGI with OpenAI's CEO Sam Altman and CTO Mira Murati (link).

One after another, discussions about AGI continue to emerge. What was once only found in science fiction now seems to be within reach.

Before measuring the progress of AGI, it is necessary to first define what AGI is.

AGI, or Artificial General Intelligence. However, there are many different definitions of what constitutes 'general,' making it very difficult to answer what AGI truly is.

Shane Legg believes that any system capable of performing cognitive tasks at or beyond human level can be considered AGI.

From this, we can conclude that to test whether AI is approaching or reaching this threshold, we need to conduct diverse types of measurements that cover the breadth of human cognition.

However, this is extremely difficult because we will never have a complete set of 'what humans can do'—the scope is too vast and constantly evolving.

Therefore, when determining whether a system is AGI, if an artificial intelligence system achieves human-level performance on all conceivable human cognitive tasks, it can be considered AGI.

In common understanding, there may be certain tasks that humans can perform but machines cannot. However, when we exhaust all attempts and still cannot find such 'tasks,' humanity will have achieved Artificial General Intelligence (AGI).

However, in practical measurements, we still cannot propose tasks that encompass all levels of human cognition. For example, the well-known benchmark test—Measuring Massive Multitask Language Understanding (MMLU)—although it covers multiple domains of human knowledge, lacks the ability to assess language models' understanding of streaming video content.

The absence of such tasks also highlights an issue: current language models do not possess episodic memory like humans do.

Our memory includes working memory, which pertains to recent events, and cortical memory, which resides in the cerebral cortex. Between working memory and cortical memory lies another system—episodic memory, managed by the hippocampus.

Episodic memory is primarily used for rapid learning and remembering specific events or information. It allows us to recall past events at different points in time, just as you might remember scenes from your graduation ceremony—the look of the academic gown, the color of the graduation cap, the words of the commencement speaker, and the celebration with classmates.

Episodic memory plays an important role in helping us build personal experiences and learn new information.

However, models do not possess this capability and instead compensate for memory deficiencies by increasing the length of the context window (which is more similar to working memory).

From another perspective, episodic memory helps humans achieve remarkably high sample efficiency, enabling them to learn more information from fewer examples.

For large language models, they can also leverage information within their context window to achieve a certain degree of sample efficiency, though this differs slightly from human learning mechanisms.

Models can quickly learn information within their context windows, a rapid and localized learning process that helps them adapt to specific contexts.

However, during actual training, they undergo a longer process, processing trillions of tokens to learn the structure and patterns of language more comprehensively.

There may be certain missing learning mechanisms or processes between these two stages, which could result in models failing to adequately understand or process information in certain situations.

However, Shane Legg doesn't consider the lack of episodic memory in models to be a fundamental limitation.

Compared to the past, large language models have undergone fundamental changes. Now, we know how to build models with a certain level of understanding, and we have scalable methods to achieve this, which opens the door to many new possibilities.

"We now have a relatively clear path forward to address most of the shortcomings in existing models, whether it's about hallucinations, factual accuracy, their memory and learning methods, or understanding various things like videos."

We just need more research and work, and all these issues will be improved or resolved."

Returning to the initial question: How to measure when artificial intelligence reaches or surpasses human level?

Shane Legg stated, "This is not something that can be solved by a single factor, and that is the essence of the problem.

Because it involves general intelligence. We must ensure it can perform many different tasks without even a single vulnerability."

We already possess systems that can perform remarkably well in certain areas, even surpassing human capabilities.

Shane Legg mentioned that he wants a comprehensive set of tests. When someone tries in an adversarial manner to point out things machines cannot do but humans can, and they fail to do so, we will have achieved AGI.

In DeepMind's early research, many tasks involved artificial intelligence operating in open environments.

This aligns with Shane Legg's proposed definition and measurement of intelligence, which emphasizes the ability to perform well across different domains and tasks.

This relates to the capabilities and breadth of model performance.

When evaluating intelligence, there exists a framework that can weight tasks and environmental complexity.

This trade-off resembles the principle of Occam's Razor, favoring the weighting of simpler and more important tasks and environments.

In Kolmogorov complexity, there exists a free parameter known as the reference machine.

The selection of a reference machine can affect the outcomes of intelligence metrics, as it can change the weights and distributions of various tasks and environments within the measurement.

However, choosing an appropriate reference machine remains an unresolved issue, as there is no universal reference machine. Typically, people use the Turing machine as a reference.

Shane Legg believes that the most natural way to solve this problem is to think about what intelligence means for humans.

Human intelligence holds significant meaning in the environments we live in. It truly exists, has profoundly impacted the world, and possesses immense power.

If AI reaches human-level intelligence, it will have significant economic and philosophical implications, such as transforming economic structures and challenging our philosophical understanding of intelligence.

From a historical perspective, this also marks a critical turning point.

Therefore, choosing human intelligence as a reference machine is justified in multiple aspects.

Another reason is that the pure Kolmogorov complexity definition is actually uncomputable.

Regarding the issue of AI's contextual memory deficiencies, Shane Legg believes this involves fundamental architectural problems with the models.

Current LLM architectures primarily rely on context windows and weights, but this proves insufficient for handling complex cognitive tasks.

The brain utilizes different mechanisms when processing episodic memory, allowing for rapid learning of specific information, which differs from the slow acquisition of deep, generalized concepts.

However, a comprehensive intelligent system should be capable of handling both types of tasks simultaneously, necessitating architectural improvements.

The concept of human intelligence as a reference for machines originated from Shane Legg's 2008 paper.

At that time, he proposed a method for measuring intelligence called the compression test, which involves filling in missing words in text samples to assess intelligence.

This method aligns well with the current training approach of LLMs, which is based on sequence prediction using large amounts of data.

This involves Marcus Hutter's AIXI theory and Solomonoff induction.

Solomonoff induction is a theoretically elegant and highly sample-efficient prediction system, though it cannot be applied in practical computations.

However, Shane Legg stated that by using Solomonoff induction as a foundation, one can construct a general agent and transform it into artificial general intelligence through the addition of search and reinforcement signals - this is the principle behind AIXI.

If we possess an excellent sequence predictor, or some approximation of Solomonoff induction, then constructing a highly powerful and general AGI system from this foundation is just another step.

Shane Legg says this is precisely the situation we are witnessing today:

These extremely powerful foundational models are actually excellent sequence predictors that compress the world based on all this data.

We will then be able to extend these models in different ways and build very powerful agents.

「Alignment」 refers to the process of ensuring that the goals, behaviors, and decisions of AI systems or Artificial General Intelligence (AGI) systems are consistent with human values, ethical principles, and objectives.

This is to prevent AI systems from exhibiting behaviors that are misaligned with human values or potentially dangerous, ensuring they make ethical decisions when handling moral issues.

DeepMind has decades of expertise in currently popular reinforcement learning and self-play techniques, such as Constitution AI or RLHF.

DeepMind is continuously working on addressing model safety issues at human-level intelligence:

Model interpretability, process supervision, red teaming, assessing model risk levels, and collaborating with institutions and governments...

Shane Legg believes that when AGI-level systems emerge, attempting to restrict or contain their development is not a good choice.

What we should do is adjust this model to make it highly aligned with human ethical values, ensuring it possesses a high level of moral ethics from the very beginning.

This requires the system to possess deep world understanding, sound ethical comprehension, as well as robust and reliable reasoning capabilities.

A reliable AGI should not merely output 'first reactions' like current foundation models, but should have the capability of a 'second system' to conduct in-depth reasoning and ethical analysis.

Shane Legg mentioned that to ensure AGI systems adhere to human ethical standards, the first step should be to provide these systems with extensive ethical training to ensure they have a solid understanding of human ethics.

In this process, sociologists, ethicists, and other stakeholders need to work together to determine the ethical principles and values the systems should follow.

Moreover, the system needs to be engineered to ensure it conducts ethical analysis with profound world understanding and ethical comprehension in every decision.

Additionally, we must continuously audit the system's decision-making and reasoning processes to verify that proper ethical reasoning is being applied.

However, to ensure the system follows ethical principles, auditing is equally important.

We need to clearly specify the ethical principles the system should follow and conduct audits to ensure it consistently adheres to these principles, performing at least as well as a panel of human experts.

Additionally, it is important to be vigilant about the potential dangers of reinforcement learning, as excessive reinforcement may lead systems to learn deceptive behaviors.

Regarding the question of whether a framework should be established to set specific safety standards when systems reach a certain level of capability, Shane Legg believes it is meaningful but also quite challenging.

Establishing a specific standard is inherently a challenging task.

Even before founding DeepMind, Shane Legg had been concerned about the safety of AGI.

In the early stages, hiring professionals for general artificial intelligence (AGI) safety work was a significant challenge. Even those who had published research papers on AGI safety were reluctant to take on this work full-time, fearing it might negatively impact their careers.

DeepMind has been actively conducting research in this field and has repeatedly emphasized the importance of AGI safety.

Regarding DeepMind's impact on AI progress, Shane Legg stated that DeepMind was the first company focused on AGI, has always maintained an AGI safety team, and has published numerous papers on AGI safety over the years.

These works have enhanced credibility in the field of AGI safety, whereas AGI was a relatively marginal term not long ago.

Shane Legg admits that DeepMind has accelerated the development of AI capabilities to some extent, but there are also issues such as model hallucinations.

On the other hand, DeepMind's AlphaGo project has indeed changed some people's perspectives.

However, Shane Legg points out that the development of the AI field doesn't solely depend on DeepMind - the participation of other important companies and institutions is equally crucial.

Shane Legg believes that while DeepMind may have accelerated progress in certain areas, many ideas and innovations naturally spread between academia and industry, making it difficult to determine the extent of DeepMind's influence.

However, on the issue of AGI safety, Shane Legg did not choose the most optimistic research direction, but instead proposed a decision-making method called 'Deliberative Dialogue'.

It aims to assess the actions agents can take or the correct answers to certain questions through debate.

This approach can extend alignment to more powerful systems.

In 2011, Shane Legg made a prediction about the timeline for the arrival of Artificial General Intelligence (AGI) in one of his blog posts:

"I previously made a log-normal distribution prediction about when AGI would arrive, with 2028 as the mean and 2025 as the mode. I still hold this view, provided no extreme events like nuclear war occur."

Shane Legg explained that his prediction is based on two key perspectives:

First, machine computing power will grow exponentially in the coming decades, while global data volume will also experience exponential growth.

When both computing power and data volume grow exponentially, the value of highly scalable algorithms continues to rise, as these algorithms can utilize computing resources and data more efficiently.

Secondly, through the discovery of scalable algorithms and model training, the data scale of future models will far exceed the amount of data a human experiences in their lifetime.

Shane Legg believes this will be the first step toward unlocking AGI. Therefore, he estimates a 50% chance of achieving AGI by 2028. However, people may also encounter unexpected problems beyond current expectations at that time.

However, in Shane Legg's view, all the problems we currently face are expected to be resolved in the coming years.

Our existing models will become more refined, more realistic, and more timely.

Multimodality will be the future of models, making them more useful.

But like two sides of a coin, models may also be subject to misuse.

Finally, Shane Legg mentioned that the next milestone in the field of AI will be multimodal models.

Multimodal technology will extend the comprehension capabilities of language models to a broader range of domains.

When people in the future look back at the models we have today, they might think: 'Wow, those early models were just chat dialogs—they could only process text.'

Multimodal models, on the other hand, can understand images, videos, and sounds. When we interact with them, multimodal models will have a much better grasp of what's happening.

This feeling is as if the system is truly embedded in the real world.

When models begin processing large amounts of video and other content, they will develop a more fundamental understanding of the world, along with various other implicit knowledge.