Which Company Has the World's Fastest AI Chip?
-
Recently, IBM quietly announced the launch of a brain-inspired chip called 'NorthPole.' Compared to Nvidia's H100 GPU at the 4nm node, NorthPole achieves five times the energy efficiency, solidifying its position as the world's most powerful AI chip.
Despite its groundbreaking performance, there has been surprisingly little coverage of this chip in China. So, what exactly is it?
First, IBM's NorthPole is a brain-inspired chip, and to understand it, we need to grasp what brain-inspired chips are.
Human thinking differs significantly from traditional chip architectures. For example, humans don't have separate memory units, dynamic random-access memory (DRAM), hash hierarchies, or shared memory. Instead, 'storage' and 'processing' are intricately intertwined in the human brain, which contains 'neurons.' In computers, digital cores exchange event-based information through 'spikes,' resembling how the human brain transmits information.
Inspired by the human brain, scientists have developed such chips to enhance performance, creating true 'artificial intelligence' that can even approach human-like thinking capabilities.
Of course, the human brain is highly complex, and the IC devices we manufacture also have their advantages. The ultimate goal is to merge the strengths of both.
In simple terms, a brain-inspired chip is designed to combine the advantages of biological brains and artificial devices, enabling it to think and learn like a human.
Creating such a chip isn't just about tweaking device structures; it requires advancements in materials, devices, circuits, architectures, algorithms, and applications. In short, it demands the integration of cutting-edge technologies.
While there are many approaches, the technical challenges are immense. Even designing such a chip is difficult, let alone manufacturing it, so the field is still in its exploratory phase and far from commercialization.
According to the author, the lack of large-scale commercialization of brain-inspired chips is due to two main reasons: First, design challenges persist. For instance, even CMOS-type chips, favored by Intel and IBM, face issues like interconnecting multiple fully digital asynchronous chips, ensuring connectivity efficiency and timeliness, and addressing software-layer interconnectivity, distributed computing, and flexible partitioning. Second, manufacturing, software, and ecosystems must be completely overhauled. While silicon-based transistor technology can be partially reused, the underlying architecture cannot be copied entirely, further complicating large-scale commercialization.
So, why go to such lengths for brain-inspired chips?
Brain-inspired chips are incredibly promising. The author notes that, in some cases, an ideal neuromorphic chip can solve problems with 1,000 times lower energy consumption than traditional solutions. This means we can pack more chips into a fixed power budget to tackle larger-scale problems.
The ultra-high energy efficiency of brain-inspired chips makes them worth the effort. Take AlphaGo as an example: it defeated humans at Go, but while the human brain consumes only 20 watts, AlphaGo required 20,000 watts.
Fields and opportunities for neuromorphic chips. Source: Nature Computational Science
Structurally, brain-inspired chips worldwide generally consist of three components: neuronal computation, synaptic weight storage, and routing communication, all using the Spiking Neural Network (SNN) model.
Based on materials, devices, and circuits, they are divided into three categories: analog-circuit-dominated neuromorphic systems (mixed-signal CMOS), fully digital-circuit neuromorphic systems (digital CMOS), and mixed-signal neuromorphic systems based on novel devices (with memristors as a candidate technology).
Globally, organizations developing neuromorphic computing chips fall into three groups: tech giants like Intel, IBM, and Qualcomm; universities/research institutions like Stanford and Tsinghua; and startups.
From discussions with Intel Research, the author learned that digital CMOS is currently the most viable for industrialization due to its mature technology and manufacturing processes, as well as fewer concerns and limitations compared to analog circuits.
However, it's important to note that digital CMOS is only the most basic form of brain-inspired chips, not fully mimicking the human brain's neuromorphic devices. Still, even this basic form outperforms any other chip in the world. IBM's NorthPole is such a digital CMOS brain-inspired chip.
Major types and development progress of brain-inspired chips. Source: Electronic Engineering World
In conclusion, NorthPole is arguably the most advanced brain-inspired chip to date, marking a significant milestone in brain-inspired computing research.
IBM began researching brain-inspired computing in 2008 and achieved its first breakthrough in 2011 with the first-generation neurosynaptic computer chip. Researchers created two prototypes: one with 262,144 programmable synapses and another with 65,536 learnable synapses, demonstrating capabilities like navigation, machine vision, pattern recognition, associative memory, and classification.
In 2014, IBM published a paper in Science showcasing a groundbreaking achievement: a brain-inspired processor with 1 million neurons, TrueNorth, meeting DARPA SyNAPSE project metrics. This sparked a wave of brain-inspired chip research.
After eight years of silence, NorthPole emerged, building on TrueNorth's foundation. TrueNorth already offered four orders of magnitude better energy efficiency than traditional microprocessors, and comparing the two highlights IBM's technological evolution.
Structurally, NorthPole, like TrueNorth, consists of a large array of computing units (16×16), each with local memory and code execution capabilities.
For memory, all storage is integrated into a single chip, allowing each core easy access. Externally, NorthPole resembles an active memory chip, simplifying system integration.
Moreover, NorthPole is currently manufactured using a 12nm node process. While the most advanced CPU technology is 3nm, IBM is developing 2nm technology, which could further enhance NorthPole's performance.
NorthPole's potential applications include image and video analysis, speech recognition, and Transformer neural networks, which power large language models (LLMs) like ChatGPT. These AI tasks could be used in autonomous vehicles, robotics, digital assistants, and satellite observations.
Some applications require neural networks too large for a single NorthPole chip. In such cases, the networks can be split into smaller parts and distributed across multiple NorthPole chips.
NorthPole's ultra-high energy efficiency means it doesn't need bulky liquid cooling systems—fans and heat sinks suffice—and it can be deployed in smaller spaces.
China has also been actively researching brain-inspired chips. Leading institutions include Tsinghua University, Zhejiang University, Fudan University, and the Chinese Academy of Sciences, alongside startups like Lingxi Technology, SynSense, and CAS Neuromorphic. Among these, Tsinghua's Tianjic and Zhejiang University's Darwin chips are the most notable.
Specifically, the main domestic achievements include:
IBM's progress signifies that this future-oriented technology is getting closer to us, while some startups are gradually forming solutions and beginning to apply them. It is not hard to foresee that in the coming years, the commercialization of this technology will gradually unfold, and by then, research outcomes will become a reality.