2024: AI Chips Enter the Warring States Era - Three Key Battles in the AI Chip Industry
-
2024 is destined to be a year of fierce competition among AI chip manufacturers.
Over the past year, the AI sector has been booming, with GPU manufacturers like NVIDIA holding the computational power advantage. Backed by AI companies expanding cloud computing scale and the demand for training large models, they have reaped the first wave of dividends in the AI era, earning substantial profits.
As we enter 2024, AI is spreading from the cloud to the edge, and many new concepts under "AI + Products" are gradually reaching the tipping point for consumer adoption. To capture these growth opportunities, leading AI chip companies across various specialized fields have begun cross-industry collaborations, launching new products while seeking new partners. Even in the relatively stable cloud computing market, there is a sense of impending change. Google, Microsoft, and Amazon have all announced or planned to launch their own AI chips. AMD's new MI300X closely competes with NVIDIA's H100 in performance, and Intel CEO Pat Gelsinger revealed, "The entire industry is being driven to reduce the market share of CUDA (NVIDIA's computing platform)."
What changes will the AI chip market face in 2024? And who will reap the immense benefits?
From cloud demand and corporate actions, developing next-generation large models will remain the core focus of competition among major tech companies in 2024. The increasing proportion of multimodal models is driving even greater demand for computing power. The rise in C-end AI applications is injecting stability into the cloud market. Previously, model pre-training accounted for the largest share of GPU consumption, around 80%. However, as more consumers begin to utilize models for inference, the future will see a reversal in the ratio of inference to training compute power. According to a report by LatePost, multiple industry experts estimate that if trillion-parameter or larger AI models become widely adopted, the cost ratio of training to inference could reach 2:8 or even 1:9.
For AI entrepreneurs and tech giants, this news is a mixed blessing.
In 2023, all those rushing into AI have complex feelings toward upstream chip suppliers. Developing AI products relies heavily on AI chips as the computational heart, but the high prices and limited production capacity also leave them perpetually surrounded by anxiety. Beyond the fear of competitors gaining an edge by acquiring chips and expanding their market presence, there's a deeper concern that the seller's market structure will persist, making it impossible to escape the fate of 'paying taxes' upstream.
Surprisingly, even those profiting handsomely as 'water sellers' in this market feel uneasy. NVIDIA's founder Jensen Huang remarked in a speech last November: "We don't need to pretend that the company is always in danger. We (truly) are always in danger."
Having stood at the eye of the AI storm for a year, Huang's statement likely reflects his genuine sentiment. The pressure first comes from peer competition.
As the only company in the market with the potential to challenge Nvidia in the GPU sector, AMD's every new chip release inevitably draws comparisons with Nvidia. You could even see sparks flying during their multiple rounds of exchanges last December.
When AMD launched the MI300x on December 6, it claimed superior performance over Nvidia's AH100. Nvidia responded by releasing its own benchmark tests, showing that the AH100 still performed better under the right settings. AMD then countered again, publishing new benchmarks that once more demonstrated the MI300X's superior performance under the correct configurations. Additionally, although Intel has rarely made significant achievements in the GPU field, it has not given up on this lucrative market. Intel claims that its Gaudi 3, set to launch in 2024, will outperform Nvidia's AI chip H100.
Apart from competitors, Nvidia's clients are also proving disloyal. Almost all major customers—Google, Amazon, Tesla, Alibaba, Baidu, Microsoft—have announced plans to develop their own AI chips.
These clients are also launching fierce attacks on Nvidia's ecosystem moat. "The entire industry is being driven to reduce CUDA's market share," Intel CEO Pat Gelsinger recently stated, noting that MLIR, Google, OpenAI, and others are shifting toward a "Python-based programming layer" to make AI training more open. Developing proprietary chips and ecosystems is no easy feat, so why has it become an industry consensus? NVIDIA's 'scythe' is simply too sharp.
A report from Caixin revealed that even if tech companies spend the same amount as NVIDIA but achieve only one-tenth of the performance, they can still turn a profit. A concrete example is Google's latest AI chip, TPUv5e, which offers lower costs than using NVIDIA's A100 or H100 for training and inferencing models with fewer than 200 billion parameters.
NVIDIA's aggressive tactics to poach customers have also been hard for cloud providers to swallow. In March last year, during the peak GPU shortage, NVIDIA launched its cloud computing service, DGX Cloud. Essentially, it repurchases GPUs sold to cloud providers, optimizes them, and then rents them out to clients needing GPU computing power. This leaves cloud providers bearing the cost of data center construction while losing customers to NVIDIA. For Nvidia, the biggest variable may still be in China. A recent foreign media report revealed that as the performance of chips exported to China continues to be restricted, Chinese players are losing interest, which could mean losing nearly a fifth of its revenue.
Of course, as one of Silicon Valley's most combative and stubborn companies, sitting idle has never been Nvidia's style. Over the past year, with its position as the largest GPU seller holding nearly 95% of the market share, Nvidia's moat has actually widened.
Nvidia's core competitiveness is primarily composed of three pillars—supply chain advantages, software and hardware ecosystems, and investments. In terms of hardware and software ecosystems, the AI chip H200 and its accompanying ecosystem accessories expected to ship in 2024 will likely remain the most suitable 'soil' for large model growth in the market. On the investment front, NVIDIA completed over 20 investments in 2023, covering various AI-related industries. Even if stable profits aren't guaranteed from these investments, the accumulated know-how will gradually become a compelling reason for other startups to choose NVIDIA.
What truly determines the 2024 market landscape is NVIDIA's supply chain advantage.
The AI chip industry chain is lengthy and complex, with leading enterprises in each specialized segment. Only by integrating them can the highest-performance chips be produced. Nvidia's approach to securing their cooperation is by promising non-cancelable orders. Currently, Nvidia has $11.15 billion in purchase commitments, capacity obligations, and inventory obligations, along with an additional $3.81 billion in prepaid supply agreements. These committed orders include nearly 60% of TSMC's production capacity and the majority of HBM supplies from SK Hynix, Samsung, and Micron.
The impact of TSMC's packaging on chip performance goes without saying, and HBM is equally crucial. HBM (High Bandwidth Memory) chips are essential because GPUs can only maintain operational efficiency when they have sufficient memory capacity and fast data transfer speeds.
From this perspective alone, no other suppliers can match this level of supply chain control, which means they won't be able to participate in the ongoing AI frenzy – not even tech giants like Google. In summary, the cloud computing battle in 2024 is highly certain. Although NVIDIA is facing encirclement, both self-developed chips and ecosystem building require time to mature. At least until the supply chain issues are resolved, NVIDIA's dominant position will remain unshaken.
The gold rush enriches the shovel sellers first, and behind the scenes, AI chip manufacturers are already rolling up their sleeves. Breaking it down by industry, AI PC, AI Phone, and AI Car are the three most closely watched sectors.
Over the past few years, the smartphone market has struggled with growth stagnation. Even as manufacturers push hardware to its limits, they cannot mask the harsh reality of innovation drought. Apart from Huawei's Mate 60 injecting some fresh vitality into the market, even Apple's product launches have failed to tell compelling new stories that convince consumers to upgrade their devices. AI is the only driving force that can spur growth. The legend of "AI on devices" has been floating in the market since ChatGPT became available to consumers, discussed throughout the year. However, before large models are scaled down, the contradiction between hardware and model size is almost irreconcilable.
In fact, on mobile devices, it remains difficult to ensure model performance while avoiding issues like overheating and excessive memory usage, which negatively impact consumers' purchasing decisions.
At the beginning of the year, Qualcomm, one of the leading mobile chip manufacturers, demonstrated running Stable Diffusion on Android and recently announced reducing inference time to under one second. However, engineers from MediaTek, another major player, pointed out that running a small 13B model locally requires approximately 13GB of memory. Combined with Android's own 4GB usage, this already exceeds the 16GB memory capacity of most smartphones—even without downloading any additional apps. Even so, the market has cast a vote of confidence in the growth of the AI Phone sector. The rationale is that smartphones are the most frequently used electronic devices in daily life. Despite certain difficulties in local deployment, the incremental benefits brought by cloud-based AI operations cannot be overlooked. Analysis firm Canalys predicts that global smartphone shipments will recover in 2024, with an estimated growth of about 4%.
In contrast, the progress of AI PCs has been much smoother. The reason is simple: compared to smartphones, PCs have larger form factors and greater potential for chip performance improvements. IDC forecasts that 2024 will be the first year of rapid development for AI PCs, with AI PCs accounting for 55% of the overall PC market in 2024, rising to 85% by 2027.
Currently, AI chip manufacturers in the PC sector have already shown intense competition, with a fierce battle imminent among them. The competition between NPU and GPU marks the first crucial point in the battle.
GPUs need no introduction, but the question is why NPUs have entered the ring. Unlike computing centers that rely on thousands of servers integrating various chips like CPUs and GPUs, a PC's computing power is concentrated in a single "master chip." This chip consists of sub-chips such as the CPU, GPU, and NPU, each with distinct roles. The NPU, in particular, is specifically designed to handle AI-related computing tasks.
This creates a conflict. Although GPUs are defined as graphics processors, their floating-point and parallel computing capabilities also make them one of the best platforms for running AI computations, as proven in cloud computing. If AI PCs become a reality, the surge in AI computing tasks will inevitably drive GPU sales. Ahead of this year's CES, NVIDIA released three consumer-grade GPUs in a row, clearly sending a signal to the market. The competition between X86 and ARM architectures is the second critical point in this industry battle.
X86 and ARM are two distinct chip architectures. The X86 architecture is synonymous with Intel, while in the PC chip sector, Intel's direct competitors are Qualcomm and MediaTek, leading mobile chip manufacturers that adopt the ARM architecture.
The differences between X86 and ARM can be simply summarized as follows: the former offers high performance and high power consumption, while the latter provides lower performance and lower power consumption. Traditionally, ARM has been primarily used in mobile devices. However, in recent years, as the performance of ARM chips continues to improve, their low-power advantage has caught the attention of industry players. Companies like Microsoft, Apple, and Meta are now supporting ARM-based PCs. According to reports, NVIDIA and AMD are considering launching ARM-based solutions for mobile PCs in 2024. Qualcomm has long coveted the AI PC market. Last October, the Snapdragon X Elite chip released for Windows 11 demonstrated performance twice that of comparable X86 chips while consuming only one-third the power at peak performance.
Although the AI PC battlefield is already fiercely contested, it pales in comparison to the AI Car sector, which could even be described as harmonious. The reason behind this is clear: PCs and smartphones are searching for growth in a saturated market, while automotive AI represents a genuine blue ocean opportunity.
"AI in vehicles" also has two key battlegrounds: smart cockpits and autonomous driving. In the smart cockpit domain, Qualcomm's early adoption of SoC logic design for automotive MCU chips has given it a first-mover advantage, positioning it as the incumbent defender. Currently, it faces multi-pronged competition from companies like Nvidia, MediaTek, and Intel.
In June last year, MediaTek announced a collaboration with Nvidia to provide a full suite of AI-powered cockpit solutions for next-generation software-defined vehicles, covering all automotive segments from luxury to mainstream.
The collaboration model between these two powerhouses involves MediaTek developing automotive SoCs integrated with Nvidia GPUs. Nvidia, currently a major supplier of autonomous driving systems for high-end models, boasts extensive customer resources. At this year's CES, Nvidia unveiled the latest advancements in its DRIVE series automotive business. Companies like Li Auto, Great Wall, Zeekr, and Xiaomi have already adopted the DRIVE Orin platform to support their intelligent autonomous driving systems. Intel announced its significant automotive industry strategy at CES, introducing AI-enhanced software-defined vehicle system-on-chips (SoCs) specifically designed for next-generation automobiles. Its subsidiary Mobileye, a leader in autonomous driving solutions, maintains global technological leadership.
In the autonomous driving sector, a duopoly has emerged. NVIDIA and Horizon Robotics dominate the high-end and low-end vehicle segments respectively, with Horizon ranking first in terms of price range coverage breadth, followed by NVIDIA.
Similar to NVIDIA's CUDA ecosystem, Horizon Robotics has built its own technological moat. As a local supplier, Horizon also holds advantages in delivery efficiency. Chip manufacturers are fiercely competing in AI PC, AI Phone, and AI Car markets, pushing the boundaries in this new era of competition reminiscent of the 'Warring States Period.' Even though Nvidia still holds a significant lead, critical questions emerge: Who will break through in 2024? And who is the most crucial variable besides Nvidia?
Undoubtedly, this will be a battle of industry dynamics. We can make some predictions about who might emerge victorious in 2024 based on subtle clues.
While all three major scenarios—AI PC, AI Phone, and AI Car—are evolving, the absolute value of their growth varies. Considering factors like hardware performance, user demand, and market size, 'Silicon Research Lab' concludes that AI Car will see the highest growth, followed by AI Phone, with AI PC trailing last. Automotive AI chips may be the most uncertain battlefield in 2024.
The market strategies of major players can be broadly categorized into two approaches. One starts with autonomous driving and expands into smart cockpits, exemplified by NVIDIA and Mobileye (via Intel). The other begins with smart cockpits and extends into autonomous driving, as seen with Qualcomm.
In the smart cockpit domain, the competition among new energy vehicle manufacturers is less about chip performance and more about understanding real-world usage scenarios. In autonomous driving, however, safety is paramount, requiring stable hardware operation and market validation, which creates higher barriers for leading players. From this perspective, NVIDIA and Mobileye will benefit first, while second-tier manufacturers like Horizon may become automakers' alternatives with their low-cost and mature products.
However, in terms of delivery stability and market expansion, domestic players like Horizon and Huawei demonstrate stronger potential.
On one hand, against the backdrop of geopolitical tensions, the sustainability and stability of AI chip exports to China remain questionable. While performance and ecosystem are indeed barriers, they are currently being addressed. On the other hand, although Horizon is currently focusing on low-end vehicle models, the development of autonomous driving heavily relies on data accumulation. It is not impossible for the company to overtake more advanced manufacturers at a faster pace in the future.
In the mobile sector, the prerequisite for AI chip manufacturers to reap benefits is the local deployment of large models.
As is well known, there are two modes for deploying large models on devices: one is cloud access, and the other is terminal deployment. The former relies on sending user requests to the cloud for computation before returning the results to the device. The latter completes the entire process using local hardware. In terms of functionality, both are similar. The primary factor influencing consumers' purchasing decisions is their willingness to upload local data (including photos, files, and chat records) to the cloud. From this perspective, there is indeed a demand for local deployment, and the growth of smartphone AI chips is worth anticipating.
So, who will take the lead between Qualcomm and MediaTek?
From a market positioning standpoint, Qualcomm and MediaTek dominate the high-end and mid-to-low-end smartphone markets, respectively. Breaking into higher segments is far more challenging than downward compatibility, and currently, manufacturers launching AI smartphones are pricing them in the premium segment. Qualcomm's premium market positioning has led to significantly higher gross margins. According to financial reports, Qualcomm's gross margin in the third quarter was approximately 55.1%, while MediaTek's stood at 47.4%. This means Qualcomm has more "disposable income" to drive research and development. Under the snowball effect, MediaTek still has a long way to go to catch up with Qualcomm.
It's worth noting that Apple and Huawei's AI chip plans could also impact the market landscape, though to a lesser extent.
In recent years, the performance improvements of Apple's mobile chips have not been advantageous compared to competitors like Qualcomm and MediaTek. The two latest Android chips—Qualcomm's Snapdragon 8 Gen 3 and MediaTek's Dimensity 9300—have already surpassed Apple's A17 Pro in key technical metrics such as CPU multi-core performance, CPU multi-core energy efficiency, GPU peak performance, and GPU energy efficiency. For Huawei, affected by geopolitical factors, it is currently difficult to compete with top international chip manufacturers in terms of chip performance.
On the PC front, although AI PCs have been making a lot of noise recently, it will still take some time before they can truly drive sales in the consumer market.
From a consumer perspective, AI applications can be broadly categorized into two main types: productivity and entertainment. In office scenarios, although software like Microsoft, Kingsoft Office, Midjourney, and Adobe have integrated AI functionalities, their inference tasks are primarily handled by the cloud. These services have already established a pay-per-use business model, eliminating the need for high-performance local hardware.
In entertainment scenarios, even before the AI boom, there was already a high demand for GPU-based graphics processing power to ensure smooth gaming experiences. Given the current scale of models that can be 'packed' into PCs, GPUs previously used for rendering game graphics now have ample computing power to support AI-driven applications.
Taking NVIDIA as an example, although they released three new consumer-grade GPUs before the CES opening, none of them surpass the performance of the RTX 4090 released in 2022. In summary, in 2024, NVIDIA will continue to maintain its position as the most valuable 'water seller' in traditional fields like cloud computing, thanks to its supply chain advantages and other moats, and this status is unlikely to change in the short term.
In the three major scenarios of AI PC, AI Phone, and AI Car, NVIDIA and Horizon, which have advantages in technology and channels respectively, will be the first to benefit from the AI Car sector. Meanwhile, Qualcomm is expected to break through in the AI Phone category, which urgently needs new growth drivers. As for AI PC, the market share and sales of related companies remain to be seen until new scenarios and work paradigms emerge, driving stronger hardware demand.
2024 is a year of immense wealth for AI chip manufacturers, but at the same time, intense changes are brewing beneath the surface. Companies and investors can certainly dive into this blue ocean, but it's best to remain vigilant at all times.