The 'Moat' of AI Chips: An Insurmountable Challenge
-
In the recent drama about OpenAI's future unfolding in Silicon Valley, a subplot involves its CEO Sam Altman's ambitious plan to establish a chip company.
Before being ousted and then reinstated, Altman had sought to raise up to $100 billion from Middle Eastern investors and SoftBank founder Masayoshi Son to create a competitor to industry giants Nvidia and TSMC. This would be an arduous task. For AI chips, even $100 billion might not go very far.
Given that Nvidia and TSMC are crucial to everything related to generative AI, Altman is unlikely to be the only one hoping to compete with them. The moat in the AI chip sector is insurmountable.
Nvidia holds about a 95% share in the graphics processing unit (GPU) market. These computer processors were initially designed for graphics but have become increasingly important in fields like machine learning. TSMC dominates 90% of the global advanced chip market. These businesses are highly profitable, with TSMC's gross margin close to 60% and Nvidia's at 74%. TSMC's annual sales amount to $76 billion. These impressive figures might suggest there's enough room for more competitors.
The global shortage of NVIDIA's artificial intelligence chips has made the prospect of vertical integration more attractive. With the rapid increase in the number of GPUs required to develop and train advanced AI models, the key to profitability for AI companies lies in stable access to GPUs. This explains why global tech giants have been racing to develop streamlined chips optimized for their workflows, such as those for data center servers that train and run large AI language models.
Designing custom chips is one thing. However, NVIDIA's profitability does not come from making chips cost-effective but from providing a one-stop solution for a wide range of tasks and industries. For example, NVIDIA's HGX H100 system, priced at approximately $300,000 per unit, is used to accelerate workloads ranging from financial applications to analytics.
The HGX H100 system consists of 35,000 components. To find a viable competitor for this system, more than just designing a new chip is required. NVIDIA has been developing GPUs for over two decades. This leading advantage includes hardware and related software libraries, protected by thousands of patents.
Even setting aside the challenges of designing new AI chips, manufacturing is the real hurdle. Establishing a wafer fab is the first obstacle. Despite TSMC's more than 30 years of experience in building 'fabs,' it is expected to take over three more years for its U.S. factory under construction in Arizona to begin production. The total investment in this facility is projected to be around $40 billion.
Operating these factories requires a highly skilled workforce with advanced degrees in electrical engineering, physics, or materials science. The shortage of skilled workers has already delayed the opening of factories in Arizona.
Another issue is procuring chip manufacturing equipment. Dutch manufacturer ASML monopolizes the production of extreme ultraviolet (EUV) lithography machines, which are critical for advanced chips. The average waiting time for these machines is about two years, with each machine costing over $300 million.
However, patents remain the biggest hurdle so far. TSMC, one of the world's largest patent holders, owns over 52,000 patents related to chip manufacturing. Among these, about 3,000 involve advanced packaging—a key technology for AI chips that enhances performance, giving TSMC an edge over competitors like Samsung in contract manufacturing. TSMC's eight-plus years of investment in this technology further raises the entry barrier.
For newcomers, navigating these challenges means facing long lead times, which is particularly risky in a fast-evolving industry. Meanwhile, the substantial profits of Nvidia and TSMC allow for more R&D spending, accelerating the pace of next-generation technology releases. TSMC alone spends over $30 billion annually on capital expenditures. Over the past year, as momentum favors Nvidia and TSMC, the gap between them and their competitors has widened. Currently, even their largest rivals lack the capacity to close this gap, let alone new entrants.
NVIDIA's 'Hegemony' in AI Chip Sector Hinders Startup Funding
According to reports, investors indicate that NVIDIA's dominant position in artificial intelligence computer chip manufacturing has chilled venture capital for potential competitors, with the number of US deals in Q3 declining by 80% compared to the previous year.
As NVIDIA grows stronger in the AI chip sector, companies attempting to manufacture competing chips are finding it increasingly difficult. Venture capitalists view these startups as higher-risk bets and have recently become reluctant to provide large funding injections. Developing a chip design into a working prototype can cost over $500 million, making this setback a significant threat to startup prospects.
Greg Reichow, Partner at Eclipse Ventures, stated: 'NVIDIA's continued dominance speaks volumes about how difficult it is to break into this market. This has led to reduced investment in these companies, or at least in many of them.'
According to PitchBook data, US chip startups had raised $881.4 million by the end of August. In comparison, the total investment in the first three quarters of 2022 was $1.79 billion. The number of deals dropped from 23 to just 4 by August.
As reported by The Register, AI chip startup Mythic, which had raised approximately $160 million, nearly ceased operations last year after running out of cash. However, it managed to secure a modest $13 million investment in March. Mythic CEO Dave Rick stated that NVIDIA has "indirectly" worsened the AI chip funding landscape, as investors now seek "big, home-run investments with substantial returns." Rick noted that challenging economic conditions have exacerbated the cyclical semiconductor industry downturn.
Chip startups seeking funding now face stricter investor requirements. Sources indicate that companies must demonstrate a product ready for launch within months or already generating sales. Two years ago, new investments typically ranged between $200-300 million for chip startups. PitchBook analyst Brendan Burke revealed this figure has now dropped to around $100 million.
To secure $100 million in funding this August, Tenstorrent highlighted its CEO Jim Keller—a near-legendary chip architect who previously designed chips for Apple, AMD, and Tesla.
D-Matrix expects this year's revenue to be below $10 million, but with Microsoft's financial support and the Windows manufacturer's commitment to test D-Matrix's new AI chips after their launch next year, D-Matrix has raised $110 million.
While chip manufacturers in Nvidia's shadow face tough conditions, startups in the AI software and related technology sectors aren't subject to the same constraints. According to PitchBook data, they've raised approximately $24 billion in funding as of August this year.