Representative image of a Nvidia chip.
Ni Tao is IE’s columnist, giving exclusive insight into China’s technology and engineering ecosystem. His Inside China column explores the issues that shape discussions and understanding about Chinese innovation, providing fresh perspectives not found elsewhere.
Huawei’s recent release of its latest GPU (graphics processing unit) has been the talk of the town inside China, with its impact rippling across the global tech landscape.
The South China Morning Post reported that several large Chinese companies and internet firms have received samples of Huawei’s Ascend 910C chip for testing.
This new chip, an upgraded version of the 910B, showcases the Chinese powerhouse’s intensified push into the GPU market. Along with other emerging domestic chipmakers, Huawei is making bold attempts to challenge NVIDIA, which currently holds some 88 percent of the global GPU market.
Within China, AI chipmaking is critical to the country’s high-stakes maneuvers to break free from a US-led chip “chokehold.” At the heart of this struggle is GPUs’ growing importance, mainly supplied by a few companies like NVIDIA and AMD.
Their products have become the backbone of AI, supercomputing, robotics, and autonomous driving, among other technologies.
NVIDIA’s GPUs have been a huge accelerator of the global AI race, thanks to their features that enable massive parallel-computing tasks.
Unlike traditional CPUs, GPUs are far better suited for the repetitive and complex computations that AI requires, particularly in deep learning and neural network. As a result, they have become indispensable to data center operators and companies building large-scale AI models.
Recognizing the crucial role of GPU in fulfilling its AI ambitions, China has invested heavily in its own GPU technology over the past few years. The results so far have been less than satisfactory despite the lofty sums of state financial support and policy incentives.
A wave of startups has entered this field in recent years. Developing GPUs is a cash-burning business. A few of them, such as Cambricon, addressed their funding woes by going public, and a dozen others are in the IPO pipeline.
Flourishing as the industry might be, the broader Chinese market remains heavily dependent on NVIDIA, which grossed an astonishing 80.6 billion yuan ($11.5 billion) in revenue from China in 2023 alone.
Despite the best efforts by Chinese companies, they remain stuck in a cycle of playing catch-up. Whatever earnings they reported were a fraction of NVIDIA’s. The reason for this, according to experts, is the lack of practical applications, innovative iterations, and commercial prospects.
Chinese media reported that some domestic supercomputing centers and large language model (LLM) developers, currently the big patrons of advanced AI chips, promised to purchase homegrown alternatives to NVIDIA. In reality, many haven’t followed through, instead turning to NVIDIA’s offerings.
They cannot be blamed for making this choice. Buying NVIDIA GPUs doesn’t just mean acquiring the computing power necessary for AI training. It’s also a vote of confidence in its world-famous CUDA software and GPU-to-GPU communication, which are the backbone of NVIDIA’s dominance.
One of its trump cards is the CUDA architecture, which serves as a parallel computing platform and application programming interface for developers. Without CUDA, programmers would have been clueless about performing accelerated general-purpose processing on GPUs.
According to Andrew Ng, former Chief Scientist at Chinese tech giant Baidu, fewer than 100 developers globally could program for GPUs before CUDA. Today, millions of them highlight how deeply embedded the software has become across AI and other tech sectors.
Suppose Huawei, or any other Chinese company, wants to challenge NVIDIA’s monopoly. In that case, it will undoubtedly have to compete against the latter’s powerful CUDA ecosystem—a quixotic battle in the eyes of many.
NVIDIA showcased its foresight many years ago by deploying the CUDA system ahead of its time.
Since 2006, it has invested hundreds of millions of dollars annually in CUDA research and development, which has helped it rapidly gain traction in AI and general-purpose computing.
In 2016, it launched the world’s first AI supercomputer, the DGX-1, and donated it to OpenAI, establishing early ties with the LLM community.
Likewise, NVIDIA leads the industry in inter-GPU connection technology. When other manufacturers relied on traditional PCIe, NVIDIA had already been innovating for over a decade. PCIe, an abbreviation for Peripheral Component Interconnect Express, is a type of connection standard or protocol used for high-speed data transfer between electronic components within a computer.
In 2014, NVIDIA launched its proprietary NVLink 1.0, achieving five times that of PCIe 3 bandwidth between its P100 GPUs. Following a series of iterations over the years, NVLink today has literally evolved into a beast. It can offer up to 600GB per second of bandwidth between GPUs, a staggering tenfold increase over PCIe 4.0.
This multi-layered ecosystem creates a formidable barrier, making it difficult for anyone to circumvent NVIDIA if they were to succeed. Many Chinese supercomputing centers or end-users had little choice but to scramble for NVIDIA GPUs.
Amid tightening US chip embargoes against China, NVIDIA developed the H20 chip specifically for the Chinese market. Unsurprisingly, with only 20 percent of the performance on its flagship H100, the market response to H20 was tepid. Chinese firms chose to buy the more sophisticated H100 or A100 chips on the black market at a premium rather than settle for inferior substitutes.
Eyeing the lucrative prospects of homegrown GPUs, Huawei has tried to fill the gap with its Ascend 910B and now 910C chips. Their real-world performances aside, whether they can inspire a broader industry restructuring and consolidation remains to be seen.
The market is characterized by segmentation, with resources spread too thin and efficiency woefully low. A disruptor the size of Huawei may help pool resources and set standards for the industry.
This doesn’t suggest that NVIDIA is in a safe place. Its dominance has made itself a target for the whole industry. In a strategic move away from relying on NVIDIA, Apple has started using Google’s TPUs (tensor processing units) to train smaller AI models.
Meanwhile, Chinese companies are chipping away at NVIDIA’s software supremacy. Startups like Moore Threads and Biren Technology have begun building software compatible with CUDA, aiming to weaken its grip.
Inter-GPU connection is another realm of innovation Chinese companies are targeting. NVIDIA’s NVLink has long kept competition at bay, offering unparalleled data transfer speeds between GPUs.
In response, major global players like AMD, Google, Intel, and Microsoft announced in March 2024 the development of UALink, a new interconnection standard for AI data centers meant to rival NVLink’s primacy.
For Chinese GPU manufacturers, it’s time to rethink their approach. Competing on pure hardware specs alone is futile, as they will always be one step behind NVIDIA. Instead, they should focus on expanding their ecosystems, building stronger collaborative networks, and gradually increasing their market share.
As NVIDIA did in its early days, Chinese companies must double down on R&D, but this effort must be matched with relevant commercialization plans. Continuous iteration is unlikely without commercial adoption and customer feedback.
In the long term, replacing NVIDIA may be an inevitable step. In the face of intensifying US chip sanctions, Chinese AI developers will be forced to wean themselves off dependence on NVIDIA. In fields like humanoid robotics, this has already begun.
Given my observations, domestic roboticists are steadily supplanting NVIDIA GPUs and its iconic Isaac Gym/Sim simulation platforms with indigenous equivalents, citing a possible blanket ban imposed by the US.
As global competitors gang up on the AI behemoth, NVIDIA might face mounting pressure. However, the threat from China should be the least of its concerns. No local rival seems capable of shaking up its domination now. Until China can build a competitive ecosystem of its own, the road to GPU independence will remain long and bumpy.
Stay up-to-date on engineering, tech, space, and science news with The Blueprint.
By clicking sign up, you confirm that you accept this site's Terms of Use and Privacy Policy
Ni Tao Ni Tao worked with state-owned Chinese media for over a decade before he decided to quit and venture down the rabbit hole of mass communication and part-time teaching. Toward the end of his stint as a journalist, he developed a keen interest in China's booming tech ecosystem. Since then, he has been an avid follower of news from sectors like robotics, AI, autonomous driving, intelligent hardware, and eVTOL. When he's not writing, you can expect him to be on his beloved Yanagisawa saxophones, trying to play some jazz riffs, often in vain and occasionally against the protests of an angry neighbor.
