April 30, 2024
BEIJING, TIANJIN – China makes progress in leveraging supercomputers and domestic artificial intelligence chips for the training of AI large language models, as the country handles Washington’s restrictions on exporting the US company Nvidia’s most advanced AI chips to Beijing, renowned experts told in exclusive interviews with China Daily.
Zheng Weimin, academician of the Chinese Academy of Engineering, said currently, there are 14 national-level supercomputer centers in China, with some of them having surplus computing power.
“Through coordinated hardware and software design, these resources can be leveraged for large language model training and inference,” he said.
A team led by Zheng has conducted AI training on the large language models, such as LLAMA 7B and Baichuan 7B, by using a domestically-developed supercomputer, which “has achieved results aligned with those from training on Nvidia’s chip platforms but at only one-sixth of the cost of renting Nvidia chips”, Zheng said.
The remarks came as the prices of Nvidia’s most advanced AI chips surged in China after the US government enacted tougher restrictions on exporting these chips to the country.
Amid such a context, Zheng is not alone in exploring ways to harness supercomputers for AI training.
Kang Bo, head of the data intelligence department at the National Supercomputer Center in Tianjin, told China Daily that “Domestic supercomputers do have a cost advantage in terms of usage, as Nvidia chips’ power consumption and prices are very high now.”
According to him, the National Supercomputer Center in Tianjin unveiled an in-house large language model roughly translated as Tianhe Tianyuan in May last year by leveraging the center’s supercomputing power. The center is renowned for developing and deploying China’s top-ranking supercomputers, including the world-renowned Tianhe-1.
“Our supercomputer can handle both scientific computing and AI computing, including precise reasoning. I can utilize this integrated supercomputing system to accomplish tasks that previously required multiple systems to switch between. This is our advantage,” Kang said.
According to him, the center has been approved by the Ministry of Science and Technology to establish the national public computing open innovation platform for next-generation AI to expand its cooperation with more parties.
Earlier this month, the National Supercomputer Center in Tianjin said it has collaborated with partners in medicine, port, public security, customs, traditional Chinese medicine and environmental protection to promote the application of Tianhe Tianyuan in more specific industries.
Kang said based on the Tianhe Tianyuan, the center has developed a “hospital AI brain” in collaboration with partners, which can help doctors assess diseases and provide treatment recommendations, and an AI assistant which can help customs analyze data and provide environmental protection suggestions.