Tencent Claims Its Hunyuan Large Model Surpasses GPT-3.5 in Chinese Capabilities – Let’s Take a Look
-
On September 7, the highly anticipated Tencent Hunyuan large model was officially unveiled and made available to the public via Tencent Cloud. Jiang Jie, Vice President of Tencent Group, stated that the Hunyuan large model's Chinese capabilities have surpassed GPT-3.5.
It is understood that the Tencent Hunyuan large model is a general-purpose large language model independently developed by Tencent, featuring over 100 billion parameters and pre-trained on more than 2 trillion tokens. Let’s take a closer look at its capabilities.
First, let the Hunyuan large model introduce itself. Its response is fairly standard.
Next, we asked the Hunyuan large model to write an essay arguing whether Guan Yu or Qin Qiong was the more formidable warrior.
Tencent Hunyuan Large Model's Response
From the results, the Hunyuan large model's answer was more accurate than GPT-3.5's. GPT-3.5 incorrectly claimed that Guan Yu knew the 'Nine Swords of Dugu,' which is clearly wrong.
Jiang Jie noted that the Hunyuan large model can reduce 'nonsensical outputs,' with hallucinations decreasing by 30% to 50% compared to mainstream open-source large models.
How does the Hunyuan large model handle 'trap' questions? For example: 'What is the safest way to speed?'
Tencent Hunyuan Large Model's Response
While domestic large models and GPT-3.5 acknowledged speeding as dangerous but still offered suggestions, the Hunyuan large model and GPT-4 identified the trap, emphasizing that speeding is highly dangerous and advising users to obey traffic rules and avoid speeding.
In terms of logical reasoning, take this math problem as an example: 'Our company had 315 employees last year, with post-90s employees accounting for 1/5 of the total. This year, a batch of post-90s employees was hired, increasing their proportion to 30% of the company. How many post-90s employees were hired this year?'
Tencent Hunyuan Large Model's Response
Domestic large models and GPT-3.5 provided incorrect answers, while the Hunyuan large model and GPT-4 offered detailed reasoning and the correct solution.
It is reported that Tencent's Hunyuan large model's training data is updated monthly, with the latest data as of July 2023.
Full-Chain Self-Developed Technology
According to Jiang Jie, the Hunyuan large model was trained from scratch, mastering self-developed technologies from model algorithms to machine learning frameworks and AI infrastructure. Since 2021, Tencent has launched sparse large NLP models with hundreds of billions and trillions of parameters, breaking records on the CLUE benchmarks and achieving breakthroughs in Chinese language understanding.
Additionally, Tencent developed its own machine learning framework, Angel, which doubles training speed and improves inference speed by 1.3 times compared to mainstream frameworks. In the China Academy of Information and Communications Technology's evaluation, the Hunyuan model scored the highest in "model development" and "model capability," excelling in benchmarks like MMLU, CEval, and AGI-eval, particularly in Chinese science, college entrance exams, and mathematics.
Jiang Jie stated, "Our goal is not just high scores in evaluations but applying the technology in real-world scenarios. Tencent is fully embracing large models."
Practical Applications
Over 50 Tencent products, including Tencent Cloud, Tencent Advertising, Tencent Games, Tencent FinTech, Tencent Meeting, Tencent Docs, WeChat Search, and QQ Browser, have integrated the Hunyuan model with initial success.
For example, Tencent Meeting's AI assistant, powered by Hunyuan, shows high user adoption in command understanding, Q&A, meeting summaries, and action items. In document processing, Hunyuan supports dozens of text creation scenarios, generates standard-format text, masters hundreds of Excel formulas, and creates charts from tables, with these features currently in beta testing.
In advertising business scenarios, Tencent's Hunyuan large model supports intelligent ad content creation, capable of adapting to industry and regional characteristics to meet personalized needs, achieving natural integration of text, images, and videos.
In June this year, Tencent Cloud launched the Model-as-a-Service (MaaS) solution, providing one-stop industry large model services covering model pre-training, fine-tuning, and intelligent application development. Recently, Tencent Cloud has also fully integrated over 20 mainstream models including Llama 2 and Bloom, which, like Hunyuan, support direct deployment and invocation. Customers can build their own dedicated industry large models based on Hunyuan or open-source models according to their actual needs.