Kunlun Wanwei Tests "Tiangong" Large Model: Redefining Search Boundaries with AI
-
On Kunlun Wanwei's AI journey, the Tiangong AI search exemplifies the company's development strategy and technological capabilities...
On November 3rd, Kunlun Wanwei's "Tiangong" large model passed the "Interim Measures for the Management of Generative Artificial Intelligence Services" and opened its services to the public. Kunlun Wanwei stated that it will continue to advance the Tiangong large model and AIGC business to new heights, improve the user experience of multiple generative AI products, and explore the unknown world to create a better future.
Unlike previous large models regulated by the 'Interim Measures for the Management of Generative AI Services,' the Tiangong model directly offers users a generative AI product with internet connectivity. Given the current state of China's large model market, products with internet capabilities are rare due to risks related to data and privacy.
As an important development direction in the search engine field, AI search holds broad market prospects and certain competitive advantages. Through natural language interaction, AI search enables conversational Q&A, delivering an intelligent, natural, and highly efficient search experience to meet users' diverse information needs.
Image Source: Large Model Home
In this regard, Large Model Home adopts the 'AQUA' evaluation system proposed in the 'Artificial Intelligence Large Model Industry Innovation Value Research Report', conducting a multi-angle and comprehensive evaluation of the 'Tiangong Large Model' across six dimensions, including model capability, task processing ability, and application ecosystem.
Model Scale: The TianGong large model possesses a dual-trillion parameter scale, comprising a trillion-parameter pre-trained base model and a trillion-parameter RLHF model. The TianGong large model not only includes the 13-billion-parameter Skywork-13B-Base and Skywork-13B-Math models but also features the trillion-parameter Skywork-100B-Base and Skywork-100B-Math models. Increasing the model scale enhances its expressive power and generalization capabilities, enabling it to handle more complex natural language tasks.
Training Data Volume: The TianGong large model was trained using nearly 3 trillion multilingual high-quality data points. The training dataset primarily comes from various sources such as the internet, libraries, encyclopedias, news media, and social networks, covering over 30 languages including Chinese, English, German, and Russian. Increasing the volume of training data enhances the model's knowledge coverage and language comprehension, enabling it to answer a wider range of user queries, including those that are cross-domain, cross-lingual, or cross-cultural. The TianGong large model not only understands user inputs but also generates more personalized and human-like responses based on user preferences, emotions, and interests.
Training Compute Power: The TianGong large model is trained on one of China's largest GPU clusters, equipped with robust computational resources and optimization technologies. Increased training compute power improves the efficiency and stability of model training, enabling rapid iteration and optimization. The TianGong model utilizes Tencent Cloud's supercomputing cluster, which boasts over 100,000 NVIDIA V100 GPUs, making it one of the largest GPU clusters globally.
In addition, the TianGong model employs innovative training methods such as Zero-Infinity, enabling the training of trillion-parameter NLP models in just one day using 256 GPUs. These methods effectively reduce memory usage, improve model parallelism and convergence speed, and lower training costs.
In the task processing capability testing phase, Big Model Home discovered that the Tiangong large model occasionally produces incorrect answers in math tests even when the reasoning approach, formulas, and problem-solving methods are correct.
Image source: Tiangong Large Model
In large model computations, algorithm implementation errors are not uncommon. Even if the overall concept of the model is correct, various issues can arise during specific implementations. Additionally, since computer real numbers are represented using finite binary or hexadecimal digits, this leads to precision loss. When performing floating-point addition, subtraction, multiplication, or division, this precision loss can accumulate, ultimately affecting the final computation results. Large models are particularly susceptible to such precision issues during mathematical operations, especially when involving multiple iterations or complex calculations.
Image source: TianGong Large Model
No matter how large or complex a model is, it remains a model—an approximation of the real world. This means it may struggle with edge cases or have limitations in understanding certain problems. In some scenarios, even if the model's formulas and reasoning are correct, it might still produce incorrect answers due to statistical factors. This is particularly relevant for the TianGong large model, which specializes in AI-powered search functionality. After answering test questions, it also provides relevant search links and even video materials based on prompts.
Image source: TianGong large model
In multi-turn dialogues, after receiving the instruction "You got this question wrong, please do it again," the Tiangong large model still provided the correct answer for Model Home without omitting the previous calculation process. This demonstrates that while the Tiangong large model may occasionally encounter model limitations or algorithmic errors, it can self-correct, learn, and adapt through contextual understanding, continuously optimizing answers during user interactions to provide more reliable and accurate information.
Notably, the Tiangong large model also offers users a "Copilot" feature. According to reports, this feature is built on more advanced large models to help users analyze optimal personalized search terms, deeply examine search results, and refine high-quality content. However, currently this feature is limited to just four experiences per day for regular users, with these four experiences restricted to four Q&A sessions rather than four dialogue rounds.
Image Source: Tiangong Large Model
Tiangong Large Model is China's first AI search engine integrated with large model capabilities. It introduces a new direction in the search engine field, offering users a novel way to search. This approach breaks through certain limitations of traditional search and plays a leading role in the development trend of AI-powered search.
Generalization capability
In terms of generalization capability, the TianGong large model also offers document understanding functionality. By uploading files to TianGong AI Search and specifying their requirements, users can obtain interpretations generated by "TianGong." Currently, TianGong cannot analyze images present in documents.
Image source: TianGong large model
Since documents often contain complex contextual relationships, Big Model Home has found that the Tiangong large model's responses demonstrate an ability to understand inter-sentence relationships and logical connections between paragraphs, enabling comprehensive and accurate comprehension of entire documents. In practical applications, text may appear in various forms, such as spelling errors, non-standard grammar, or slang usage. After extensive training, large models can adapt to these text variations and accurately understand document content.
Image source: Tiangong Large Model
When processing semantic abstract concepts in documents, such as metaphors, analogies, and coreference resolution, the Tiangong large model can comprehend the deeper meanings within the text, not just the superficial information. This semantic abstraction capability is also a crucial manifestation of generalization ability, as it requires the model to accurately understand the profound meanings of text across different contexts.
Image source: Kunlun Wanwei
In addition to AI search functionality, Kunlun Wanwei has released the open-source code for four AIGC products on GitHub: SkyPaint, SkyMusic, SkyText, and SkyCode. These products span multiple domains, including chat, painting, text generation, and programming.
By integrating and innovating with advanced technologies like Stable Diffusion and GPT-3, these products bring new possibilities to the AI field and expand the application scenarios for the TianGong large model. This diversified approach enhances the practicality of the TianGong model, making it adaptable to the needs of different users.
Image source: Internet
Particularly at the text level (SkyText), the SkyText model places special emphasis on Chinese language processing capabilities, including Chinese chat, Q&A, translation, and text generation. This further enhances the model's ability to handle the Chinese language, making it more adaptable for Chinese users.
Image source: Big Model Home
With the continuous development of artificial intelligence technology, AI search has emerged as a significant direction in the field of search engines. AI search enables conversational interactions through natural language processing, providing intelligent, natural, and highly efficient search experiences to meet users' diverse information needs.
From a commercial perspective, AI search offers strong support for the commercialization of large language models. As AI technology matures and application scenarios expand, an increasing number of enterprises are integrating AI into their operations. The emergence of models like TianGong provides businesses with more efficient and intelligent solutions, facilitating digital transformation and intelligent upgrades. This commercial application is not limited to specific industries but extends across various sectors, driving the intelligent evolution of society as a whole.