The Wave of Large Models Drives the Development of AI Middle Platforms: New Opportunities for Digital Transformation
-
As large models move toward commercial implementation, a batch of 'old concepts' are experiencing a new wave of applications.
Similar to data middle platforms and business middle platforms, AI middle platforms are not a new term. Generally, various 'siloed' independent business system architectures aim to share intelligent capabilities through AI middle platforms, lowering the development threshold for AI business scenarios. However, before the emergence of large model technology, AI applications were often isolated islands, lacking generalized and universal implementation scenarios.
The emergent capabilities of large model technology have made AI middle platforms a focal point for enterprises, especially financial institutions, to explore. As an intermediate foundational tool between AI applications and large models, large model middleware has also become a hot topic for investment institutions. Faced with the comprehensive reshaping of infrastructure, tools, and application layers by AI, how will investment institutions 'strike gold'?
'Without large models, we used to ponder how AI should be developed,' admitted the head of the IT department at a fund company. However, AI applications were always 'siloed' in the past. Whenever a new, uncovered problem arose, the business department had to redevelop the corresponding model, resulting in low efficiency in computing power usage.
'Large model technology has had a significant impact on traditional AI vendors and IT service providers,' said Bai Shuo, Chief Scientist at Hundsun Electronics. In the past, AI systems were siloed, with each part being a small model requiring data training. The learning effectiveness of small models also had room for improvement. However, large language model technology, based on vast parameters, enables machines to possess the general capability to perform different tasks simultaneously.
Meanwhile, the business scope of AI and IT vendors is also changing. 'The past scenario where IT vendors and AI stayed out of each other's way or only dabbled in AI has been broken,' Bai Shuo pointed out. AI empowerment has become a battleground for IT service providers, and AI technology applications in asset management are undergoing a transformation. 'Technologically, whether it's interfaces, database access, business processes, or API calls, there is a trend toward low-code solutions. AI tools are widely used to improve quality and efficiency,' Bai Shuo added.
When considering the application of emerging technologies, security is paramount.
Implementing large model applications while ensuring data security, consolidating general data construction capabilities, and building AI middle platforms to enable various business lines to share general intelligent capabilities have become the preferred solution for many financial institutions.
'Enterprise-specific, privately controlled large models are an inevitable form. Institutions need to ensure 100% security and ownership of large models,' Gao Jingjun, CTO of Kejie Technology, told reporters. Currently, there are three main paths for implementing large model applications: First, conducting full-scale model training, deploying large models privately within the enterprise, and combining them with private enterprise data to create specialized large models for vertical fields. Second, fine-tuning parameters based on open-source or commercial large models combined with private enterprise data, enabling large models to deeply understand the enterprise's private data. Third, using general large model interfaces to help enterprises build data knowledge bases and create proprietary AI applications.
'The first solution requires building a large model from scratch, demanding significant talent and financial investment. Currently, fine-tuning and model training based on open-source large models combined with private data is a more common approach,' Gao Jingjun admitted.
The aforementioned head of the IT department at the fund company revealed that after the emergence of large model technology, their institution conducted joint tests with multiple systems and attempted localized deployment. They fine-tuned and refined large models, exploring AI middle platform construction based on the company's existing quantitative platforms, machine learning achievements, and computing power middle platforms.
He pointed out that building an AI middleware platform requires using large models as the entry point for all applications, constructing corresponding business middleware based on core needs of business scenarios, and ultimately achieving the application of large models in specific businesses. However, this brings challenges: insufficient computational power during fine-tuning and the complexity of selecting algorithms from different vendors during implementation. "The hardest part is the data," he told reporters. For example, in scenarios involving automatic code generation, at least 30 high-quality project codes are required to make a project usable, which poses a high barrier for institutions.
In the process of deploying financial large models, how can we shorten the AI development cycle, reduce AI application costs, and improve actual work efficiency?
Driven by the wave of large models, middleware—a foundational software layer between AI applications and large models—has regained market attention for its role in solving commercialization challenges. It primarily addresses issues such as resource scheduling, data integration, model training, application integration, and the fusion of knowledge bases with large models during deployment.
In key industries like government, finance, telecommunications, and transportation, middleware, as a critical component of information innovation, has experienced rapid growth. Listed companies Dongfangtong (300379.SZ) and Baolande (688058.SH) are the two with the highest market share in the domestic financial middleware sector. In 2020, Dongfangtong's net profit increased by over 72% year-on-year, but due to the impact of the pandemic on revenue recognition and business operations, both companies performed poorly last year and in the first quarter of this year.
With the commercial deployment of large model technology, the middleware industry is expected to see new market demand. On June 9 this year, Dongfangtong stated on an interactive platform that AI large model training relies on extensive hardware infrastructure for processing and optimizing massive datasets, requiring high standards in data handling and deployment management. Efficient data processing and management middleware are needed to support large-scale data storage, transmission, and processing, as well as rapid model deployment, monitoring, and optimization. Middleware must also accommodate diverse hardware and software environments to meet varying scenario needs.
"For enterprises, the key to reducing the barriers to large model implementation lies in enterprise-grade middleware for large models," Gao Jingjun noted. For instance, AI OPS capabilities can accelerate large model applications by providing a suite of tools to address engineering challenges in private environment model training. These tools enable fully automated training environment setup, low-code one-stop solutions for data ingestion, labeling, supervised fine-tuning, reinforcement learning feedback, and model deployment, while supporting data feedback loops and continuous model iteration.
Large models are reshaping both infrastructure and application layers. Faced with these new opportunities, how do investment institutions view this emerging market?
Lu Yi, a partner at venture capital firm XVC, told reporters that the global development of generative AI currently spans three layers: the model layer, the infrastructure and tools layer, and the application layer.
"The model layer is evolving rapidly," Lu Yi mentioned. Whether it's OpenAI's ChatGPT or Meta AI's Llama2, advancements in parameter scale and performance are accelerating. In the tools layer, infrastructure like databases and development frameworks is keeping pace with model progress. From an application perspective, the current "blossoming" of large models has made them productivity-enhancing tools across multiple vertical sectors.
Discussing the rapid evolution of foundational models, Kong Lei, head of the Amazon Web Services ecosystem architect team, noted that this is largely due to the growing market popularity of open-source large model ecosystems. "In a few months, you might find large models becoming very accessible, with increasingly lower barriers. Open-source foundational models are also commercially viable, significantly reducing costs." He believes that as consumer-facing large model applications proliferate, the next wave will focus primarily on business-to-business (B2B) scenarios.
Meanwhile, the enormous parameter requirements for training large language models make the entire process highly specialized and systematic. "For example, vector databases, which have recently attracted market attention, represent a new business direction for traditional data companies," Kong Lei added. In the process of large model deployment, vendors in computing power, algorithms, and data will find new opportunities as the large model ecosystem expands.
"We have currently invested in two companies related to large models - one focusing on personal emotional companionship chatbots based on large models, and another providing infrastructure for large model and AI application implementation," Lu Yi told reporters. Currently, China's large model market is still in the early stages of rapid development. Although many major tech companies and industry leaders have already entered the field, there remain numerous opportunities for startups in both large model infrastructure and application companies, which investment institutions are closely monitoring.