AI Large Models Become the Next New Battlefield in Security
-
The OpenAI 'power struggle' drama is about to come to an end.
Since CEO Sam Altman was suddenly announced to be fired by the board, and Chairman and President Greg Brockman resigned; followed by OpenAI employees threatening to resign to demand Altman's return; and then the OpenAI board replaced members, and Altman returned to OpenAI.
On the surface, this seems to be a battle for control over a uniquely dominant tech startup. However, from various signs, the trigger for this 'power struggle' stems more from differences in the vision for the future development of AI: one faction carries the banner of 'accelerationism,' hoping AI will accelerate under the leadership of tech elites to transform the world; the other is a conservative faction rooted in altruistic theory, striving to keep AI's development under human control.
From crafting lifelike artworks to mimicking human language with near-perfect precision, generative AI is rewriting the rules of innovation and automation.
The high energy consumption of large AI model training, AI's grasp of linguistic nuances and ethical boundaries, its potential manipulation of misinformation and public opinion, and generative AI's role in human creativity... As generative AI accelerates forward, these issues still warrant careful consideration.
In March this year, shortly after Samsung Electronics introduced ChatGPT services internally, three confidential data leaks occurred. Some employees input sensitive semiconductor production codes and internal meeting details into ChatGPT, resulting in these materials being uploaded to US servers with high likelihood of exposure.
Following the incident, Samsung swiftly implemented measures to restrict employee usage scenarios of ChatGPT, sparking industry-wide discussions about data privacy and security risks posed by such large language model technologies.
Objectively speaking, in the internet era, any act of uploading data to the cloud carries potential security risks. During the early days of cloud computing, many enterprises worried about sensitive data being leaked by cloud service providers and refused to upload it to the cloud.
Even today, a large number of companies still store private data locally to enhance security, as cloud service providers have not yet fully gained the trust of enterprises.
The rise of generative AI has further aggravated this issue. On one hand, due to the extremely high costs required for training and operating large models, very few enterprises can afford the massive investment needed to build and maintain their own large-scale AI services locally.
On the other hand, large model services provided by cloud vendors require vast amounts of data for training and interaction, especially domain-specific data. The more domain data a large model possesses—particularly data related to enterprise R&D and operations—the more satisfactory its output tends to be.
For example, when enterprise developers use AI code assistance tools, they typically need to upload the company's existing codebase to enable the large model to provide more accurate code predictions. Similarly, marketing professionals can input past marketing materials into the large model to automatically generate high-quality marketing content, thereby improving work efficiency.
To achieve this, enterprises and research institutions often collect data including user-generated content such as text and images. These raw training datasets may contain sensitive private information of users.
If data collection is improper, contains biases or incorrect labels, or if the data is poisoned, it may lead to erroneous outputs, discriminatory behavior, or other negative effects from the large model. Additionally, during the application of such data, risks like data leaks and privacy exposure emerge. These issues not only pose legal risks but also trigger public trust crises regarding AI.
Furthermore, cloud providers offering large model services typically serve multiple clients simultaneously. After acquiring data from various enterprises, ensuring strict isolation of this data within each client's service scope becomes a significant challenge for both cloud providers and enterprises.
Once data isolation fails, data obtained from Client A may be used in interactive responses provided to Client B, resulting in data leakage.
If a large amount of private and confidential data uploaded by enterprises is not adequately protected, malicious attackers or insiders within cloud service providers may exploit software vulnerabilities or their authority to access this information. This not only leads to improper gains but also causes immeasurable harm to the enterprises.
Considering the vast amount of training and interaction data required by large models, which far exceeds the scale of data previously uploaded to the cloud by enterprises, this risk has grown exponentially compared to the past.
Currently, generative AI has demonstrated unprecedented levels of intelligence, positioning it as a critical component in enterprise IT infrastructure. Given its importance, the frequency of attacks targeting generative AI will make it a new battleground for security, following cloud computing, big data, IoT, and mobile internet.
Meanwhile, large model technology will significantly improve the efficiency of cybersecurity operations, fundamentally transforming the landscape of cybersecurity at a deeper level.
Some experts also believe that generative AI will further widen the poverty gap and deepen the digital divide. As an emerging AI technology, generative AI requires vast amounts of data and substantial computing power. This means it can only be widely adopted in technologically advanced countries and regions, remaining under the control of a few economies.
The digital poverty gap continues to expand, with the voices of technologically disadvantaged regions being overlooked. The potential risk is the widespread dissemination of values from developed economies, leading to an irreversible deepening of the digital divide.
With the advent of the large model era, their powerful capabilities provide new ideas for the transformation of security protection technologies. "Using AI to combat AI" has become a trending direction.
In fact, the concept of adversarial offense and defense is not exclusive to model security. As early as the last decade, in the face of various security threats, the field of artificial intelligence gradually formed a security philosophy of "testing defense through offense—promoting defense through offense—integrating offense and defense." By simulating various attack scenarios, it continuously explores the weaknesses of models and systems, thereby driving improvements in algorithmic and engineering defense capabilities.
However, traditional security protection primarily relied on machine learning algorithm models, which required extensive professional data knowledge and faced challenges such as knowledge gaps and the untimely cold start of small samples. Leveraging large model technology can enable more intelligent security prevention and control.
Currently, the security issues faced by generative AI can be divided into three levels. The primary issues are technical attacks, such as cyber attacks, vulnerability attacks, and data attacks—particularly providing poor data to large models or contaminating their training data, which may lead to erroneous results. These problems are relatively easier to resolve.
The more challenging intermediate issues primarily involve content security. For instance, large models can serve as excellent assistants but can also become tools for malicious actors. They can help a less skilled hacker write better attack code or phishing emails.
How to make its content more controllable? How to prevent large AI models from doing harm? This has gone beyond the scope of technology. Although some built-in "safety guardrails" have been implemented for large models, they are still vulnerable to injection attacks or algorithmic attacks.
From a higher-level perspective, large models can integrate all human knowledge and then interact with humans through prompts. But when AI's capabilities surpass those of humans, will it still be content to remain a tool for humans? These technological challenges must ultimately be addressed by technology itself, as it continues to evolve. We cannot halt the development of related technologies simply because of potential future problems.
From the current state of the industry, there is still a lack of user-friendly and standardized evaluation tools and rules to assess whether AI is safe or not.
This is another aspect that can be addressed in the defense of large models. By leveraging large model technology to learn risk knowledge and standard rules, we can enhance AI's ability to recognize and understand risks. This enables the use of large models to counter large models, achieving rapid defense and quick cold-start capabilities.
Facing the cybersecurity challenges brought by large models requires joint efforts from enterprises, security teams, and regulatory agencies to address.
First, strengthen the training and management of large models. By adopting more effective data augmentation and processing techniques, issues such as overfitting and gradient explosion in large models can be reduced, enhancing the robustness and reliability of the models. At the same time, the training and management of large models also require enhanced supervision and auditing to ensure data privacy and security.
Second, develop new security defense technologies. As attack methods targeting large models continue to evolve and update, new security defense technologies need to be developed to counter them. For example, artificial intelligence technologies can be utilized to detect and defend against malicious code and phishing attacks.
Taking text security as an example, large models can be trained based on security standard rules, domain risk knowledge, and historical risk samples to improve the model's understanding of risk criteria and content, thereby enhancing risk detection capabilities.
Additionally, the generative capabilities of large models can be combined with security knowledge graphs to construct attack samples for continuous iterative optimization of detection models.
Third, strengthening data security protection. Beyond focusing on offensive and defensive interactions of large models on the internet, the security and privacy issues inherent to large models themselves have also raised concerns.
To address potential data security issues during the training and usage of large models, a series of privacy protection technologies must be employed. For example, techniques such as homomorphic encryption, differential privacy, secure multi-party computation, model watermarking, and fingerprinting can be used to safeguard data privacy and security.
Fourth, strengthen regulation and legal protection. Technological legislation often lags behind the pace of technological development. Without the guidance and regulation of laws and regulations, an increasing number of companies using generative AI are finding that the security of their systems is under significant challenge.
Faced with cybersecurity challenges under large models, regulatory and legal institutions also need to strengthen management and oversight. For example, government regulatory agencies should effectively guide and coordinate the design and evolution of generative AI through policy legislation.
The formulated policy frameworks need to be consistent with the legislative and regulatory backgrounds of various countries and must be updated as AI technology advances, continuously improving the applicability of existing regulations. While regulating, freedom for AI innovation should be maintained to collaboratively create higher-quality content.
Generative AI is an extremely complex issue, with unprecedented levels of complexity in ethics, data, training, and other areas. It is a new field and a proposition placed before everyone.
From the history of technology enterprises, in the early stages of new technology development, participants with different approaches and philosophies often unite and collaborate to advance technological dissemination together.
However, once technology becomes widespread, differences in commercialization and implementation paths may lead to divergent directions. On the eve of AI's widespread and deep application, the debates on how future technologies should be disseminated and regulated are just beginning.
In the face of future security trends and challenges, enterprises should join hands to establish measurable security systems, creating an inherently adaptive 'security immunity' to address the new trends of offense and defense in the intelligent era.