The Pitfalls Behind Open-Source AI Models: How to Identify and Avoid Pseudo-Open Source
-
The advent of ChatGPT has given more people access to powerful artificial intelligence (AI), but the inner workings of this chatbot remain an undisclosed secret.
Thus, in recent months, efforts to make AI more 'open' have seemingly garnered more attention.
In May of this year, Meta's 'Llama' model was leaked, allowing external access to its underlying code and model weights that determine its behavior. In July, Meta released an even more powerful model, Llama 2, claiming it was available for anyone to download, modify, and reuse. Since then, Meta's Llama series has become the foundation for many companies, researchers, and AI enthusiasts to build tools and applications with ChatGPT-like capabilities.
"We have a broad base of supporters worldwide who believe in our open approach to AI today... Researchers are committed to using the model for research, and people in the tech industry, academia, and policy sectors, like us, see the benefits of Llama and open platforms," Meta said upon releasing Llama 2. Recently, Meta also released another model—Llama 2 Code—fine-tuned specifically for coding.
For decades, the open-source approach has democratized software access, ensured transparency, and improved security. Now, it seems poised to have a similar impact on AI.
But this may still fall far short.
A research team from Carnegie Mellon University, the AI Now Institute, and the Signal Foundation explored the reality of 'Llama 2' and other AI models labeled as 'open source' in a paper titled "Open (For Business): Big Tech, Concentrated Power, and the Political Economy of Open AI." They pointed out that models branded as 'open source' may come with certain 'pitfalls.'
According to the paper, while Llama 2 can be downloaded, modified, and deployed for free, it does not fall under traditional open-source licenses. Meta's license prohibits using Llama 2 to train other language models, and developers deploying it in apps or services with over 700 million daily users require a special license.
This level of control allows Llama 2 to provide Meta with significant technical and strategic advantages—for example, benefiting from improvements made by external companies using the model in their applications.
Researchers noted that models released under standard open-source licenses, such as EleutherAI's GPT Neo, offer greater openness. However, these projects struggle to compete with models backed by large corporations.
First, the data required to train advanced models is often confidential. Second, the software frameworks needed to build such models are typically controlled by large corporations—TensorFlow and PyTorch, the two most popular frameworks, are maintained by Google and Meta, respectively. Third, the computational power required to train large AI models is beyond the reach of most developers or companies, often costing tens or hundreds of millions of dollars per training run. Finally, the human resources needed to refine and improve these models are also largely accessible only to major corporations.
As a result, the research team argues that given current trends, one of the most important technologies of recent decades may ultimately only enrich and empower a handful of companies, including OpenAI, Microsoft, Meta, and Google. If AI is truly a world-changing technology, broader access and democratization could maximize its global benefits.
"Our analysis shows that this level of 'open source' does not democratize AI," said Meredith Whittaker, one of the paper's authors, in an interview with Wired. "In fact, companies and institutions can—and already do—leverage 'open' technologies to consolidate and expand centralized power."
Furthermore, even the most 'open' AI systems do not inherently ensure democratic access or meaningful competition, nor do they resolve issues of oversight and accountability.
Whittaker added that open source should be a key consideration in the urgently needed AI regulations. "We truly need meaningful alternative technologies to replace those defined and dominated by large monopolies—especially when AI systems are integrated into highly sensitive areas with significant public impact, such as healthcare, finance, and education. Creating conditions to make such alternatives possible is a project that can coexist with, and even be supported by, regulatory movements like antitrust reforms."
The research team also stated that beyond balancing the power of large corporations, making AI more open may be crucial to unlocking its full potential while avoiding its worst tendencies.
If we want to understand the capabilities of the most advanced AI models and mitigate the risks associated with their deployment and further development, it would be best to open these models to scientists worldwide.
Just as "security through obscurity" can never truly ensure the safe operation of code, protecting the workings of powerful AI models may not be the wisest approach.