Silo AI Launches New Open-Source Language Model 'Poro' for Europe
-
Helsinki-based AI startup Silo AI released Poro this week, a new open-source large language model (LLM) designed to enhance multilingual AI capabilities for European languages. Poro is the first in a planned series of open-source models that will eventually cover all 24 official EU languages. These models were jointly developed by Silo AI's SiloGen generative AI division and the TurkuNLP research group at the University of Turku.
Silo AI's CEO Peter Sarlin stated in an interview with VentureBeat: "This is a matter of digital sovereignty. You want to ensure there are models that capture foundational values, culture, and language. Ultimately, it's about value creation - ensuring not just Europe, but any company can create value and develop proprietary models that remain within Europe and within organizations."
The Poro34B model boasts 3.42 billion parameters and is named after the Finnish word for 'reindeer' (reindeer). It utilizes the BLOOM transformer architecture with ALiBi embeddings. The model was trained on a partitioned multilingual dataset of 21 trillion tokens covering English, Finnish, and programming languages such as Python and Java. Poro is currently being trained on Europe's fastest supercomputer, LUMI, located in Kajaani, Finland. This supercomputer provides 512 AMD Instinct MI250X GPUs, delivering a computational capacity of 74 petaflops.
Sarlin stated that Poro is designed to address the core challenges of training high-performance models for low-resource European languages such as Finnish. By utilizing cross-lingual training methods, the model can leverage data from high-resource languages like English.
As part of its commitment to transparency, SiloGen will document Poro's training progress through the Poro Research Checkpoints program. Sarlin explained: "We will release checkpoints at various stages of model training, which is quite a novel approach. Currently, there are no similar initiatives that provide such transparent information about model training." According to benchmark data released by Silo AI, Poro achieved state-of-the-art results after completing only 30% of its training.
Sarlin believes that open-source models like Poro represent the future of artificial intelligence, providing transparent and ethical alternatives to the closed models of major tech companies. He said: "I personally believe there will eventually be many open-source alternatives. The safest path forward is actually to embrace open-source and fully understand how these models are built and what their architectures are."
Silo AI plans to continue releasing regular Poro checkpoints throughout the training process. The ultimate goal is to create a comprehensive open-source model family covering all European languages. If the preliminary results are any indication, Poro may soon exert competitive pressure on major tech companies.
Poro represents part of the ongoing collaboration between Silo AI and the University of Turku. This partnership combines Silo AI's applied artificial intelligence expertise and computational resources with the University of Turku's leadership in multilingual language modeling research. Sarlin stated that this exemplifies how industry and academia can jointly advance AI capabilities, particularly for low-resource European languages.
The release of Poro marks a new era of open collaboration and transparency in the field of natural language processing. Initiatives such as Poro Research Checkpoints provide the entire community with access to tools and insights that were previously blocked by major tech companies. Sarlin stated: "We collaborate with large brands like Allianz, Rolls-Royce, Honda, and Philips. We've heard that these major corporations are very concerned about what the final regulations will look like and which models they can use."
If Poro delivers on its promises, it may enable democratic access to high-performance multilingual models, providing Europe with a local alternative to compete with the systems of US tech companies. Although still in its early stages, Poro represents a significant milestone in bringing language AI from proprietary domains into the open-source sphere.