The Largest Yet! Musk's AI Model Grok Goes Open Source with 314 Billion Parameters
-
According to reports, Musk's AI startup xAI has officially announced the open-sourcing of its large language model Grok-1, making it available for public download.
It is understood that Grok-1 is a large language model built on the Mixture-of-Experts (MoE) technology, boasting 314 billion parameters, far surpassing OpenAI's GPT-3.5 with 175 billion. This is currently the largest open-source large language model by parameter count, released under the Apache 2.0 license, which includes the model weights and architecture. xAI stated that Grok-1 has always been trained independently by xAI, with its pre-training phase completed in October 2023. This release is the raw base model checkpoint at the end of Grok-1's pre-training phase, meaning the model has not been fine-tuned for any specific applications.
One of xAI's objectives is to compete with rivals like OpenAI, Google, and Microsoft in the field of large models. Its team comprises members from prominent organizations such as OpenAI, Google DeepMind, Google Research, and Microsoft Research.
Currently, xAI has not disclosed specific benchmark results for Grok-1. Its competition with other companies' large models will be a key focus in the industry.