Xiaomi surged about 3% during the session, announcing the open-source of its first inference large model "Xiaomi MiMo"
Xiaomi Group surged during the trading session, currently up about 3%. On the news front, Xiaomi announced today the open-sourcing of its first large model designed for inference, "Xiaomi MiMo," which integrates pre-training and post-training to comprehensively enhance inference capabilities
Today, Xiaomi open-sourced its first large model designed for reasoning, "Xiaomi MiMo," linking pre-training to post-training to comprehensively enhance reasoning capabilities.
In the public evaluation sets for mathematical reasoning (AIME 24-25) and coding competitions (LiveCodeBench v5), MiMo, with only 7B parameters, surpassed OpenAI's closed-source reasoning model o1-mini and Alibaba's larger open-source reasoning model QwQ-32B-Preview.
The enhancement of MiMo's reasoning capabilities is driven by innovations in data and algorithms across multiple dimensions during the pre-training and post-training phases, including:
Pre-training: The core is to expose the model to more reasoning patterns
- Data: Focused on mining rich reasoning corpora and synthesizing approximately 200B tokens of reasoning data.
- Training: Conducted three-stage training, gradually increasing training difficulty, totaling 25T tokens of training.
Post-training: The core is an efficient and stable reinforcement learning algorithm and framework
- Algorithm: Proposed Test Difficulty Driven Reward to alleviate the reward sparsity problem in difficult algorithm issues and introduced Easy Data Re-Sampling strategy to stabilize RL training.
- Framework: Designed the Seamless Rollout system, accelerating RL training by 2.29 times and validation by 1.96 times.
MiMo-7B has open-sourced 4 models to HuggingFace: https://huggingface.co/XiaomiMiMo
Technical details can be found in the technical report: [https://github.com/XiaomiMiMo/MiMo/blob/main/MiMo-7B-Technical-Report.pdf](https://www.oschina.net/action/GoToLink?url=https%3A%2F%2Fgithub.com%2FXiaomiMiMo%2FMiMo%2Fblob%2Fmain%2FMiMo-7B-Technical-Report.pdf)
