MiMo-V2.6：数据与后训练配方带来的进步
MiMo-V2.6 是“简单”地目前最好的（暂且如此）。尽管其架构设计简单，但它目前在开放权重基准测试中排名第一（加权平均）。
用“简单”一词，我指的是经典的分组查询注意力（GQA），搭配滑动窗口注意力（SWA），窗口大小仅有微小的 128 个 token。
因此，这强调了我近几个月来一直试图提出的观点之一：大多数进步仍来自数据和后训练配方的改进。花哨的注意力变体大多只是效率调整。
训练数据改进和配方改进有哪些？MiMo 团队分享了一份相当详细的技术报告。其中有很多内容需要仔细消化，但简而言之，有几点特别突出：
-
智能体任务数量增加；同时在不同测试框架中进行训练（在保留测试框架上的平均 DeepSWE pass@1 准确率从大约 50% 提高到 66%）。
-
更好的奖励信号：他们用一个智能体式评分器替换了简单的正确性验证器，该评分器还会查看执行轨迹。
-
大型 RL 批次（1,568 个提示 × 16 次 rollout = 25,088 条轨迹）以及每次更新 27–37 亿个训练 token（不过，前代模型使用了什么尚不清楚）。
对照原文
MiMo-V2.6 is "simply" the best (for now). Despite its simple architecture design it's currently No.1 in the open-weight benchmarks (weighted average). With "simple," I mean a classic Grouped Query Attention (GQA) with Sliding Window Attention (SWA) at a tiny 128-token window size. So, that underlines one of the points I've been trying to make in recent months: most of the progress still comes from the data and post-training recipe improvements. Fancy attention variants are just mostly efficiency tweaks. What are some of the training data improvements and recipe improvements? The MiMo team shared a pretty detailed technical report. Lots to carefully digest there, but in short, there are a few things that stood out: 1. An increase in agent tasks; also training across different harnesses (the average DeepSWE pass@1 accuracy on held-out harnesses improved from approximately 50% -> 66%). 2. Better reward signals: they replaced a simple correctness verifier with an agentic grader that looks at the execution traces as well. 3. Large RL batches (1,568 prompts × 16 rollouts = 25,088 trajectories) and 2.7–3.7 billion training tokens per update (unclear, though, what the predecessor used).