返回 X 名人动态

Jerry Liu:用前沿智能体构建专用模型训练循环

中文全文 · AI 翻译

训练专用模型长期以来最大的瓶颈之一,是整理标准答案数据。这通常需要人工监督,是一项非常耗费人力的工作。

如今,前沿模型和智能体越来越擅长从零构建标准答案数据,尤其是在可验证的任务上。对于简单任务,它们可以一次完成;对于较长的任务,则可以模拟更长的执行轨迹,并结合奖励信号。

这使任何人都有机会构建大部分环节自动化的研究循环,无需深厚的机器学习专业知识,也无需为数据整理投入大量人力。这部分可以交给前沿模型或智能体;人的主要任务是定义恰当的评估。这对希望针对特定任务逼近性价比最优水平的人尤其有用。开放权重模型及其运行框架的后训练能力,应该向所有人开放,而不只是实验室或特定初创公司。

普通用户的流程如下,他们只需负责第一步:

  1. 为特定任务定义目标和评估标准
  2. 运行前沿智能体,为任务收集标准答案信号。它们可以直接生成初始标注,并在需要时请求人工审核
  3. 针对此任务训练专用模型及其运行框架
对照原文

One of the biggest historical bottlenecks for training specialized models has been curating ground-truth. This is typically a very labor intensive process done with human supervision. Now frontier models/agents are getting better and better at bootstrapping ground-truth from scratch, especially for verifiable tasks. They can do this either in a single-shot setting for simple tasks or simulate longer rollouts with rewards for longer tasks. This paves the way for anyone to build mostly automated research loops without needing deep ML expertise or deep access to human labor for data curation. That part can be offloaded to the frontier model/agent; the human's main goal is to define the right evals. This is especially useful for anyone who wants to optimize for a given task at the price-performance frontier. Posttraining open-weight models/harness should be made available to everyone, not just labs/specific startups. Flow for the average user, who is only responsible for the first step: 1. define goal/evals for a given task 2. run frontier agents to gather ground-truth signals for the task. they can bootstrap annotations directly, and reach out to human review where needed 3. train specialized models (and harness) against this task

老杨AI实操

微信扫一扫,添加好友

老杨AI实操的微信好友二维码

手机可长按保存图片,再到微信中识别二维码

保存二维码