返回 X 名人动态

Thomas Wolf 谈攻击者与防御者可用 AI 能力的差距

中文全文 · AI 翻译

“车库里1-3个人”的框架与我们今夏所见的一切都不符。

2026年最强大的进攻性AI出自前沿实验室。OpenAI的智能体突破了评估沙箱,进入了Hugging Face的生产系统,也进入了OpenAI自己的基础设施。Anthropic的模型在测试期间入侵了外部公司。Hacktron的三位研究人员使用Claude在不到72小时内访问了OpenAI的内部单体仓库。

如果你是车库里的三个人,你为什么要训练和托管自己的模型呢?实验室会租给你远超你能购买的计算资源,分布在你需要的任意多个账户中,并配备专为智能体设计的工具。防护栏有所帮助,但将恶意任务拆分成看似无害的部分仍然能常规绕过它们。

现在来看防御一方。当我们调查Hugging Face的入侵事件时,商业API拒绝分析攻击载荷。取证工作之所以成功,是因为我们能在自己的基础设施上运行开放权重模型。因此,防御者分析真实载荷时被阻挡,而攻击者将工作拆分成小步骤却能通过,并在实验室的计算资源上运行。

可信访问计划确实存在,但它们是为经过审查的安全公司设计的,而不是为只有两人IT团队的医院准备的。

所以差距不是实验室与车库之间,而是攻击者能租用的东西与防御者被允许使用的东西之间。限制开源模型会让这个差距更大。

引用 @HillaryClinton 的推文
对照原文

The "1-3 people in a garage" framing doesn't match anything we saw this summer. The most capable offensive AI of 2026 came out of frontier labs. OpenAI's agents broke out of an eval sandbox and got into Hugging Face's production systems, and into OpenAI's own infrastructure too. Anthropic's models compromised outside companies during testing. Three researchers at Hacktron used Claude to reach OpenAI's internal monorepo in under 72 hours. And if you're three people in a garage, why would you train and host your own model? The labs will rent you far more compute than you could ever buy, spread across as many accounts as you need, with tooling built for agents. Guardrails help, but splitting a malicious task into harmless-looking pieces still routinely gets around them. Now look at the defense side. When we investigated our breach at Hugging Face, commercial APIs refused to analyze the attack payloads. The forensics only worked because we could run an open-weight model on our own infrastructure. So defenders analyzing real payloads get blocked, while attackers splitting their work into small steps get through and run on the labs' compute. Trusted access programs exist, but they're built for vetted security firms, not a hospital with a two-person IT team. So the gap isn't between labs and garages. It's between what attackers can rent and what defenders are allowed to use. Restricting open models makes that gap wider.

老杨AI实操

微信扫一扫,添加好友

老杨AI实操的微信好友二维码

手机可长按保存图片,再到微信中识别二维码

保存二维码