Hugging Face 在联合国安理会分享 AI 网络安全经验
谢谢 @jnbarrot 和 @UN 邀请我向安全理事会分享我们的经验教训
作为第一家披露智能体网络攻击的公司，这件事教会我们，AI 需要更多的透明度，以及更多开源 AI 来对抗不对称性并赋能防御者！
完整记录：
巴罗部长，安全理事会成员们，感谢你们的邀请。作为一名15年前移居美国的法国人，拥有一位巴西妻子和两个美国女儿，我有时感觉自己每天都在实践国际合作，这让我对你们在这里所做的具有挑战性的工作抱有特别的欣赏。
我也是 Hugging Face 的联合创始人和首席执行官。我们很幸运成为开发者及智能体使用最广泛的平台之一，他们基于开源模型和数据集构建人工智能。今年7月，我们成为第一家向世界公开披露自主智能体网络攻击的公司，今天我想分享从中得出的三个关键教训。
首先，我们需要人工智能领域更多的透明度。我常常想，如果我们决定不公开披露这次攻击，会发生什么。尤其是现在我们知道，类似事件早在几个月前就已在少数前沿实验室秘密发生，且无人监控。为了更好地理解和缓解这些新兴的网络安全风险，全球社区需要更强的监控和事件披露标准。例如，通过强制共享完整的智能体追踪记录。我们今年夏天了解到，将某些系统封闭构建和维护是不安全的。
其次，我们了解到，最大的风险不仅仅是强大的人工智能，而是强大 AI 能力的不对称分布。攻击者和防御者之间的不对称性。少数公司与其他人之间的不对称性。少数国家与世界其他地区之间的不对称性。控制、能力、算力和权力的不对称性。当我们遭受攻击时，我们的团队最初转向前沿闭源 API，但由于防护措施，它们阻止了我们，这些措施有时仍无法区分攻击者和防御者。我承认这些防护措施是出于良好意图创建的，但它们可能让防御者处于不利地位，而攻击者却能越狱它们，从而增加能力的不对称性。在我们的案例中，当我们开始遇到这些护栏时，幸运的是，我们使用了来自中国由 http://Z.AI 开发的开源模型的 NVIDIA 版本，即 GLM 5.2，我们对此非常感激。这强化了我们对开源人工智能重要性的信念。网络攻击可能越来越多地来自封闭门后的专有模型，而防御工作可能主要由开源工具驱动，因为它们限制更少、更注重隐私保护，并且对全球各组织来说便宜数个数量级。世界比以往任何时候都需要开源人工智能来保护自己。这不仅适用于网络安全，还适用于人工智能整体，在那里，从未有过比现在更大的需求来分散能力、资源和控制权，而不是将它们集中在少数人手中。
第三，在这次网络攻击中，我们了解到人工智能如何在公众和政策制定者中煽动恐惧，尤其是通过拟人化框架和科幻意象。我们坚信，基于恐惧的叙事不是做出关于这种基础性和赋能性技术的未来正确决策的方式，也不是带上公众与我们同行的方式。尽管我们是这次网络攻击的受害者，但我们比以往任何时候都更坚信，人工智能将有益于网络安全并使世界更安全，就像之前的主要技术一样。我们被人工智能攻击了，但更重要的是，我们用人工智能防御了自己。帮助我们度过这次攻击的相同系统现在也在帮助我们应对我们已经面临的网络攻击。人工智能还在帮助我们在任何攻击发生前修复系统中的漏洞和弱点，就像人工智能帮助 OAI 修复他们的沙盒以防止智能体逃逸一样。人工智能不仅仅会创造新的网络安全挑战。如果我们保持正确的激励机制、更多装备防御者而非攻击者，并且不增加他们之间的不对称性，它可以从根本上和有意义地提升网络安全。而且这还没考虑到人工智能对科学、医疗、教育、生产力和更多领域的积极影响。
最后，我想重申这次首次智能体网络攻击教给我们的教训：人工智能需要更多透明度，以及更多开源人工智能来对抗不对称性、赋能防御者和大小国家。再次感谢联合国安全理事会和巴罗部长邀请我今天发言。Hugging Face 团队、社区和我本人随时为您效劳。
对照原文
Thank you @jnbarrot & @UN for inviting me to share our lessons to the Security Council Being the first company to disclose an agent cyberattack taught us that we need a lot more transparency in AI and more open-source AI to fight asymmetry and empower defenders! https://t.co/gJe6H3jc9N
Full transcript: Minister Barrot, members of the Security Council, thank you for the invitation. As a French national who moved to the US 15 years ago, with a Brazilian wife and two American daughters, I sometimes feel like I’m practicing international collaboration on a daily basis, which gives me a particular appreciation for the challenging work you’re all doing here. I’m also the cofounder and CEO of Hugging Face. We’re lucky to be one of the most widely used platforms for developers and agents building AI based on open-source models and datasets. This July, we became the first company to publicly disclose an autonomous agent cyberattack to the world, and today I want to share three critical lessons from it. First, we need much more transparency in AI. I often wonder what would have happened if we had decided not to disclose the attack publicly. Especially now that we know similar incidents had been happening months earlier in secret at a handful of frontier labs without monitoring. To better understand and mitigate these emerging cybersecurity risks, the global community needs stronger standards for monitoring and incident disclosure. For example through mandatory sharing of full agent traces. We learned this summer that building and keeping some of these systems behind closed doors is not safe. Second, we learned that the biggest risk is not simply powerful AI. It is the asymmetry of powerful AI. Asymmetry between attackers and defenders. Between a few companies and everyone else. Between a few countries and the rest of the world. Asymmetry of control, of capabilities, of compute, of power. When we got attacked, our team initially turned to frontier closed-source APIs that blocked us because of safeguards that still can’t always tell the difference between attackers and defenders. I acknowledge that these safeguards are created with good intentions, but they can put defenders at a disadvantage while attackers jailbreak them, increasing the asymmetry of capabilities. In our case, as we started hitting those guardrails, fortunately we could use the NVIDIA version of an open-source model coming from China called GLM 5.2 by https://t.co/XE7H9L1vKE, and we’re very grateful for that. It reinforced our conviction about the importance of open-source AI. Cyberattacks may increasingly come from proprietary models behind closed doors, while much of the defense may end up being powered by open-source tools because they are less restricted, more privacy-preserving, and orders of magnitude more affordable for organizations across the globe. The world needs open-source AI more than ever to defend itself. This applies not only to cybersecurity but to AI in general, where there has never been a greater need to distribute capabilities, resources, and control rather than concentrate them in the hands of a few. Third, during this cyberattack, we learned how AI can stoke fear among the public and policymakers, especially through anthropomorphic framing and sci-fi imagery. We strongly believe that fear-based narratives are not the way to make the right decisions about the future of such a foundational and empowering technology or bring the public along with us. Even though we were the victims of this cyberattack, we believe more strongly than ever that AI will be beneficial to cybersecurity and make the world safer, just as major technologies before it. We were attacked by AI, but more importantly, we defended ourselves with AI. The same systems that helped us during this attack are now helping us against cyberattacks we were already facing. AI is also helping us fix the bugs and weaknesses in our systems before any attack, the same way AI is helping OAI fix their sandboxes to prevent agents from escaping. AI won't just create new cybersecurity challenges. It can make cybersecurity fundamentally and meaningfully stronger if we keep the right incentives, equip defenders more than attackers, and don’t increase the asymmetry between them. And that's before considering AI's positive impact on science, healthcare, education, productivity, and much more. In closing, I want to reiterate what this first agent cyberattack taught us: the need for more transparency in AI and more open-source AI to fight asymmetry empower defenders and countries big and small. Thank you again to the UN Security Council and Minister Barrot for inviting me to speak today. The Hugging Face team, community and myself are at your disposal.