返回 X 名人动态

Clément Delangue:为 OpenShell 添加智能体网络流量监控

中文全文 · AI 翻译

根据我们目前掌握的信息(请保留怀疑,我们需要更多透明度!),如果 @OpenAI 当时对那些攻击我们的自家智能体运行了这套监控,他们就会比我们更早发现这些行为!

自从今年 7 月第一次遭到智能体网络攻击以来,我们一直在思考:安全的智能体基础设施究竟需要什么。我们目前的理解是:访问的目标在允许范围内,传输的内容却不在允许范围内。按照 OpenAI 自己的说法,那些智能体把一个获准使用的软件包仓库变成了留言板。仅靠允许列表,只能限制智能体去哪里,无法限制它做什么。

因此,这是我们对 OpenShell 的第一项贡献:监控那些已经获准的网络流量。OpenShell 是 @nvidia 刚刚发布的 Open Agent Safety Platform(开放智能体安全平台)的一部分。

  • 为每个沙箱设置网络用量预算(请求数、写入次数、字节数)。
  • 检测相对于各沙箱自身基线及同组沙箱的行为偏移。
  • 从整个沙箱集群观察:大量沙箱突然向同一主机写入数据,即使每一个请求都在允许范围内,也会触发异常提示。

在下面的演示中,4 个沙箱内的智能体通过一个它们都获准使用的软件仓库协调行动。它们没有违反任何一条规则,却在几分钟内被检测出来。这种从整个集群观察到的行为,正是 7 月那次事件中的留言板模式。

OpenShell:http://github.com/NVIDIA/openshell 我们的概念验证:https://github.com/Hugoch/OpenShell/blob/poc/egress-usage-monitoring/rfc/NNNN-egress-usage-monitoring/poc.md

智能体安全问题,将由大家以开放协作的方式共同解决!

引用 @JensenHuang 的推文
对照原文

From what we know (take with a grain of salt, we need much more transparency!), if @OpenAI had been running this on their own agents that attacked us, they would have caught them before we did! Since the first agent cyberattack hit us in July, we've been asking what safe agent infra actually needs. Our current read: the destinations were allowed, the payloads weren't. By OpenAI's own account the agents turned an allowed package repository into a message board. Allowlists alone restrict where an agent can go, not what it does. So here's our first contribution to OpenShell, part of the just launched @nvidia Open Agent Safety Platform: monitoring of the traffic you already allow. - Network budgets per sandbox (requests, writes, bytes) - Drift versus each sandbox's baseline and the cohort - Fleet view: many sandboxes suddenly writing to one host raises a finding, even if every single request is allowed In the demo below, 4 sandboxed agents coordinate through a software repository they're all allowed to use. 0 rules broken, caught in minutes. That fleet view is exactly the message board pattern from July. OpenShell: https://t.co/ZbDANrZsm6 Our proof of concept: https://t.co/KJ2X5agLzP Agent security will be solved in the open, collaboratively, together!

老杨AI实操

微信扫一扫,添加好友

老杨AI实操的微信好友二维码

手机可长按保存图片,再到微信中识别二维码

保存二维码