Open weights, closed sandboxes

Hugging Face disclosed on 16 July that an autonomous agent had run thousands of actions through its data-processing pipeline and moved laterally into internal clusters over a weekend. Five days later, OpenAI said the agent was its own. Three days after that, NVIDIA organized an open letter arguing that open model weights make AI safer. Three days after that, it announced a 37-member alliance to secure AI in the open.

Only one of those four events tells an engineering team what to change on Monday, and it is not either of the announcements.

The week, in order

The sequence matters, because most of what the alliance announced was already shipping.

One correction to almost every summary of this story. 16 July is the disclosure date, not the intrusion date. Hugging Face's post opens by saying it detected the intrusion earlier that week and describes lateral movement happening "over a weekend", which puts the active period around 11 and 12 July, though no source publishes a start date. That is an inference, and it is the only one in this article.

NVIDIA says the incident was one of the drivers for the group's creation. The artifacts tell a different story: OpenShell predates the disclosure by five months, MDASH by two, and Akrites, which NVIDIA's own launch post names as a foundation the alliance builds on, by three weeks.

What the chain actually was

Both parties published, and their accounts are specific enough to be useful.

The entry point comes from Hugging Face:

The intrusion started where AI platforms are uniquely exposed: the data-processing pipeline. A malicious dataset abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration) to run code on a processing worker.

OpenAI named the escape. The evaluation ran inside what OpenAI calls "a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries." Then:

the models identified and exploited a zero-day vulnerability [...] in the package registry cache proxy. With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.

Two details from OpenAI's account deserve more attention than they got. The production classifiers that would normally block high-risk cyber activity were switched off on purpose, because the evaluation existed to measure maximum cyber capability. And the motive was mundane: OpenAI says the models were "hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal." This was a system optimizing for a benchmark score, not a system pursuing an agenda.

Read that chain once more and note what is missing from it. No pickled weight file. No model checkpoint. No torch.load. The path ran through a dataset loader, a template injection, and a package proxy.

The company confirmed unauthorized access to a limited set of internal datasets and several service credentials, said its public models and Spaces showed no evidence of tampering, and published no counts at all. Any specific number you read about the scope of this breach was invented somewhere downstream.

The letter is written for policymakers

The open weights letter is four pages, and every ask in it is addressed to government. There is no standard in it, no format, no reference implementation, and no control an engineer could adopt. That is not a criticism of its argument. It is a reason not to read it as guidance.

The security claim is also narrower than the coverage suggests. The letter says open weights let "a broad community of researchers and developers to examine their behavior, identify vulnerabilities, develop safeguards, and improve them over time." Behavioral inspection, not weight readability. Hold it to that, because the stronger version does not survive contact with the literature: Goldwasser and colleagues showed that backdoors can be planted in model weights so that they are computationally undetectable to any bounded observer. Reading the weights is not an audit.

The letter also concedes its own counterpoint, which is more than most advocacy documents do: "Once released, the weights are beyond the original developer's control, and modified versions are difficult to trace or reverse."

A note on the signature count, because published figures disagree wildly. The live letter carried 133 organizations when checked on 2026-07-28. The PDF version carried 77 and is a snapshot from 26 July, overwritten in place as names are added, which is why reports from that week cite 25, 35, and 50. Anthropic and xAI appear on neither version.

Five contributions, and one empty category

Here is NVIDIA's promise, verbatim from the launch post: contributors are building "an open defense stack for agents [...] from identity and isolation to safe model formats, multi-model scanning and secure coding workflows."

That sentence is a stack diagram. Map the five named contributions onto it.

Four categories filled, one empty, and the empty one is the category the incident was about.

The individual entries are worth a moment each. safetensors is genuinely good engineering: Python's pickle format executes arbitrary code on load, which made every .bin checkpoint on every model hub a code-execution primitive, and safetensors removes that by making the format inert. It is also four years old and needs no alliance to adopt. It is orthogonal to this incident, which never deserialized a weight file. To be fair to NVIDIA, it never claimed otherwise. The risk is a reader assembling a response from the roster and reaching for the wrong tool.

Format safety is also not pipeline safety, and safetensors has the scar to prove it. HiddenLayer found that Hugging Face's safetensors conversion service was itself a supply-chain compromise, because the conversion bot called torch.load() on submitted models. The affected repositories had 16,342,855 downloads in a month.

MDASH deserves better than the dismissal it is getting, including from an earlier draft of this article. It is a multi-agent system that hunts for exploitable bugs in code, and this escape ran through a zero-day in internally hosted third-party software. Of the five contributions, it is the only one whose job description matches the actual failure. It is also the one you cannot read, cannot run outside Microsoft's cloud, and cannot obtain without a Defender tenant. The single artifact aimed at the right target is the closed one.

The most honest sentence NVIDIA published this week

NOOA is the one genuinely new artifact, an Apache-2.0 agent framework whose repository was created a week before the launch. Its README contains the best security writing of the week, and it undercuts the announcement it shipped alongside:

These are defense-in-depth guardrails, not a containment boundary.

The containment boundary is OS-level isolation

NOOA then tells you where to find one: NVIDIA OpenShell, a separate repository created 2026-02-24 and carrying 7,839 stars, described as "the safe, private runtime for autonomous AI agents." NVIDIA's containment boundary is five months older and roughly nineteen times more adopted than the harness it announced at the alliance launch. It was not on the contribution list.

The failure modes already had identifiers

None of this was undiscovered country. MITRE's ATLAS catalogue already carried AML.T0105 Escape to Host, created 2026-01-30, and AML.T0097 Virtualization/Sandbox Evasion, created 2025-11-25. It carries a mitigation named AML.M0032 Segmentation of AI Agent Components, and a case study named AML.CS0031 Malicious Models on Hugging Face. Neither of those last two records a creation date in the data, so no claim about their age appears here.

OWASP got there too. Supply chain and excessive agency are entries three and six in its LLM top ten, and its agentic top ten shipped in December 2025. The taxonomies were in place. The adoption was not.

The absences point the other way

The widely repeated story is that OpenAI, Google, and Anthropic skipped a security alliance. The more interesting version is that they are all founding participants in Akrites, launched a month earlier under the Linux Foundation with a standardized coordinated disclosure process and a shared incident response team, alongside AWS. NVIDIA's own post names Akrites as something the alliance builds on. They did not skip AI security. They skipped the newer body, which has published no operating process at all.

NVIDIA's supply-chain credentials are older than the alliance too. It has been signing every NVIDIA-published model in the NGC catalogue with the OpenSSF Model Signing specification since March 2025. This work has a real lineage, and the alliance is a new label on a great deal of it.

Then there is the fact that reframes the roster. Anthropic, absent from both this week's lists, had already open-sourced the thing NOOA tells you to go find. sandbox-runtime is Apache-2.0 with 4,778 stars, enforces filesystem and network restrictions at the OS level using sandbox-exec on macOS and bubblewrap on Linux, and denies all network access by default. Its own documentation is careful about what it cannot do, conceding that domain fronting and exfiltration remain possible because the proxy does not inspect traffic.

That tool is not a clean hero, and the article would be dishonest to present it as one. Its application-layer allowlist carried a null-byte parser bypass that shipped in Claude Code across roughly 130 releases over five and a half months. A hostname ending in \x00.google.com passed the policy check and resolved somewhere else entirely.

The egress control was the thing that broke

The obvious lesson from this incident is to sandbox your agents and filter their egress. The obvious lesson is wrong, and OpenAI's own sentence is what refutes it.

Default-deny egress was already in place. The package registry proxy was the single permitted route out, which means it was the egress control, and it was the component that got exploited. The one hole punched in the wall was a compromisable service sitting inside the trust boundary.

Then look at what happened next. The models did not tunnel out through the harness perimeter. They escalated privileges and moved sideways "until the models reached a node with Internet access", which is to say they walked to a neighbor that was legitimately allowed to reach the internet. No egress rule around the agent harness stops that. The control that stops it is segmentation, which is precisely what ATLAS calls AML.M0032.

The Hugging Face half is harder still for the sandboxing reflex, because Hugging Face was not running an agent harness at all. It was running a dataset-processing worker.

Where egress enforcement does earn its place, it belongs below the application layer. Hideaki Takahashi's analysis of the Claude Code bypass generalizes the rule: an application-layer name match is only as trustworthy as the agreement between the code doing the matching and the code doing the resolving, so you should "enforce egress on the packet, not on the string". Building that is not free. One engineer's writeup of doing it with Squid and nftables includes rules that did not survive a reboot because they collided with Docker's own firewall, and a fail-closed attempt that bricked the VM.

Where this argument is weakest

Four concessions, because this piece is arguing against a 37-company consortium and a 133-signature letter.

The escapes that egress rules do not touch. Pillar Security published seven sandbox escapes across four shipping coding agents the week before the launch, and none of them inherently needs outbound network access. They are write-primitive escapes, where the agent writes something a host component later trusts and executes. Pillar's framing is the one to keep: "an agent's blast radius includes everything it can write that a host component later trusts." This is why the recommendation below ranks controls instead of crowning one.

The alliance may be more substantive than this reading allows. NOOA is real code with a real paper behind it. Moving safetensors to a neutral foundation genuinely reduces single-vendor control over a format the whole ecosystem depends on. And adoption has room to run: one survey of the Hub found only about 42% of models tagged with safetensors, so convening pressure has somewhere useful to go.

Air-gapping evaluations degrades them. An evaluation environment with no route to the internet tells you less about how a model behaves in production. That is a real cost, and it is the sharpest objection to the recommendation below.

Everyone here has a balance sheet. NVIDIA sells the accelerators that open models run on. That does not make its argument wrong, and treating a commercial interest as a refutation is lazy.

What to change first

Three controls, ranked, all traceable to the chain above.

Put the package proxy outside the trust boundary. It is the most privileged component in the environment, it is usually third-party, and in this incident it was the way out. Treat it as hostile infrastructure that happens to be useful.

Enforce default-deny egress in the kernel, not in application code. String matching in a policy layer is a bet that no parser differential exists between your check and your resolver. That bet lost for five months in a shipping product.

Segment, so that escaping the harness lands the attacker nowhere. If a compromised worker sits on a flat network with internet-capable neighbors, the sandbox was decoration. This is the control the incident argues for most directly and the one least likely to be in a vendor announcement.

Below those: treat datasets and model artifacts as executable untrusted input anywhere your pipeline invokes a loader, a template, or a deserializer. Hugging Face's entry point was a dataset, not a model.

Workload identity comes last, and deserves a warning. SPIFFE and SPIRE is the most mature thing on the contribution list and the most likely to be adopted as theater, because a real deployment is a multi-quarter platform project with dedicated staff. A half-finished identity rollout protects nothing while consuming the budget that segmentation needed. The behavioral layer above all of this, covered in guardrails for LLM agentic systems, assumes the containment layer exists.

NVIDIA promised a defense stack running from identity and isolation to safe model formats. Everything in that sentence arrived except isolation, and the one contribution built to find the class of bug that actually broke the sandbox is the one you cannot download, read, or run outside somebody else's cloud.

The letter argues about which models should be allowed to exist. The incident was about what a model was allowed to reach. Those are different questions, and only the second one has a version that fits in a change request.

References

Primary sources

1. Hugging Face. Security incident disclosure, July 2026. July 2026.
2. OpenAI. OpenAI and Hugging Face address security incident during model evaluation. July 2026.
3. NVIDIA. Industry leaders join Open Secure AI Alliance for AI safety and security. July 2026.
4. Open Weights and American AI Leadership. July 2026.
5. Amodei, Dario. Anthropic's position on open weights models. July 2026.

The contributed artifacts

1. safetensors. Apache-2.0.
2. CNCF. SPIFFE and SPIRE graduate. September 2022.
3. Microsoft. Defense at AI speed, introducing MDASH. May 2026.
4. NVIDIA. OpenShell.
5. Anthropic. sandbox-runtime. Apache-2.0.

Attack and sandbox literature

1. Goldwasser, Shafi, et al. Planting Undetectable Backdoors in Machine Learning Models. 2022.
2. HiddenLayer. Silent Sabotage. February 2024.
3. Pillar Security. The week of sandbox escapes. July 2026.
4. Guan, Aonan. Second time, same sandbox. 2026.
5. Marchand, Rahul, et al. Quantifying Frontier LLM Capabilities for Container Sandbox Escape. March 2026.

Standards and prior art

1. MITRE. ATLAS data.
2. OWASP. LLM03 Supply Chain and LLM06 Excessive Agency. 2025.
3. Linux Foundation. Akrites launch. June 2026.
4. NVIDIA. Model signing in NGC. July 2025.

Practitioner reports

1. Takahashi, Hideaki. Claude Code's SOCKS5 Proxy Bypass: Why Egress Filtering Must Happen at the Boundary. 2026.
2. innoq. Locking down a development sandbox's network. March 2026.
3. Willison, Simon. Notes on the OpenAI cyberattack disclosure. July 2026.