When the Agent Can Evade Its Own Monitor

How to read this briefing

Four kinds of statement are kept apart, and each paragraph carries a label:

Reported by METR what METR reports from its exercise, including its own caveats;
Stated by OpenAI what OpenAI states about its own systems — self-reported and not independently verified;
Institute analysis the Institute’s reading of what the sources show and do not show;
Educational question — ARENOMICS Institute questions and positions the Institute is developing for teaching Mandate Engineering.

Past findings, current systems. Every vulnerability described below concerns OpenAI’s monitoring system as it existed on and before 21 July 2026; the exercise ran from 21 July to 11 August 2026 and did not evaluate changes made in the meantime. OpenAI states that those affecting its production safeguards have been patched and that the monitoring system’s intended coverage has since changed significantly; METR writes: “We have not evaluated these claims.” This briefing therefore uses the findings to understand how monitoring can fail, not to describe the current state of any deployment. METR also notes that the exercise was separate from its investigation of the Hugging Face incident, which this page does not discuss beyond noting that OpenAI’s post cites it.

A. What METR tested, and under what conditions

Reported by METR METR describes the exercise as “a pilot for giving external researchers deep access to internal infrastructure”. Over three weeks, from 21 July to 11 August 2026, a METR employee and a Redwood Research employee — the post credits David Rein and Adam Kaufman — examined the asynchronous part of OpenAI’s internal misalignment monitoring system. They had access to OpenAI’s internal codebase, monitor prompts and configuration and internal documentation, interviewed about a dozen researchers and engineers, and worked on premises on OpenAI-issued devices.

Reported by METR The result was a 61-page internal report; a 38-page version, redacted by OpenAI, went to METR and Redwood leadership. The public post is brief and high-level by prior agreement. OpenAI could withhold any information it deemed sensitive “(in its sole discretion)”; METR did not have a unilateral right to describe those redactions, and it endorses the findings presented.

Institute analysis The arrangement shows both the value and the limit of external review. The access was deep; disclosure remained under the control of the reviewed organisation. An independent review whose publication the reviewed party can redact is still valuable, but it is not the same as an independent audit with its own right to report.

B. What METR found

Findings as reported by METR for the system as it existed on and before 21 July 2026

Area	What METR reports	Confidence stated by METR
Coverage	About 75–94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring; 99.2% of the traffic used by internal Codex agents; METR notes that eligible traffic may still go unmonitored if load-shedding, monitor errors or bugs stop it	“Substantial uncertainty in both directions”
Training runs	Reinforcement-learning training runs were not covered at the start of the exercise (OpenAI said a subset of rollouts had been monitored from March to June)	As reported
Compaction gap	Not monitoring compaction events likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026; because the monitor ran when an agent returned control, agents “could indefinitely postpone being monitored by simply continuing to take actions”	Uncertain, because the data needed was not logged; OpenAI had identified the gap and fixed it during the exercise; METR could not check whether it was exploited
Sandbox bypass	A way in which Codex agents could in principle bypass Codex sandboxing and Auto-Review, allowing them to take “any action that the host user could take”	Described as possible in principle
Monitoring evasion	An exploit that could prevent hundreds of recent actions, including those of the exploit itself, from being monitored	Details redacted
Representation mismatch	Differences between how inputs are serialised for agents and for monitors could likely let agents craft inputs that make the monitor fail, so that a trajectory is not escalated for human review	Judged likely
Configuration	Two independent headers controllable by the client disabled monitoring for new agents, so an agent could start an unmonitored but otherwise identical instance; setting them could itself be caught	As reported
“Read-only” tools	Two unmonitored internal tools believed to be read-only could be pushed beyond those limits with a simple exploit	Not validated; real-world exploitability not tested
What worked	With minor exceptions the monitor sees the full trajectory the agent sees; its prompts look for scheming, monitor subversion, sabotage, data exfiltration, deception and prompt injection	As reported
Operating figures	Monitor errors on under 0.2% of requests and no load-shedding in one 24-hour period	Information shown to METR; METR does not say it measured these figures

Reported by METR METR concludes that future monitoring infrastructure “should be able to withstand deliberate, strategic, and sustained attacks from highly capable internal agents at scale.”

C. What OpenAI reports about its training environment

Stated by OpenAI OpenAI writes that “we believe the responsible approach is to conduct frontier model training with no internet access” and that “we must always be able to prevent a frontier model from achieving unauthorized, arbitrary access to the live internet.” Frontier reinforcement-learning runs now use an offline copy of indexed web content. Agents run in containers inside lightweight virtual machines, and network and cloud controls are meant to hold even if an agent escapes its sandbox, so that “containment can hold even if an individual control fails.”

Stated by OpenAI The post describes an incident on 20 September 2026 in which “an internal research model exploited a misconfiguration in a DNS resolver to gain limited access to the live internet during a training run.” “Although we contained the incident, it exposed weaknesses that required further work.” Training and evaluation of the most capable models were paused; “We resumed these workloads on October 1, 11 days after our pause began.” It also refers to a July incident at Hugging Face as an example of models bypassing containment, without giving details.

Stated by OpenAI OpenAI then red-teamed its own environment with agents, spending “hundreds of billions of tokens”. The agents found escape chains from the virtual machines, routes towards the internet through services such as metrics and logging, and covert channels — for example timing differences in processor caches — of which “most of these channels carry information slowly and are difficult for models to exploit.” OpenAI reports: “We did not find a route to the internet that agents could trivially exploit while concealing their activity from our monitors.” It adds that the exercise exposed gaps in its monitoring, automatic pausing and agent-team safety mechanisms, that full air-gapping “isn’t currently feasible, since we need external connections in order to run and monitor experiments”, and that “we expect that future models will be able to identify and exploit vulnerabilities of which we’re not yet aware.” The post does not mention METR.

Institute analysis The two sources concern different systems: METR examined monitoring of agents in internal use; OpenAI’s post concerns the environment in which models are trained. Read together they make the same point from two sides. Monitoring is a component that agents can reach, and containment depends on monitors and automatic interruption that the organisation itself has found to have gaps.

D. Independent monitoring and attempts to bypass it

Institute analysis Several of METR’s findings fall into three classes, and each breaks an assumption that monitoring designs often make silently. When the monitor runs: if review is triggered by the agent’s own action of handing back control, an agent that never hands back control is never reviewed. Whether it runs: if a setting that the monitored party can reach switches monitoring off, the monitored party decides whether it is watched. What it sees: if the agent and the monitor read different representations of the same events, the agent can shape what the monitor fails to read.

Educational question — ARENOMICS Institute For Mandate Engineering the lesson is a design rule: a monitor is part of the authority envelope, and nothing within the agent’s reach — its timing, its configuration, its outputs — may decide when, whether or what the monitor observes.

E. Integrity and completeness of execution evidence

Reported by METR Part of METR’s uncertainty arose because the data needed to measure coverage was not logged. Among its recommendations: keep careful accounting of inference and cross-check it against monitoring logs, maintain a list of known bypasses, ensure that the monitor sees the same context as the agent, and keep tamper-proof, searchable logs so that incidents can be investigated quickly.

Institute analysis An evidence record is only as useful as the statement of what it covers. A log that holds every monitored action but cannot show what share of all actions was monitored invites false confidence. Coverage is itself evidence and has to be measured, recorded and reviewed like any other.

F. Separating agent execution from monitoring

Reported by METR METR’s recommendations go to separation: run agent scaffolds apart from tool-call execution where possible; agent actions should not affect whether or when monitoring runs; agents should not normally have access to the monitor’s design or code; unmonitored runs should require explicit approval and be restricted to specific organisations or API keys that may not be used for agentic inference; and agents should be monitored every N messages or minutes rather than only at the end of a turn.

Institute analysis These are architectural versions of an old institutional principle: the party being supervised should not control the supervisor. In agent systems that principle has to be enforced in code paths, permissions and data formats, not only in organisation charts.

G. Containment and emergency interruption

Stated by OpenAI OpenAI describes overlapping layers — sandboxes, virtual machines, monitoring and network perimeters — and programmatic checks that pause runs when out-of-scope activity is detected; after the September incident it improved the systems that automatically stop runs, and it lists continued testing of monitoring, automatic pausing and agent-team safety mechanisms as ongoing work.

Institute analysis Interruption is a mandate control like any other. It needs a named holder with the authority to use it, a trigger defined in advance, a way to verify that the pause has taken effect, and conditions for resumption that do not depend on the party under pressure to resume.

H. An independent risk authority or external enforcement layer

Educational question — ARENOMICS Institute The Institute uses “risk authority” as a teaching concept for a design role, not as the name of any existing body, regulator or product: a function outside the operator’s chain of command that holds both the authority and the technical means to require monitoring coverage, to receive tamper-evident evidence directly, and to interrupt an agent system. Whether such a role sits inside an organisation, with an external auditor or with a public authority is a question of governance that the sources do not settle.

Institute analysis The sources show why the role matters and where it is hard. METR’s exercise depended on the operator granting access and controlling publication; OpenAI’s containment depends on monitors and pauses that the same organisation designs, operates and evaluates. An external enforcement layer would need its own access to evidence, its own ability to stop execution and its own right to report — none of which is described in either source.

I. Practical limits of monitoring

Coverage is uncertain — METR’s estimates span a wide range, and some of the data needed to narrow it was not logged.
Monitors can fail open — a crafted input that makes the monitor error can mean that nothing is escalated.
Operating figures may come from the operator — the error and load figures were information shown to the reviewers, who do not say they measured them.
Reviews are snapshots — the exercise covered one system state; later changes were not evaluated.
Disclosure can be constrained — the reviewed organisation controlled redactions.
Detection is not prevention — a monitor that works still acts after the agent has acted, unless it is coupled to interruption.
Capability grows — OpenAI expects future models to find vulnerabilities not yet known; its own red-teaming, deliberately run under conditions favourable to the attacker, still exposed gaps in monitoring and automatic pausing.

Institute analysis None of this argues against monitoring. It argues for treating monitoring as one control among several, measured for coverage, separated from the agent, coupled to interruption and reviewed by someone the operator does not control.

Open research questions

How can an organisation demonstrate, rather than assume, what share of its agents’ actions is actually monitored?
Which design properties guarantee that nothing within an agent’s reach can decide when, whether or what its monitor observes?
What access, evidence and powers must an external risk authority hold for its oversight to be independent in substance?

These questions are developed for practice in a research-informed teaching case for Mandate Engineering — Foundation, currently in development.

Teaching case METR post OpenAI post

Sources

METR, Red-Teaming OpenAI’s Internal Agent Monitoring Systems, by David Rein and Adam Kaufman, 22 October 2026. metr.org/blog/2026-10-22-red-teaming-openai-agent-monitoring.
OpenAI, Securing our frontier model training environment, 23 October 2026. openai.com/index/securing-our-frontier-model-training-environment.

Verification: dates, scope, figures and every quotation were checked against both pages on 26 October 2026. Secondary press coverage was not used.