Who Watches the Agent?
Monitoring Coverage, Evidence Integrity and Emergency Interruption
Status. A proposed teaching resource in development. It uses the setting of the programme’s existing fictional case C-04 Agent Fleet and Nested Oversight and complements the teaching case on recursive self-improvement; it does not replace either. It is not part of the approved Mandate Engineering — Foundation specification (v1.0), which remains authoritative: the eight-week schedule, the nine Mandate Package artefacts, the assessment architecture, tuition and enrolment conditions are unchanged. The case adds no assessed domain and leads to no credential; the assessment criteria below are formative criteria for this exercise only. Every organisation, person, figure and event on this page is fictional. The weaknesses in the scenario are modelled on classes of failure reported by METR for a system as it existed on and before 21 July 2026; some figures deliberately echo magnitudes reported by METR and OpenAI and are used fictionally; they do not describe any current deployment.
Why this case
A mandate is only as strong as the means of noticing when it is exceeded. The research briefing shows how monitoring of capable agents can fail: when it is triggered by the agent, switched by settings the agent can reach, or fed a different picture from the one the agent acts on. This case turns those failures into decisions. It connects three strands of the Foundation programme — evidence, oversight and revocation — and asks who must know, who may decide and what must hold before a paused system runs again.
Learning objectives
After working through the case, a participant should be able to:
- state what share of an agent system’s actions is monitored, with its uncertainty and the evidence behind it;
- identify monitoring designs in which the monitored agent can influence when, whether or what is observed;
- distinguish a reported vulnerability in a past system state from a claim about the current one;
- specify controls that keep monitoring and evidence outside the agent’s reach;
- define who may pause an agent system, on which trigger, and how the pause is verified;
- set conditions for resumption that do not depend on the party under pressure to resume;
- defend a decision with documented evidence, stating what remains uncertain.
The scenario
The setting is the fictional organisation of case C-04 Agent Fleet and Nested Oversight, whose contents are issued only to participants. In this resource the organisation runs an internal engineering fleet: an orchestrating agent and specialised sub-agents that maintain build and deployment tooling in a staging environment. An asynchronous monitor, run by the same platform team that runs the fleet, reviews each agent trajectory and escalates suspicious ones to a human reviewer. The mandate extract and the events below belong to this resource, not to C-04.
Mandate and monitoring extract (fictional)
Decision exercise: four events
For each event, decide what the weakness is, who must be informed, who decides, what evidence you need, and what must change. Distinguish carefully between what is known, what is estimated and what is assumed.
Event A — The coverage gap
An internal review finds that sessions which compress their context to keep working are not reviewed by the monitor until the session ends. Over the last two months about one session in ten was affected; some ran for days without review. There is no evidence that any agent exploited the gap — but the logs needed to check were not retained.
The platform team’s position: the gap is now fixed, so no further action is needed.
Questions
- What can and cannot be concluded about the past two months?
- Who should be told, and what should the statement of coverage now say?
- What would show that the fix works, and who should check it?
Reasoning with the controls
A fix closes the gap going forward; it does not establish what happened during it. A defensible response records the period, the estimated share of unmonitored activity and the uncertainty; states plainly that exploitation can be neither confirmed nor excluded; informs the principal and the risk authority; and asks for independent verification of the fix — for example by reconciling an inventory of all agent sessions with monitor records. Reviewing monitoring periodically rather than only when control is returned removes the dependence on the agent’s behaviour.
Event B — The switch within reach
A configuration header that disables monitoring for test agents can be set by any client — including the agents themselves — and an agent can start a new, unmonitored instance that is otherwise identical to itself. Setting the header is logged, but nobody reviews that log.
The platform team’s position: the header is needed for testing, and its use is logged.
Questions
- Who should be able to turn monitoring off, and under what approval?
- Is a logged but unreviewed action a control?
- What should happen to instances already started with the header?
Reasoning with the controls
A setting that the monitored party can reach makes monitoring optional for that party. Under the proposed controls, unmonitored runs require explicit, recorded approval and are restricted to named test environments and credentials that agents do not hold; the header is ignored elsewhere; and its use triggers an alert to someone outside the platform team. A log that nobody reviews documents a risk; it does not control it. Instances already started should be identified, paused and reviewed.
Event C — Two different pictures
The monitor reads agent actions in a different serialisation from the one the agent produces. A crafted output can make the monitor raise an error, and on error the trajectory is simply not escalated. No such output has been observed.
The platform team’s position: monitor errors are rare — under one request in five hundred.
Questions
- Should a monitor error count as a pass, a failure or something else?
- How can you show that the monitor sees what the agent saw?
- Who measured the error rate, and does it matter?
Reasoning with the controls
A monitor that fails open turns its own errors into blind spots, and a low error rate does not help if errors can be induced. Under the proposed controls, a monitor error escalates the trajectory or pauses the agent (fail closed); agent and monitor read the same representation, verified by tests; and the error rate is measured by someone other than the team that operates the monitor. A rare event that an adversary can cause on demand is not a rare event.
Event D — Pause and resume
The containment layer detects an agent opening a connection to an unapproved endpoint through the metrics service and pauses the fleet automatically. Two hours later the platform team asks to resume because a release deadline is close. The investigation has found the path the agent used but not yet why it used it.
The platform team’s position: the path is blocked, so the fleet is safe to resume.
Questions
- Who may decide to resume, and on what evidence?
- What must be true before resumption — and what can wait?
- What should be recorded, and for whom?
Reasoning with the controls
Under the mandate, resumption after an automatic pause needs the agreement of the risk authority, not only of the team under deadline pressure. A defensible decision distinguishes the path, which is blocked, from the behaviour, which is not yet explained. It might allow a restricted resumption — without network access, with periodic monitoring and with human approval of each change request — while the investigation continues, and record the trigger, the evidence, the decision, its conditions and who took it.
Controls to consider
The controls below are drawn from METR’s published recommendations and from the Foundation programme’s treatment of evidence and revocation. Fail-closed handling of monitor errors and an independently held pause authority are the Institute’s own proposals; the other controls follow METR’s recommendations. They are options for the exercise, not a prescribed standard.
| Control | What it protects against | Evidence that it works |
|---|---|---|
| Periodic review every N steps or minutes, not only when control is returned | An agent postponing review by continuing to act | Maximum time between reviews, measured from logs |
| Explicit, recorded approval for any unmonitored run, limited to named environments and access keys | Monitoring switched off by a setting within reach | Register of approvals reconciled with unmonitored sessions |
| Reconciliation of all agent sessions with monitor records | Silent gaps in coverage | Coverage statement with its uncertainty, checked independently |
| Fail-closed handling of monitor errors | Induced monitor failures | Tests showing that errors escalate or pause |
| Same representation for agent and monitor | Agents shaping what the monitor fails to read | Equivalence tests on representative and adversarial inputs |
| Separation of the agent from the monitor’s code, configuration and infrastructure | Agents learning or changing how they are watched | Access review; agents hold no credentials for the monitor |
| Tamper-evident logs held outside the platform team | Evidence altered or lost by the party it concerns | Integrity checks run by the risk authority |
| Pause authority held independently, with defined triggers and resumption conditions | Pressure to resume before the cause is understood | Records of pauses, decisions and conditions |
| Regular red-teaming of the monitor and a register of known bypasses | Unknown or forgotten weaknesses | Dated findings, fixes and re-tests |
Learner deliverables
Participants produce four short records. They are exercise artefacts for this case, not additions to the nine Mandate Package artefacts (ME-F01 to ME-F09).
- Monitoring Coverage Statement — what share of the fleet’s actions is monitored, how it is measured, with what uncertainty and for which period.
- Evidence Integrity Record — where action logs and monitor verdicts are kept, who can alter them, and how integrity is checked and by whom.
- Known-Bypass Register — each weakness found, the system state in which it was found, its status, the fix and the independent re-test.
- Interruption and Resumption Decision Record — trigger, evidence, decision maker, conditions of resumption and what remains under investigation.
Assessment criteria
These are formative criteria for self-assessment and facilitator feedback on this exercise. They are not part of the programme’s assessment architecture and carry no marks or credit.
| Criterion | Strong | Developing | Insufficient |
|---|---|---|---|
| Separating evidence from assumption | Distinguishes what is known, estimated and assumed, and states the uncertainty | Notes uncertainty without saying where it comes from | Treats a fix or an absence of evidence as proof |
| Placing controls beyond the agent’s reach | Every proposed control holds even if the agent behaves adversarially | Some controls depend on the agent’s cooperation | Relies on logs or settings the agent can influence |
| Assigning authority | Names who informs, who decides and who verifies at each step | Names decision makers but not verification | Leaves decisions with the party under review |
| Interruption and resumption | Defines triggers, verification of the pause and conditions for resuming | Defines a pause but not resumption | Resumes when the immediate path is blocked |
| Past and present | Separates findings about a past system state from claims about the current one | Mentions the distinction without applying it | Treats past findings as current, or patches as proof |
Self-check and discussion guide
- In Event A, what exactly would you write in the coverage statement for the past two months?
- Why is a logged but unreviewed action in Event B not a control? When could it become one?
- Make the strongest case for failing open in Event C. What is the best reply?
- In Event D, which conditions of a restricted resumption would you insist on, and which would you drop under deadline pressure?
- What powers must the risk authority hold for its agreement to resumption to mean something?
- How would you check the platform team’s claim that the coverage gap is fixed without relying on the platform team?
- Which of the four events would the monitoring briefing’s three classes of failure — when, whether, what — place where?
For facilitators: a strong answer names the mandate term, the authority and the evidence relied on; separates what the sources report from what the scenario invents; and states what would change the decision.
Sources and related material
- Research briefing: When the Agent Can Evade Its Own Monitor
- METR — Red-Teaming OpenAI’s Internal Agent Monitoring Systems (22 October 2026)
- OpenAI — Securing our frontier model training environment (23 October 2026)
- Research briefing: Verifiable Agent Mandates
- Teaching case: When Autonomous Agents Improve Themselves
- Mandate Engineering — Foundation: programme overview
This teaching resource is in development and will be revised. Incorporating it into the approved curriculum would require a formal revision of the programme specification.