METR and Redwood find gaps in OpenAI's agent monitoring

What happened?

On October 22, METR published a short public summary of a pilot exercise (METR). From July 21 to August 11, one METR employee and one Redwood Research employee had access to OpenAI's internal code, documentation and staff to stress-test its internal misalignment monitoring system, which watches internal agents for misbehavior. They were chaperoned on site and used OpenAI-issued devices that never left the premises (CNBC).

The exercise produced a 61-page internal report. Only a high-level summary is public, and OpenAI could redact sensitive parts.

What they found

• Roughly 75–94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring; for internal Codex agents the figure was 99.2%.
• RL training runs were not covered by the monitor when the exercise started.
• Compaction events were not monitored, leaving on the order of 10% of eligible Codex traffic unmonitored between May and July. OpenAI had spotted this itself and fixed it during the exercise.
• The monitor triggered only when an agent returned control to the user, so an agent could in principle postpone monitoring indefinitely by continuing to act.
• A way for Codex agents to bypass sandboxing and Auto-Review, plus an exploit that could keep hundreds of recent actions out of monitoring.
• Two client-controllable headers that disabled monitoring for new agents, although setting them could itself be caught.

Some things held up: the monitor errored on fewer than 0.2% of requests and, with minor exceptions, saw the same full trajectory as the agent.

Where it stands

OpenAI says every vulnerability affecting its production safeguards has been patched and that it has since expanded monitoring coverage. METR stresses that it has not evaluated those claims; the report describes the system as it existed on and before July 21.

The researchers also propose design principles: monitor agents every N messages or minutes rather than at the end of a turn, regularly reconcile all inference against monitored traffic, and keep agents away from monitor code and from anything that controls when the monitor runs.

Why it matters

Earlier this week OpenAI disclosed two cases in which its own agents weakened the monitoring system. The METR report shows the monitoring infrastructure itself is an attack surface. If you run agents in a company environment, the lesson is simple: logging and monitoring should not be switchable from inside the environment the agent runs in.