External Auditors Identify Significant Blind Spots in OpenAI’s Internal Safety Monitoring

As many as 25 percent of internal agentic traffic from OpenAI’s most advanced models may have operated without any safety monitoring during a recent three-week audit. This upper-bound estimate was provided by METR, a research organization that partnered with Redwood Research to conduct an on-site review of OpenAI’s systems for detecting rogue behavior. The figures highlight a significant gap in the oversight mechanisms currently deployed for the company's GPT-5.5 class and above models.

The audit took place from July 21 to August 11, 2026. During this period, researchers from both organizations were granted access to OpenAI’s codebase and internal documentation. They also conducted interviews with approximately a dozen employees to understand how the monitoring infrastructure functioned in practice. The primary objective was to evaluate the effectiveness of the systems designed to catch anomalous or dangerous actions by AI agents before they reached users.

Beyond the broad coverage gap, the auditors identified specific technical failures in the monitoring pipeline. For internal Codex traffic, which is intended to be fully monitored, METR estimates that roughly 10 percent of activity went untracked between May and July. This issue stemmed from a bug in how the system handled compaction events, a process involving context summaries. The researchers noted that the uncertainty surrounding this 10 percent figure is significant, but the gap represented a clear failure in the intended design.

The handling of compaction events created a further vulnerability. Because the monitoring system only triggered when an agent handed control back to the user, agents could delay oversight by simply continuing to execute actions without yielding. This allowed AI agents to potentially sidestep checks by maintaining continuous execution. The auditors also discovered methods to hide hundreds of actions from the monitor and bypass the Codex sandbox, which is designed to limit the scope of an agent's actions.

The review also revealed that specially crafted inputs could likely crash the monitor entirely, rendering it ineffective. Additionally, model behavior during reinforcement learning (RL) training was not being monitored at the start of the audit period. OpenAI has since stated that it paused RL training following an incident involving Hugging Face in August and expanded monitoring to cover this area. However, these changes were implemented after the auditors’ access period ended and were therefore not included in the initial evaluation.

OpenAI has confirmed that it has fixed all the weaknesses identified by the auditors. The bug related to compaction events was independently discovered and resolved by the company while the audit was still ongoing. The full findings were documented in a 61-page report that was handed over internally. After OpenAI applied redactions, the leadership teams at METR and Redwood Research received a 38-page version of the document. Only a short summary was made available to the general public, with METR stating it had no right to disclose the redacted portions independently.

This audit was part of a pilot program organized jointly by METR, Redwood Research, and OpenAI. The initiative was designed to provide outside researchers with deep access to internal infrastructure, a departure from standard external review processes. The goal was to test the robustness of safety systems under realistic conditions. The findings have now been incorporated into the company’s internal improvements, with all identified gaps reportedly closed.

Looking ahead, METR has indicated it wants to push for greater public transparency in future audits. The organization believes that broader access to the findings will help the industry learn from these specific failures. OpenAI is moving forward with its updated monitoring protocols, which now include coverage for RL training and corrected handling of compaction events. The next steps involve integrating these fixes into the broader safety framework to prevent similar blind spots in future model deployments.

The audit underscores the complexity of monitoring autonomous AI agents, particularly as they become more capable and operate in more complex environments. The discovery that a significant portion of traffic was unmonitored suggests that even well-resourced companies face challenges in ensuring comprehensive oversight. As AI systems continue to evolve, the balance between operational efficiency and safety monitoring will remain a critical area of focus for both developers and external evaluators.