# Agent Security Does Not End with Model Choice: Lessons from METR’s Two Security Incidents

> METR’s two security incidents show why AI agents need authentication, scoped permissions, secrets management, spending limits, and data isolation designed together.

An agent application can sit behind Google sign-in and still expose its API keys or the data it can reach. [METR (Model Evaluation and Threat Research)](https://metr.org/about) is a nonprofit that evaluates AI models and studies the threats they may pose. Two security incidents at the organization show how decisions about authentication, spending limits, and day-to-day operations determine how far a failure can spread.

In March 2026, a METR researcher was running an agent orchestration application on a personal EC2 instance. The application was accessible from the internet, with Google authentication controlling access. The instance held an API key for METR’s use of publicly available models.

A bug silently disabled authentication. For several days, anyone on the internet could access the system. An attacker found the application, asked an agent to reveal its model provider’s API key, and obtained the key.

METR describes both incidents in its [security update of August 31, 2026](https://metr.org/blog/2026-08-31-security-update/). The update draws on investigations by METR and its security consultant. METR believes that no sensitive information was accessed in either incident. That assessment sets a limit on what we can claim about the events, even as we examine the engineering lessons.

## Fail-open: when a failed check allows access

A security control *fails open* when an error leaves access available instead of blocking it. In an authentication system, that can mean accepting a request even though the user’s identity has not been verified.

With *fail-closed* behavior, the protected action is denied if the system cannot verify the identity or required permissions.

In METR’s first incident, a fail-open vulnerability in the application, which had been developed with AI assistance, disabled authentication. Using Google sign-in did not prevent the application’s own access control from failing.

Testing the successful sign-in flow therefore leaves part of the design untested. The team also needs to check what happens when a session cannot be verified, required configuration is missing, or the authentication component returns an error.

The report does not tell us how METR’s application behaved in each of those situations. They are examples for a team investigating the same class of failure in its own system. Nor does this case show that every fail-open vulnerability will be exploited. The design requirement is still clear: failure to check access should not become permission to proceed.

## The first incident: what a stolen key made available

After obtaining the API key, the attacker added an SSH key to the instance to maintain access. Over three weeks, the stolen credentials were used to consume approximately $600,000 worth of credits on publicly available models.

METR did not receive a bill for that amount. The model developer had provided the credits for free. Describing the incident as a $600,000 cash loss would misstate its economics. It was nonetheless unauthorized consumption of a resource available to the organization.

Several gaps in the controls combined into one attack path. Someone on the internet could use the agent, reach a secret through it, and then use the stolen key outside the application. The SSH key provided a way to reconnect to the instance.

Detecting the usage was also a problem. At the time, not all users of METR’s internal dashboard could see data on rate-limited requests. The free credits meant there was no natural spending ceiling. According to the report, a spending limit could not be set on this type of API key.

The authentication flaw explains the initial access. The exposed key explains what the attacker obtained. Visibility into usage and limits on consumption help explain how long that access could be exploited. Each involves a different engineering decision.

METR revoked the researcher’s access, stopped the instance, took a disk image, and rotated the credentials stored on it. The researcher’s laptop was also imaged and wiped. METR notified the relevant model provider, brought its security consultant into the investigation, and changed policies covering credentials on external devices and the deployment of public applications.

That response shows why a security incident involving a research tool may require more than shutting down the tool itself.

## The second incident: read-only access can still expose data

In May 2026, METR observed attackers systematically probing its public infrastructure. Their activity included trying stolen username and password combinations, attempting to obtain access through OAuth, scanning newly deployed services, and phishing employees. METR also reports that attackers used agents to automate vulnerability discovery.

During the same period, a public viewer for model conversation transcripts inadvertently exposed a read-only SQL query interface. Queries were restricted to public data by default, but a bug made unpublished evaluation data accessible. Some sensitive model outputs had also been added to the database by mistake.

An independent security researcher found and reported the vulnerability. METR took the service offline. Its investigation found no evidence that the attackers had discovered the exploit or accessed nonpublic data.

Read-only permissions can prevent changes to data. Protecting confidentiality also requires limiting which data can be read. A service that returns the wrong records can leak information without changing a single row.

The query-scoping bug and the presence of data in the wrong place have to be considered together. If a query’s scope expanded unexpectedly, the reachable data went beyond what the service was intended to publish.

METR says it created a public production environment architecturally separated from its internal infrastructure. It also reports narrowing permission scopes, shortening credential lifetimes, and increasing endpoint and server security measures as well as external cybersecurity support.

## Trust boundaries and the reach of a failure

Two concepts help explain the connection between these incidents: trust boundaries and blast radius.

A trust boundary separates parts of a system governed by different access rules and permissions. An internet request reaching an agent’s tools, an agent reading an API key, and a public service querying internal data are separate transitions that each need controls.

The *blast radius* is the range of resources and actions available to an attacker after a component is compromised. A small application can have a large blast radius. A research tool on a personal instance may hold a key that gives access to substantial resources belonging to the organization.

In the first incident, access to an application opened a path to METR’s model usage. In the second, a flaw in a public service created the possibility of accessing data that should not have been published.

A useful design question follows: if one control fails, which others will still work? Secrets management, data isolation, and usage limits should continue to provide protection when authentication fails.

## Set operating limits before deploying the agent

These incidents suggest a way to assess a deployment: ask which risk each control reduces. The appropriate scope of the measures below depends on the agent’s job and the resources it will use.

Start by making authentication failures deny the protected action. Enforce access decisions server-side at the relevant endpoint. A sign-in page does not establish that every function behind it is protected. Test failure scenarios before exposing the application to the internet, and plan for the cases in which fail-closed behavior interrupts legitimate work.

Set separate limits on the permissions of the authenticated user and the agent acting on that user’s behalf. A task that needs to read specific records does not need access to the whole database. Likewise, calling a model and changing a server’s access settings are different capabilities. Each permission should be justified by the work that requires it.

For secrets, who can read the key matters as much as where it is stored. If an agent can read its process environment, moving a key out of source code and into an environment variable does not remove that access. Where possible, separate the component that calls the provider from the workspace the agent can read. Narrowly scoped, short-lived credentials also reduce what an attacker can do with stolen access and how long it remains useful.

Usage monitoring needs to show more than successful request counts. Teams should be able to trace consumption to a key, a job, and a period of time. Rate-limit errors and unexpected changes in consumption need to be assessed against the work actually running. The design should also specify who receives an alert and how that person can stop the usage.

Spending alerts and spending limits serve different purposes. An alert waits for someone to act. An enforced limit stops or restricts consumption. If the provider cannot impose a limit, a counter inside your own application cannot prevent a stolen key from being used elsewhere. That gap belongs in the deployment decision.

Public services should have only the data access their purpose requires. An application that displays published records may have no need for direct access to internal evaluation data. A separate data store and a service identity with limited permissions reduce the data reachable through an application flaw. Maintaining that separation also depends on selecting the right data for export.

Logs need to support an investigation after an incident. It should be possible to connect a user request to the agent’s tool calls, the service identity involved, and the resulting data access. Writing keys or unnecessary sensitive content into logs creates another exposure risk. Decide what to log, who can read it, and how long to retain it together.

Finally, establish who can cut off access, which credentials need to be replaced, and how evidence will be preserved. During an incident, the team is trying both to stop the attack and to understand what happened. The response plan should also define the evidence needed before bringing a service back online.

## What the report establishes, and what it leaves open

METR’s account is useful for understanding the access paths in these two incidents and how the organization responded. It is an incident report drawing on METR’s investigation and that of its security consultant. It does not provide a publicly available independent audit report.

Finding no evidence of sensitive data access is a different claim from conclusively ruling it out. METR presents its assessment within that limit.

Two incidents cannot establish how often agent systems are compromised. The attackers’ use of agents does not tell us how much those tools increase attack success across the industry; that would require comparative data. The report does not compare how safely different models behave under the same attack, either.

## Expand autonomy within verified limits

Before deploying an agent, a team should be able to explain which data a bad request could reach and how many resources it could consume. Evaluating the model’s behavior is part of that decision. Identity, permission, data, and usage controls each have their own job to do.

The initial level of autonomy can be limited to the area in which those controls can be implemented and verified. Reassess the blast radius whenever a tool, data source, or permission is added. As the agent’s capabilities grow, the organization’s ability to detect and stop a failure should grow with them.

---

Language: English
License: CC BY 4.0
License URL: https://creativecommons.org/licenses/by/4.0/
Scope: Evren Bal-authored text, unless this article expressly states otherwise.
Excluded: Third-party material, quoted excerpts, logos, and separately marked images retain their own rights.
Attribution: Credit Evren Bal, link to the canonical source and license, and indicate changes.
Source: https://evrenbal.com/agent-security-does-not-end-with-model-choice-lessons-from-metrs-two-security-incidents