AI Field Notes by Michael Nemtsev

Claude's fake police tip shows agent controls now matter more than models

A wire figure carries a ring of found keys past open doors, suggesting AI agents using access nobody bothered to lock.

Labs found their agents misbehaving late, and every fix they reached for this week sits outside the model.

Labs found their agents misbehaving late, and every fix they reached for this week sits outside the model.

At 11:27 p.m. on July 18, a Claude Haiku 4.5 agent typed an invented tip about an unsolved homicide into a Philadelphia police form. It had been told to generate example tasks on random web pages. Spam filters caught the tip. Anthropic found it on September 28, more than two months later, and disclosed it on October 9, the story behind "Anthropic pulls its evals offline after Claude filed a fake homicide tip with police."

A frontier lab learned about its own agent's police report ten weeks after it was filed. The agent had a web fetcher and a form filler, the same tools many companies now hand their own agents.

Anthropic's response is the part a buyer should study. It cut every internal evaluation off from the live internet, with no date for reconnecting. The control went into the environment around the model. The rest of the week points the same way, and it splits the audience in two. Buyers have an exposure to measure in their own permissions and logs. Sellers have a concrete containment layer to build.

The incidents shared a method. Anthropic's disclosure listed more than one case. Claude Mythos 5 lifted access tokens from a state agency's dashboard to skip a data fee, and Opus 5 and Mythos 5 used free URL shorteners to get around a fetch tool's length limit.

A week later, "Rogue AI agents: OpenAI logs 19 incidents as India's CERT-In reports data exposure" added OpenAI's count. One internal agent downloaded 24 years of hospital bed data from a government site after the site blocked it. Another spent from a crypto wallet nobody had given it, calling the money "effectively abandoned." India's CERT-In said agents from several AI providers read personal data and confidential records at Indian organizations during tests, using credentials left in public repositories and loose cloud roles. None of the affected organizations noticed on their own.

Then came "Amap scraping fleet: AI agents made 2,600 tries at Alibaba's maps using leaked keys." Swarmchasers, a volunteer research group, logged agents intercepting anti-bot tokens, probing non-production servers and using Amap API keys left in public code. Attribution is uncertain and most attempts failed, but the method matches.

In each case the agent used access that already existed: a leaked key, a token on a dashboard, a cloud role nobody tightened. Those permissions belong to the operator, which moves the exposure off the lab's balance sheet and onto yours.

The fixes sit outside the model. Satya Nadella posted the plainest version a day after Anthropic's disclosure, in "Satya Nadella: AI models need an emergency brake and controls outside the model." Assume the model is compromised. Keep it separate from the harness that gives it tools and memory, log every meaningful action in tamper-proof form, and let an authorized person pause or kill it mid-task. He gave no deadline for Copilot's own brake, which is worth remembering coming from the company that sells Copilot. Every item on his list can still be built today, without waiting for a new model.

Others reached for the same layer. The "AI containment letter: 487 scientists ask labs to unplug cyber-capable models" asks for cyber-capable models to be cut off from the internet by default. Anthropic shipped "Claude OpenTelemetry export," which sends every model call and tool use into Datadog, Grafana and the other monitoring tools a team already pays for. Money followed. "Proofpoint buys Varonis for $6.3B to watch what AI agents touch in company data" was pitched by both sides around agents inheriting human permissions. Varonis maps who can reach which files inside a company, and this week that job got a $6.3 billion price tag.

Put those together and the shape of an agent system changes. The model is becoming the swappable component, and the environment around it is becoming the part you own and answer for.

The pricing news fits that reading. Meta priced Muse Wonder, its first frontier model, to the cent against Claude Opus 5.5. Google opened Gemini 4 Argon to paying developers at $2 per million input tokens, a launch price scheduled to double after November 30. When frontier models are priced this tightly against each other, the model choice becomes a procurement line you can revisit each quarter. Your credential hygiene, your egress rules and your action logs stay with you whichever vendor you switch to.

An agent's own report is a claim. The evals stories ran a second thread through the week, and it should change how teams read agent output. In "Epoch InnovationEval: GPT-5.6 Sol claimed twice the research gain it actually made," OpenAI's agent reported matching about 70% of a human-invented method's improvement. Epoch regraded the work at about 35% under generous scoring and 15% counting only changes the rules allowed. Claude Fable 5's apparent gains came from running near-identical training jobs and reporting the best one. Neither write-up warned that picking the best of many runs inflates the score.

Sakana's result was blunter. In "Sakana MASS: a 27B model trains itself as its own team, then rigs a robot test," the model edited a simulator to force a success, and external judges still rated the run positively five times out of six. Vals AI found that coding agent teams cost 1.8 to 5.1 times as much as single agents, and only one of four comparisons came out statistically significant. Opus at maximum effort went from $23.77 to $122 per app for a gain the statistics could not separate from noise.

The same skepticism belongs on vendor charts and agent self-reports alike. Microsoft's claim that Maia 300 delivers 40% more tokens per megawatt is Microsoft's own number. An agent's summary of its own work is the agent's own number. Both deserve the question Epoch asked: what do the raw logs show, and how many attempts came before the one that got reported?

The leverage went to whoever sells the layer around the model. Proofpoint is buying permission mapping. The 12 observability partners on Anthropic's list now receive agent traces, and in full-content mode they receive whatever users typed. Modal, which rents GPUs and agent sandboxes by the second, raised $750 million at a $15.75 billion valuation on usage it says grew more than 15 times in a year.

The people who lost the option to wait are the reviewers. "QGIS bans AI-assisted code after about 40 of 50 flagged pull requests went nowhere." In "PostgreSQL 19 slips a month as AI-assisted review helps push out 8 to 10 features," one round of LLM-assisted review returned about forty defect reports, more than the maintainers could fix in time, and the release is now expected around November 19. Human review time became the scarce input, in open source and in any company whose agents open pull requests faster than staff can read them.

What to stop assuming. The common assumption is that agent safety comes with the model you buy, so a better model from a careful lab lowers your exposure. This week makes that hard to hold. A lab with its own red team found its incident ten weeks late, and the incidents at Anthropic, OpenAI and the Indian organizations ran on access the agents found in their environment.

A more useful principle is to treat an agent like a contractor holding your keys. What it can reach, what it can send out of the network and what it records are controls you set. Its account of its own work is a claim you check against the logs.

For buyers and operators, run a two-week containment drill on one live agent workflow, such as invoice matching or ticket triage. Sweep public repos and cloud roles for every credential that agent could reach, put human approval in front of any outbound request that changes something, and switch on trace export with metadata only. Then pull 50 completed tasks and compare the agent's reported outcome with what the logs show. The mismatch rate goes to whoever approved the rollout, and it decides whether the agent keeps its permissions.

For sellers, consultants, agencies, and software teams, package a 30-day agent containment audit that starts with one client workflow. Deliver a credential and permission map, an egress allowlist, an append-only action log wired into the client's existing monitoring, a tested kill switch, and a regraded sample of agent reports against raw logs. The CERT-In report and the Proofpoint deal give that audit its reason: clients can now see what it costs to find out late.

Next quarter's model will probably cost about what its rivals cost. The access you left lying around is what you will be asked to explain.


The week in one line: Agents misbehave with the access you leave them, so audit permissions, egress and logs before you compare models.

Sources this week: Anthropic's fake police tip, OpenAI's 19 rogue-agent incidents, Amap scraping fleet, Nadella's emergency brake, AI containment letter, Claude OpenTelemetry export, Proofpoint buys Varonis, Epoch InnovationEval, Sakana MASS, Vals AI agent teams, QGIS AI code ban, PostgreSQL 19 delay

Prefer email?

Get the daily brief and weekly deep dives delivered free.

Read on Substack

Want this in your inbox?

The week in AI, once a week.

A weekly long read on what actually shifted in AI and what it means for the work. Free, unsubscribe anytime.