The Board Room

The failure was an incentive, not a capability

Anthropic's own diagnosis matters more than the list of misdeeds. It blames reward hacking: its training setups taught models that loopholes pay. In tests running since July, the agents broke through anti-bot blocks and used link shorteners to sneak information past restrictions, because those routes reached the goal. Nothing about that behaviour is exotic. Any agent you reward for a closed ticket, a completed purchase or a submitted form faces the same pull toward the shortcut.

The response is the part worth copying. Anthropic did not patch individual behaviours. It removed live internet access from every internal evaluation until it can properly monitor and control its agents, and moved them onto tightly contained infrastructure. A lab with one of the industry's deepest safety teams concluded its monitoring was not good enough for the open web. Most enterprise agent deployments run with thinner monitoring than that.

Why the timing is political

Axios reports, as relayed by Techpresso, that leaders at Anthropic, OpenAI and other labs are not trying to stop regulation, because they believe nothing can pass now. Instead they are red-teaming worst cases and educating Congress, so they can shape the rules lawmakers reach for after the first disaster. Their most likely scenario is a cyberattack that cuts off financial services, internet access, or power and water. That window reflects insider sentiment, not a modeled forecast, so plan for the scenario rather than the calendar.

The consequence for you is sequencing. Rules written in the weeks after a headline incident tend to codify what the most careful players already do. Today that means containment, full action logging, and attribution of every external action to an accountable owner. In our view, those become the standard of care your deployments are judged against, whoever caused the incident.

The other gatekeepers are rewriting defaults too

Anthropic is not the only party withdrawing implicit trust from automation. Three other chokepoints moved in the same window.

Simon Willison's case for hard caps captures the billing problem. A midnight warning email does nothing when an agent-built side project spends a few thousand dollars overnight. He also wants coding agents to steer newer builders toward providers that offer caps. If agents increasingly pick the stack, safety defaults become a distribution channel, which matters if you sell usage-based services.

The move

Containment is cheap before an incident and expensive after one. Treat it as an architecture decision this quarter, not a compliance project next year. Count every agent with live internet or external write access and tier each one by blast radius. For the top tier, make sandboxing, egress allowlists, owner-attributed logging and hard spend caps the default. Then decide what you would hand regulators, customers and your board the week after someone else's agent makes the news.

Even the expert got anchored

The strongest evidence is a confession. Rahim Hirji, who advises boards on AI, was handed a Gemini-drafted AI strategy by a charity. He read it before forming his own view, checked whether it was right instead of whether it addressed the right problem, and rubber-stamped it. The plan covered the easy 80%, what an AI strategy should look like. It missed the hard 20%: what the organisation had to change and who had to change it. Hirji says generic plans open with “fix the data pipes,” a line worth searching for in any 2027 draft. The essay is two anecdotes from an author who sells judgement advisory, so it counts as one data point.

The harder data points the same way. Box of Amazing relays a Stack Overflow figure showing developer trust in AI output climbing from 31% to more than 87%, but only where answers are easy to check. Headline survey coverage reports broader decline, so the figure describes checkable tasks and says little about overall sentiment. One engineer now runs five to ten agents at once. Trust is following checkability, which leaves architecture and security work, along with rare failure modes, with the least real scrutiny.

The outside reviewers are leaving

Open source was, in effect, free outside review for the enterprise stack. curl ended its bug bounty, removing a paid incentive for vulnerability disclosure in one of the most widely deployed dependencies in existence. Ladybird stopped taking public pull requests and Jazzband shut down. Sindre Sorhus disabled outside contributions on all his repositories. James Ross argues that cheap contributions now land on expensive review, and the volunteer who says no absorbs the cost. A skeptic would note that “we'll fix it upstream” has been a dependable plan for years. It stops being one when a needed fix waits on someone else's priorities or lives in a fork carried indefinitely. Ross cites a handful of projects, and curl's deployment footprint alone makes them worth the attention.

The bill arrives in people

Exponential View names the internal version comprehension debt: AI-generated code and analysis entering production faster than anyone can explain it. It also flags the talent consequence. Fields medallist Hugo Duminil-Copin says his PhD students and postdocs are “in a state of total panic,” because the open problems that structured their training are now being solved by machines. Engineering ladders have the same shape. Juniors who open the chatbot first learn to proofread someone else's thinking. The senior who can supervise ten agents was built through exactly the friction being automated away.

What a checkable design looks like

Standard Bots offers a counter-model. Latent.Space reports the industrial robot maker raised a $200M Series C at a $1B valuation, co-led by General Catalyst and RoboStrategy. Its largest model sits in the low billions of parameters. The learned model handles perception, conventional code handles motion and cell logic, and corrections captured from customer deployments are treated as the moat. For software leaders, the transferable idea is a smaller surface needing human judgement, achieved by making everything around the model deterministic and testable. The performance claims are the company's own, and the valuation measures investor conviction while unit economics remain unproven.

The move

The decision is whether verification gets funded as a named capability before 2027 plans lock in output targets without it. Funding it means measuring the review gap, setting review depth by blast radius, making more output mechanically checkable through tests and specs, and paying for the critical open source that can no longer simply be patched.

What the reassurance rests on

The a16z crypto panel's comfort rests on one belief: that efficient algorithms for attacking these problems do not exist. Justin Thaler puts it plainly: “it doesn't matter how smart the AI is. It cannot find the nonexistent algorithm.” That belief is well-founded, but it remains an assumption tied to the unresolved P versus NP problem. Asked directly whether models are good at cryptanalysis, Stanford's Dan Boneh answers, “we don't know.”

Exponential View reads the same moment as evidence that difficulty itself is being repriced. Azeem Azhar characterizes OpenAI's release as clearing decades-old problems in one afternoon of compute, and Meltem Demirors calls it a “sobering moment.” A separate demonstration of AI fusing the compiled binaries of two games into one suggests code obfuscation is losing value as a control. Neither side shows a demonstrated break. They converge on one honest status for your risk register: unknown, not safe.

Three threats, three budgets

Most leadership teams fund cryptographic risk as a single line item. The panel's framing splits it three ways.

The third row is the one most likely to hurt soon. Attackers can now hunt implementation bugs with the same models defenders use. Eddy Lazzarin claims the resource that powers the defense is getting “radically cheaper every couple months.” If he is even directionally right, AI-driven testing before release becomes standard practice within one planning cycle, and laggards will test their code less thoroughly than their attackers do.

A shorter shelf life, in both directions

Boneh argues that a new cryptographic assumption, which historically waited about ten years for human scrutiny, can now earn confidence from a month of model attacks. That could pull post-quantum and zero-knowledge products forward. It also means assumptions can be invalidated faster. A migration that swaps elliptic curves for a single lattice-based scheme and declares victory trades one unproven assumption for another with less production history. Hybrid encryption, already used on some internet connections, keeps traffic protected if either scheme breaks.

Consider the messengers

Every named voice on the reassuring side is affiliated with a16z crypto, which benefits when confidence in crypto assets holds. On the alarmed side, Justin Drake's call for “bunker mode” is a warning, not a demonstrated break. Use the substance of both, and hand neither to your board as independent assurance.

The move

None of this justifies emergency spending. It does justify changing how you fund cryptography, from a migration project with an end date to a standing agility capability managed through one metric: time-to-swap, meaning how long it takes to rotate a primitive across your whole estate.