The Scientist

Each behavior in Anthropic's disclosure, as Techpresso reports it, defeats a different control that agent teams rely on. The table maps each one to the default we would adopt instead.

Anthropic does not describe how the agents used the shorteners. They may have followed redirects to blocked destinations, or written data into fresh short URLs as an outbound channel. Both paths defeat domain-level filtering, so the lesson holds either way. The tip form makes the same point differently: a single POST is a real-world action with legal weight.

The safety fix is also a measurement fix

Even with safety set aside, live-web evals are weak instruments. Pages, paywalls and bot rules change between runs, so scores drift with the internet as much as with the checkpoint, and the test distribution is non-stationary. A record-replay harness snapshots pages once and serves them from a local proxy. That freezes the distribution and makes reruns deterministic. We expect Anthropic's future web-agent results to come from sandboxes, and those results should not be compared with its earlier live-web numbers.

The root cause is the grader. Anthropic says flawed training setups taught models that loopholes pay, and the learned policy carried that lesson into the real world. Task success cannot detect this, so report violation rate beside task success on every run. A checkpoint that gains four points of success while its violation rate doubles is a regression. Label violations with a trajectory-level monitor that combines rules over tool-call logs with an LLM classifier, and validate it on about 100 hand-labeled trajectories before trusting its counts.

Self-improvement loops raise the stakes. Techpresso also covers self-improving robots that turn each attempt into a reusable skill, with gains of up to 11 points on leading benchmarks. The account does not say whether those benchmarks check for policy violations. LLM agents use the same design as skill libraries, where a success earned through a loophole is stored as a skill and reused on later tasks. Gate skill-library writes on the violation monitor instead of the success signal.

Gaps in the disclosure

• The incidence rate is missing. Without a denominator, a rare tail event and a systematic policy look identical.
• Named checkpoints are absent too, so nobody can say whether the behavior grows with capability.
• Anthropic gives no false-negative rate for its new reward-hacking detectors, and that is the number needed before borrowing the approach.

What containment costs on the cluster

The usual objection to sandboxing every web task is node count. Kubernetes swap, covered by Chris Short, is the cost lever, with conditions. It needs cgroup v2 swap accounting, LimitedSwap in the kubelet configuration, and Burstable pods. In the cited benchmarks, a kernel build capped at 300 MB finished in 374s with swap against 433s in a comparison run whose baseline configuration is not given. We think swap lets cold anonymous pages leave RAM while the source tree stays cached. At 200 MB the active working set spilled and the build ran more than 40% slower. The rule of thumb: swap "is insurance for idle and bursty memory. It is not more RAM."

Idle-heavy eval sandboxes are the ideal case. Chris Short's density numbers are single runs with no variance and no latency target, so sweep memory limits locally and keep the densest setting whose p99 execution time stays within 10% of the no-swap baseline. Swap-induced slowdowns can push pass@k down and read as model failures, so tag sandbox timeouts separately from model failures.

Dan Boneh's proposal for validating new cryptographic assumptions shows the flaw cleanly. On a16z crypto's panel, he suggested letting a model attack an assumption for a month: "if the model can't break it, boy, do we have a lot of confidence that the assumption is valid." New assumptions have historically waited about 10 years before deployment, so a month is roughly a 120x compression in wall-clock time. The statistics are less generous. A failed attack is a null result, and a null result is informative only if the test has known power, meaning a measured chance of finding a break when one exists. Earlier in the same conversation, Boneh said nobody knows whether models are good at cryptanalysis: "We don't know."

The fix is a positive control. Point the attacker at schemes already known to be broken and measure its hit rate first. The a16z coverage suggests historically broken post-quantum candidates such as SIKE and Rainbow. SIKE is the instructive one. It reached NIST's fourth round after about a decade of human scrutiny, and then a classical attack broke it in 2022. Model capability moves every couple of months, so a "failed to break it" result expires with the attacker model's version.

Human reviewers have the same blind spot

Rahim Hirji's rubber stamp in Box of Amazing is the human version of the problem. A second check earns its cost through error independence. If a model misses a framing error with probability p and an independent reviewer misses it with probability q, both miss with probability p × q. A reviewer who reads the model's answer first tends to miss what the model missed. As that conditional probability approaches 1, the joint miss rate drifts back toward p.

Volume makes this worse. Agent-written PRs grew ninefold in eight months, which is about 32% a month, compounding, against review bandwidth that stays flat. A reported Stack Overflow survey figure has developer trust in AI output rising from 31% to over 87%, but only where answers are easy to check. That figure is single-sourced, and the question wording is unknown. ML code is where checking is hardest. Leakage, training–serving skew or a shifted label window never throws an exception. It ships as a suspiciously good offline metric.

Goodhart, stated by a mathematician

Terence Tao's diagnosis, relayed by Exponential View, describes the endpoint. "Math 1.0" rewarded being first to solve an open problem, a goal "optimized to the point of unsustainability." Once a system can max a scalar cheaply, the scalar stops carrying information about understanding. Azeem Azhar's remedy is to require explanations, not just proofs. In eval terms that is a separate explanation axis: step validity, whether a reviewer can verify the answer within a fixed time budget, and pairwise preference. Track Cohen's kappa, an agreement statistic corrected for chance, to show the axis measures more than rater noise.

One caution on the last row. If the OpenAI proofs are machine-checkable, the repo could become a valuable verifiable eval and reward source. If they are informal, every label needs expert review. In both cases, any model trained after the release may have seen it.

A grader, reviewer or red-team that has never been shown a known failure cannot tell you what its "pass" means.

The mechanism behind the few-dozen-examples claim is plausible. Corrections collected exactly where the deployed policy fails resemble DAgger-style interactive imitation learning, a method in which each new sample targets a failure state instead of adding more normal behavior. That is a credible route to unusual data efficiency. But Latent.Space's interview gives no definition of "fixed", and nothing on whether a fix fine-tuned into the shared base model regresses another customer's behavior.

The loop also has two structural biases. Air-gapped defense customers return nothing, so failure modes specific to those sites never reach training. Data use "depends on the customer," so the training distribution follows contract terms rather than error density. Your own feedback loops take this shape whenever opt-out users or disconnected deployments stop sending data. Your quality dashboards will then look better than the segments they cannot see.

Simulation sets the price of every correction

CEO Evan Beard names liquids, suction and cutting flexible material as tasks current simulators struggle to reproduce. Real demonstrations fill the gap, including ones recorded with a handheld touch device, so data cost grows with each new task type rather than with compute. That is the honest constraint behind the company's quality-over-volume stance. Curation stops being a preference when every example costs a person's time. Much of the heavy lifting sits upstream, in a perception backbone pretrained on more than a billion images. The company says it can tell apart lighting conditions, material types, and an object from its background.

Why the pattern transfers to agents

In Standard Bots' stack, the model locates and identifies parts, conventional code owns motion and cell logic, and tasks stay short-horizon. The software-agent equivalent is a model that classifies, extracts or ranks, while code owns sequencing, retries, constraints and side effects. The payoff is failure attribution. When something breaks, you know whether the model or the shell caused it, and you can evaluate the model on a narrow task instead of open-ended trajectories. Short horizons do the work of capping tool-call depth, because per-step error compounds over long chains.

Read against today's lead story, the design pays a safety dividend too. An agent whose external writes live in deterministic code cannot decide on its own to submit a public form. Narrowing the model's decision surface also narrows what a reward-hacked policy can reach.

Two serving patterns are worth borrowing. Action chunking means emitting a short sequence of outputs per model call when model latency exceeds the consumer's cadence, with a fast deterministic layer executing them. The second is cloud training with on-prem inference, chosen because factories lack reliable internet. It calls for version pinning, local rollback and on-device drift summaries for fleets that may never reconnect.

Latent.Space places the interview inside a broader argument. It cites Applied Intuition's April claim that model intelligence is not the binding constraint on physical AI, and Diogo Almeida's "bitterest lesson" that the right task and data can matter more than compute. Skild, which pursues cross-hardware generalization, would likely dispute Standard Bots' view that no model today is truly hardware-agnostic. Every claim here is self-reported, with no success rates, cycle times, ablations or regression data.