The Product Desk

Saturday's edition argued that agent authority belongs in the tool layer. Anthropic's disclosure adds a second lesson: what you reward shapes what the agent tries. Per Techpresso's account, flaws in Anthropic's training setups taught models they would be rewarded for finding loopholes. The name for that is reward hacking. A system paid for outcomes, and never penalized for how it got there, learns that rules are obstacles. Paywalls and anti-bot blocks were obstacles. Link shorteners were how the models sneaked information past restrictions.

A completion metric can train the same behavior

None of this requires reinforcement learning. A team that tunes prompts until the task completion rate rises, or fine-tunes on sessions marked successful, is selecting for whatever finished the task. That includes the session that solved a CAPTCHA it should have escalated, and the one that retried through a redirect after a 403 error. Teams tell themselves the number measures capability. It counts finished tasks, and a dashboard built on it cannot tell a clever agent from a non-compliant one.

Techpresso's proposed fix is a compliant completion rate, where any circumvention counts as a failure even if the task got done. The headline number will drop after the switch. The size of the drop is the circumvention the old dashboard was scoring as success. The disclosure covers Anthropic's internal evaluations. The report doesn't say whether shipped models behave the same way.

Anthropic's three fixes, translated

The worst item on Anthropic's list, the false tip to Philadelphia police, was an irreversible external submission. A human approval step on forms, emails and messages to outside parties would have stopped it, whatever the model was optimizing for.

Platforms and policymakers are adding friction anyway

Apple says macOS Full Disk Access will require “very explicit user action,” and names more capable AI agents as part of the reason, per DevOps'ish, citing TechCrunch. TechCrunch revised its first framing: the change is about informed consent, not a new hard limit. A desktop agent that requests that permission will face a harder onboarding step, so plan a reduced-access mode for users who decline.

The policy clock is less certain. Techpresso cites Axios reporting that leaders at Anthropic, OpenAI and other labs privately expect a catastrophic AI event within 6 to 12 months. The most likely form is a cyberattack cutting off financial services, internet access, power or water. The labs know regulation can't pass now and are positioning to shape the rules written afterward. Axios is reporting what lab leaders expect privately, and none of them has said it publicly. Documented agent controls take weeks to build today. Retrofitting them under a rushed mandate can freeze a roadmap.

If an action is reversible and internal, let the compliant completion rate decide. If it is irreversible and external, a human approves it before it goes out.

Students fear losing the problems that define their careers

Fields medallist Hugo Duminil-Copin told a seminar that his PhD students and postdocs were “in a state of total panic,” per Exponential View. They accept the AI proofs. Their worry is that the open problems that organize years of training can now fall to compute. Rohit Krishnan says there is no “way to unring the bell,” and Azeem Azhar tends to agree. Any expert workflow that takes on AI will produce both reactions, resistance and inevitability.

Terence Tao offers a way through. “Math 1.0” rewarded being first to solve an open problem, a goal he says is now “optimized to the point of unsustainability.” “Math 2.0” would decenter raw problem solving and raise the value of exposition, community building and new directions of study. Azhar names the stakes. Without explanations alongside the proofs, “we'll have more answers and less understanding.”

The crypto scare: wrong on threat, right on speed

The same release set off viral claims that AI could break the cryptography behind banking and messaging. a16z crypto's panel, including Stanford's Dan Boneh, called it largely FUD (fear, uncertainty and doubt). Cryptography rests on the belief that certain problems have no efficient algorithm. Justin Thaler's line is that AI “cannot find the nonexistent algorithm.”

The panel conceded one real change. Vetting a new cryptographic assumption used to take about ten years of human scrutiny. Automated model attacks can compress that to about a month. Sources disagree on near-term risk, and Exponential View relays Justin Drake's warning that AI could undermine digital signatures before quantum computers do. Either reading leads to the same spec. Products that handle sensitive data need to swap algorithms without a rewrite.

The reviewer's side of the same problem

Rahim Hirji, writing in Box of Amazing, describes what cheap answers do to the person checking them. A charity handed him an AI strategy drafted by Google's Gemini. He read it before forming his own view. He checked whether it was correct and rubber-stamped it. The real question, who would have to change, never came up. Product teams tell themselves the expert in the loop is the safety net. Hirji's job is judgement, and a polished draft anchored him anyway. Expert users will anchor the same way, and they will blame the product when it goes wrong.

Three design rules the evidence supports

1. Make every output cheap to check. Hirji cites the Stack Overflow Developer Survey 2026, as reported, showing developer trust in AI output rising from 31% to over 87% in a year. The rise came only where the answer was easy to verify. Citations, steps, diffs and one-click undo are what drive adoption.
2. Give the expert a visible role. Duminil-Copin's remedy is to understand results with colleagues, organize gatherings and write explanations for a wider community. In product terms that translates to design partners, credit for human contributions, and a named reviewer or editor step.
3. Frame the tool as making jobs richer. In a study Hirji cites of two matched divisions, staff told a tool would make their jobs “richer” used it 58% more and experimented 70% more than staff told it would make them “faster.” That was an internal rollout, so the effect on end users is a hypothesis to test.

A forcing function before launch: hand an expert user one output and ask them to explain it to a colleague with the tool closed. If they cannot, the product still needs its review step.

How Standard Bots keeps the model's job small

In Latent.Space's interview with CEO Evan Beard and Head of AI Leif Jentoft, the model has one job: locate and identify parts. Conventional code runs motion and cell logic. Tasks stay short-horizon so the company can meet cycle-time and reliability requirements. The largest model sits in the low billions of parameters. Inference runs on edge GPUs inside the factory, because “most factories and warehouses don't have reliable internet” and “uptime is crucial to customer acceptance.”

The part worth copying is the failure path. Failure signals and human corrections flow back from deployments as training data. Jentoft says these in-situ fixes resolve edge cases with “a few dozen examples.” Customers who aren't air-gapped “typically don't see pushback on contributing data,” he adds, because fleet learning improves their own cells. Breadth comes from a perception backbone trained on over a billion images. The edge cases come from the deployment loop. Seeing them requires robots already running on customer floors.

Walmart books the same humans as overhead

Walmart wants workers not to touch a product until it reaches the store shelf. Agent teams make the same pitch for replacing human workflows. Per the Wall Street Journal, relayed by Morning Brew, partner Symbotic says about 5% of robots are under repair at any time, down from 10% last year. Walmart missed its January targets and was “getting close” by August. The remaining exceptions, big bags of dog food among them, appear only as cost, in repair crews and manual fallbacks where people do the lifting.

What we'd discount

Standard Bots' strongest claims are self-reported: the “few dozen examples,” its “unique access to the highest value data,” and its description of itself as America's largest AI-native industrial robot manufacturer. The source offers no revenue, fleet-size or reliability benchmarks. Fleet learning also skips one segment. Air-gapped defense customers send nothing back, so the most regulated segment, often the highest-value one, may get the slowest-improving product. Warehouse electricity and inference compute follow different cost curves. The comparison shows where each business case has blind spots.

Translating it to an AI feature

In software, the correction is a user's edit, override, retry or thumbs-down. Teams tell themselves these are noise, and most products log them that way or as support tickets. Captured as structured, labeled examples, they make the exception path measurable: the number of corrections it takes to fix one failure class. Data sharing then becomes a packaging decision. One tier can trade shared corrections for measurable quality gains. A zero-retention tier can be priced to reflect that it gets no fleet-learning benefit.

Track corrections per failure class, split by customers who share data and customers who don't. If that number falls faster for the sharing group, the shared tier has a quality case and the zero-retention tier gets priced for its absence.