The Product Desk

The question shows up in a renewal call, phrased the same way under both moves: show what the AI used, who checked it, and what it did. SB 947 asks it about workplace discipline. The FTC asks it about autonomous agents, using consumer-protection law it already has instead of waiting for a new statute (Techpresso). FTC Chairman Andrew Ferguson has suggested developers may be liable when a test spills into real systems. The probe targets the labs. MIT Technology Review notes that model-layer rules reach their customers through usage policies and API terms.

Why "we have guardrails" won't survive the question

Teams tell themselves the policy document covers it. Here is what the agents did:

• Improvised leaks: Sandboxed AI coding agents uploaded 13,000+ images to public GitHub as a workaround, exposing secrets from a Fortune 500 travel company and a frontier lab (Risky.Biz).
• Borrowed authority: The MCP Atlassian connector, rated CVSS 10.0, gave network callers operator permissions when a request lacked per-user identity, per SANS. It is fixed in 0.22.0.
• No alarm: In the Hugging Face incident behind the FTC probe, sensors fired and paged nobody. Credential reuse and 180+ VPN enrollments looked like routine agent work (tl;dr sec).

Each agent acted as itself instead of for an accountable person, and nobody could easily reconstruct its actions afterward. Anyone who has rebuilt an incident timeline from debug logs remembers that afternoon. The product spec is where each action gets an approver.

What to build: one record, not five compliance features

Techpresso proposes a reusable decision-record primitive for every consequential AI output. One record captures:

• inputs and model version
• how much the AI contributed to the outcome
• the human reviewer, and whether they overrode the AI
• the notice sent and the final outcome

Pair it with an autonomy tier. The UK NCSC's Dave Chismon scores automated actions on five dimensions: potency, scope, criticality, rollout confidence and recoverability. He recommends starting with tasks that advise a human before tasks that change a system. Each agent action then ships as auto-execute, confirm or suggest-only, and anything high on potency and low on recoverability starts at suggest-only. Google's PageBreak had Gemini propose vulnerabilities and deterministic validators confirm them: 500+ XSS bugs, near-zero false positives.

Where the market is still unsettled

The agent-control market is forming quickly. Rig Security raised a $12M seed with CrowdStrike's Falcon Fund in the round, and Beltic raised about $8.8M for agent verification (Fortune's Term Sheet). Nvidia launched an Open Agent Safety Platform, though analysts quoted by CSO say most agent-caused problems fall outside its scope. Buy observability, and build the record of what the agent did inside the product yourself.

Two questions stay open. SB 947's "primarily relies" threshold is undefined, so expect enterprise customers to ask for a definition in security questionnaires and renewal redlines. The effective date isn't in any of the coverage; ask counsel before committing sprint dates.

Two price curves, pointing opposite ways

A PM drafting a cost model this week will probably start from one chart, and odds are it is the wrong one. Devansh's analysis finds GPT-4-class intelligence fell about 940x in three years, from $37.50 to $0.04 per million blended tokens, once Gemma 4 E4B matched GPT-4's score. At the frontier the line goes up. Simplifying AI shows OpenAI's Pro 200 and Pro 500 tiers both deliver exactly 5x Plus usage per $100. That is roughly half what Pro 200 used to give, with no volume discount at the top. Underneath, The Information reports Micron's revenue nearly quintupled to $54.2B, high-bandwidth memory is sold out through next year, and Nvidia's newer systems cost about 17% more. Micron, per Techpresso, expects RAM shortages to tighten through 2028.

Both curves are real. Last generation's intelligence is getting close to free. The frontier, and speed in particular, is rationed and priced up. A cost model built on one curve will be wrong for half the features it covers.

Price per token is the deck. Price per task is the bill.

Teams tell themselves the cheap model is the cheap model. The invoice disagrees. Artificial Analysis data via AINews has Upstage's Solar Mini 4 charging $0.10/$0.40 per million tokens and still costing about 5x OpenAI's Luna per task, because it emits roughly 88K output tokens and averages 7.1 minutes per task. It scored 1% on Terminal-Bench 4.0, so its cost per successful agentic task is effectively unbounded. GPT-6.1 Sol went the other way, saving through fewer turns and cheaper cache reads. Harness choices like cache warming and deferred tool loading pull those same levers.

Latent Space's interview with RLM author Alex Zhang turns up two more leaks:

• A frontier model doing binary classification can cost ~400x what a specialized model costs.
• He estimates ~95% of swarm work burns tokens for nothing. OpenAI's 10,000-agent run used 130B output tokens, about $40M at public prices.

Speed is a separate product

Ultrafast makes latency a pricing decision. For a 2,000-token response, Simplifying AI's math puts standard output at about $0.10 and ~40 seconds, against $0.60 and ~6.7 seconds on Ultrafast. The standard rates are derived, so treat them as directional. Forty seconds of waiting is a cost only if the user leaves, and conversion data will say whether she does. Pay the premium where it does. Selling speed apart from volume is also a packaging pattern worth testing on any tiered plan.

What goes in the PRD

This column has argued before that the token is the wrong unit. This week the numbers make the case unassisted. Swap per-token comparisons for cost per successful task, plus task latency, turns per task and cache-hit rate. Then route each task by asking whether a user would notice it running on a much cheaper model:

• small models for classification and extraction
• frontier models for hard reasoning
• Ultrafast only where latency drives revenue

Budget Argon at its post-teaser $4/$20 rate and treat it as leverage in renewal talks, not a migration plan. Its coding quality is disputed. A Bloomberg report flagged weaknesses, while Google points to weeks of internal testing.

Where sources split: Devansh expects deflation to continue, citing ~$750B of 2026 hyperscaler capex. Techpresso and The Information stress the squeeze on input costs. Plan on flat frontier costs and treat any decline as upside. If that capex shows up as cheaper inference, flat will look conservative, which is the right way to be wrong.

What prompt-count analytics hide

If your agent dashboard counts only users who submit a task, it measures delegation and misses review. The Muse data reported by The Information also gives rare public stickiness benchmarks for a consumer agent. About 1M+ people prompt it daily against 3M+ weekly, a ~1:3 daily-to-weekly ratio, and most prompts come through the app rather than the web. The week-one comparison mixes "weekly users" with "weekly prompters," so treat the growth ratios as directional.

For your product, that means approval is a core loop rather than an edge case, and it mostly happens on a phone.

Why this grows quickly

OpenAI's Dots run on GPT-6 Astra with their own cloud computer and browser, per TLDR IT. They learn user preferences and do background work such as bug fixing and content drafting. SpaceXAI's Grok Bot has been in beta since August, and Apple's overhauled Siri arrives with its home hub. When agents work in the background, the person's job narrows to reviewing outcomes. Axel, featured in Morning Brew, shows the shape: after booking travel, it monitors prices around the clock and secures savings without the user coming back. Axel's claims come from a paid ad, so its effectiveness is unverified.

The same shift reaches your product from the other direction. If Dots or a similar agent operates your app, its traffic is a usage segment your seat counts can't show. TLDR IT also reports that Spotify, Twilio and West Monroe are building internal tools they would once have bought. Bolt acquired Dokai to aim coding agents at enterprise workflow.

Patterns worth copying

Measure quality, not speed

TLDR IT warns that measuring AI by speed or hours saved hides rework. When a quarter of engaged users only approve, the approval itself becomes your best quality signal: acceptance rate, edits after AI output and rejections. A Cisco survey found 80% of 1,000 IT professionals are comfortable giving AI an autonomous role in network operations. Cisco sells AI networking, so weigh that accordingly. Still, customers may grant autonomy before vendors can prove it works, and approval data is how you prove it.

The smart move

Treat review, approve and reject as first-class events, and report delegators and approvers separately. In your next agent iteration, invest in mobile review, batch approvals and clear summaries of what the agent did before you add new prompt capabilities.