Engineering & Technical

The Engineer

The Signal

PoeLLM finds its command server by reading four words from a poem on GitHub.

A hard-coded dictionary maps those words to the C2 address at runtime. That leaves no domain for you to blocklist and no date-seeded pattern for DGA detection to learn. The only fetch on the wire goes to the site hosting the poem, and the backdoor has held 3,400+ self-hosted open-source AI servers since April.

In Play

  1. FortiBleed and PoeLLM Hit Edge and AI Infrastructure

    Two live campaigns hit infrastructure you run. CyberScoop reports FortiBleed, an FBI/Secret-Service-flagged credential-abuse campaign, is targeting 400,000+ Fortinet firewalls and VPN gateways. PoeLLM has backdoored 3,400+ self-hosted open-source AI servers since April. If you run edge gear or Ollama/vLLM nodes, both are live right now; mechanics and actions are in the security deep dive.

  2. Open-Weight Models Reach Mythos-Class Cyberattacks

    Frontier-class offensive cyber capability is moving into open weights that no vendor can switch off. That puts attacker and defender at the same capability. This is a forward-looking risk, separate from today's live campaigns. The evidence is covered in the security deep dive.

  3. Generation Got Cheap; Verification Is the Bottleneck

    Generation now outruns review in frontier math, in chip design and in your own agent pull requests. Where generation scales, verification coverage becomes the metric that matters. The verification deep dive has the evidence and actions.

  4. Netflix Uses an LLM as a Ranker That Never Generates Text

    Netflix's GenRec uses an LLM as an encoder that scores a closed catalog and never generates text. The pattern, not the lift, is the transferable lesson for any LLM ranking or labeling task. The architecture and numbers are in the GenRec deep dive.

  5. Texas Froze Switch-On for Big Data Centers

    a16z reports ERCOT has paused energizing every data center of 75 MW or more, including 17 that cleared every other step, with the Batch Zero fix stalled until a December audit and air permits frozen too. The large-load queue grew from 63 GW at end-2024 to 474 GW by June 2026. New capacity arrives on curtailable terms: a 30-minute full-load shed, or a 1-minute cutback if your own plant trips. Any 2027–28 GPU capacity sited in ERCOT now carries schedule and curtailment risk.

Deep Dives

FortiBleed, PoeLLM, and the Open-Weight Risk Coming Next

Two live campaigns map onto infrastructure you run. Separately, offensive capability is starting to ship as weights no vendor can switch off. Together they mean detection posture, not patch count, decides your exposure.

The C2 mechanic that breaks blocklists

The best engineering in today's intel is PoeLLM's command-and-control. It uses no hard-coded domain and no date-seeded generation algorithm. The implant reads a poem posted on GitHub, pulls four specific words, maps them through a hard-coded dictionary, and builds its C2 address at runtime. That is a dead-drop resolver. The only lookup goes to GitHub, which nobody can block, so nothing odd resolves. The operator rotates infrastructure by editing the poem, and the deployed binary never changes. It is a clean design. CyberScoop says a SOC whose confidence rests on C2 domain and IP blocklists will not see this threat class.

FortiBleed is the opposite in style and the same in consequence. It is architecturally boring management-plane abuse. Attackers steal credentials and create a new admin account. They disable the legitimate admins or change their passwords, then use the access as a ransomware entry point. There is no memory-corruption exploit and no zero-day, which is why basic hygiene stops it. Orgs that skip hygiene get wrecked.

Separate risk: capability that cannot be recalled

Nothing in today's sources ties either campaign to open-weight models. The longer problem is offensive capability moving into weights nobody controls. The Information reports Anthropic found Zhipu's open-weight GLM-5.3 runs autonomous cyberattacks comparable to its own Mythos model: "a critical threshold in freely accessible capabilities has now been crossed." AI Breakfast says Mistral Large 4, at 93% on Cybench, ships ungated weights at end of October. On CyScenarioBench, Anthropic's Defense tier is blocked on 46 of 50 tasks. The Red Team tier runs with classifiers removed and is blocked on none. Partners found 129,000 verified vulnerabilities from April to July, 33,000+ rated critical or high.

Once a capability ships as weights, the refusal layer is whatever the downloader decides it is. Assume it is already in adversaries' hands and improves with every release.

Bloomberg relays Jamie Dimon's claim that Mythos pushed cyber risk up 10-fold. Dimon's figure has no disclosed metric, so I ignore it. CSO says disclosures and exploits have "doubled" with no baseline, so I read that as direction only. In all five reports the direction is identical. Automated vulnerability discovery at volume, by defenders and attackers, is shrinking the gap between a disclosed flaw and a working exploit.

What changes at 29-minute dwell

CyberScoop surfaces the constraint that forces the design. Former NSA director Nakasone says AI has compressed average adversary dwell time from hours to about 29 minutes. At that number, a containment runbook gated on hours-long human approval misses the window. The workable design is automated first-stage containment. High-confidence detections auto-isolate, auto-revoke, and auto-kill egress, with human review moved to after containment. Nakasone's exact minute figure is directional, and it points down.

What to do

  1. Diff every Fortinet firewall's admin-account list against a known-good baseline this week, enforce MFA on all management interfaces, and rotate admin credentials — rogue admin creation is FortiBleed's tell.

  2. Put every self-hosted AI inference node (Ollama, vLLM, text-generation-webui) behind a default-deny egress allowlist this sprint, with authentication and no public exposure.

  3. Have the security team scan every internet-facing asset with an AI-assisted vulnerability scanner (LLM-driven DAST or external attack-surface management tooling) before end of October, and set a 48-hour patch SLA for critical findings.

  4. Have the SOC lead turn the highest-confidence detections into automated first-stage containment playbooks in your SOAR or EDR this sprint. Each playbook should auto-isolate the host, auto-revoke sessions and credentials, and auto-kill egress, with human review moved to after containment.

Verification Is the Bottleneck, and the Verifier Is the Product

Five independent stories describe one dynamic — generation now outruns review — and the teams that win treat the deterministic checker, not the model, as the thing worth building.

Start with what works in production

The clearest template is GPT-Synopsys, OpenAI's chip-design agent. TLDR Hardware strips the branding. Underneath is a verifier-gated loop (objective, run tool, interpret output, modify, re-run) that stops only when verification passes. It is credible because electronic design automation ships with deterministic oracles. Static timing either closes or it doesn't. Power/performance/area is a number, and a verification suite passes or fails. Synopsys and OpenAI share revenue based on how much the model improves designs. That clause only works because the result is checkable. The model hands over a design that already passed the checks.

The same gap at the frontier and in the repo

OpenAI released 722 math manuscripts (372 families) from an unreleased model. The engineering fact is the trust model. Where proofs are formalized in Lean, a small trusted kernel checks them automatically, so the generator stops mattering for correctness. Azeem Azhar's coverage flags the catch. "Many, not all" verified leaves an unverified residue of unknown size. Lean checks the proof, not the statement. A theorem formalized to mean the wrong thing still passes, the way a test suite can pass while verifying the wrong behavior.

The viral "81% of all major math discoveries" claim is the cautionary half. It came from asking GPT-6 Pro and Fable 5.1 to rank three years of results. That is an OpenAI model grading OpenAI's output, and "major discovery" was never defined. An eval built that way would not survive PR review. It works as a free postmortem on any LLM-as-judge eval.

Why agents stall at 80/20

The Information Briefing covers the human cost. Anthropic's Cat Wu describes work stuck at "Claude does 80% and I do 20%." OpenCode's Dax Raad says engineers have stopped paying attention to AI-written code. Both describe vigilance decrement. A human doing a repetitive check on output that is usually correct suffers it. Bainbridge predicted this for industrial control four decades ago. Motivation does not fix it. Changing the task does: make the repetitive 20% machine-checkable so it leaves the human loop.

When generation scales by orders of magnitude, measure the share of agent-authored changes that pass a mechanical check stronger than "existing unit tests pass." A team that cannot produce that number has an unverified residue and no idea how big it is.

Turing Post extends this to agent instruction files. SKILL.md, CLAUDE.md, and AGENTS.md are untested code. Their promotion path runs from prose to an automated check to trained-in behavior. All five stories converge here. The Bitter Lesson scales generation, but verification only scales where someone built a checker. That checker is now the highest-leverage item on the roadmap.

What to do

  1. Add a 'name the verifier' gate to every agent project this sprint: each agent must loop against a deterministic oracle (tests, type checks, schema validation, benchmarks) and hand off only passing results.

  2. Audit every LLM-as-judge eval by adding a human-labeled calibration set, a judge from a different model family than the generator, and blinding so the judge can't tell which output is yours.

  3. Instrument verification coverage: track the fraction of agent-authored PRs gated by checks stronger than passing unit tests, and route auth, payments, migrations and concurrency paths to mandatory owner review via CODEOWNERS.

Netflix's GenRec: Use the LLM as an Encoder, Not a Generator

The part worth copying isn't the microscopic engagement lift — it's a serving architecture that makes hallucination structurally impossible and ties cost to input tokens you control.

The one design choice that matters

GenRec is a ranker. It reads a plain-text description of a member's behavior, scores the catalog, and stops after prefill. A generative LLM runs prefill and then decodes output tokens one at a time. GenRec takes a pooled hidden state, a compact vector, after the single prefill pass. A scoring head takes its dot product with each catalog item's learned embedding, and a softmax over the scored set returns the top K. Output length is fixed and there is no generated text to parse. The bigger win is that the whole candidate set is scored in one forward pass. Recommending an item outside the catalog is structurally impossible, because the head only scores items that have embeddings.

So the serving profile is an embedding or classification workload. Cost and tail latency track input length. Netflix measured it. Cutting context to about one third cut serving cost by a similar amount, roughly 3x, from reshaping data with no hardware change. The techniques are ordinary feature engineering written in English. Keep strong signals such as long sessions and explicit feedback. Drop weak ones such as hovers and very short views. Compress binges into one session, enrich cold-start metadata, and cap history once returns diminish.

Read the numbers honestly

Online results were +0.115% short-term and +0.006% long-term from a 4-week A/B test on ~10% of traffic. Both were reported as statistically significant. I would not use the long-term figure to justify copying this. The fair reading is that GenRec roughly matches a decade-plus of hand-tuned features and promises cheaper onboarding of new content types and surfaces. The case is architectural. Required sample size scales with 1/δ², so most A/B platforms cannot measure an effect that small, and a replication needs a different definition of success. The source is a secondhand summary with its own accuracy disclaimer. Check the primary Netflix post before quoting any figure in a design doc.

Where it maps in an existing stack

Look for any call that asks an LLM to emit a ranked list, a class label, or a JSON array of IDs. Search re-ranking, intent routing, moderation and ticket triage all qualify. Each pays decode cost and parsing risk for what is a classification problem. Pool the hidden state, add a trained head, and stop after prefill. That bounds the output space and makes latency predictable. Hallucinated IDs stop being possible. Two operational cautions apply. The backbone, head and item embeddings are trained jointly, so ship them as one artifact and never mix versions. The head scores only items with embeddings, so decide how a title released today gets ranked before the next training run.

The part of GenRec worth copying is the LLM used as an encoder, with one prefill pass against a catalog-only scoring head. Copy the architecture. At +0.006%, the lift is too small to replicate as a target.

What to do

  1. Inventory every production path where an LLM generates a ranked list, class label, or JSON array of IDs, and prototype a prefill-plus-scoring-head replacement on one of them this quarter.

  2. Add a closed-catalog question to your ML design-review template this sprint. Any proposed LLM recommender must document how it prevents out-of-catalog items and must evaluate a learned scoring head as one option.

The bottom line

Today's security and AI-engineering stories are one story told twice: machines now produce exploits, pull requests, and proofs far faster than any human can validate or contain them, and the human-in-the-loop step everyone relied on is quietly becoming a rubber stamp. The operating assumption that breaks is that you can review your way to safety — at machine speed a reviewer who repeats a check stops seeing it, and a containment path gated on human approval is already too slow. Build the layer that runs without you in it this week: a deterministic oracle in front of every agent, egress and permission walls your models cannot negotiate past, and first-stage incident containment that fires on high-confidence signals before anyone wakes up.