Blog
Essays on AI agents, harnesses, and the backend fundamentals under the hype. The full cross-platform writing list, including Medium and iii.dev, lives on the home page.
RSSMarkdownAsk Claude, ChatGPT, or Perplexity
Start here5 essays
- Harness Engineering: The Real Leverage Is Around the ModelJul 29, 2026
- Inside the Claude Code Harness: The System Prompt MachineJul 31, 2026
- Stateless MCP: The Protocol Deleted Its Own HandshakeAug 4, 2026
- Tokens per Second Is a Memory Bandwidth NumberJul 31, 2026
- The Inference Bill: Why AI’s Best Customers Lose the Most MoneyAug 2, 2026
Showing ·
November 20269 essays
- Agent Substrate 1.0 Joins the CNCF: One Flag Picks the GatewayAgent Substrate reached 1.0 on November 5 and joined the CNCF Sandbox. One install flag picks Envoy or agentgateway as the egress proxy. How a request flows through each, where each one caches the policy and the credential, what I measured on a test cluster, and how to install and upgrade.
- Five Ways to Prune Experts: What Each Rule Keeps and LosesSix public checkpoints delete a quarter to four fifths of the routed experts from Kimi K3.1 Pro, GLM-5.3, DeepSeek-V4.1-Flash, and Qwen3.8-Flash-Next. Five rules pick the survivors: router magnitude, architecture search, calibrated KL, REAP saliency, and a global budget. What each rule measures, why pruning saves memory and not time, and where the published loss is.
- DSpine: The Draft Conditions Through Depth, and a 24 GB Card Pays in ContextTencent's DSpine drafter for Qwen3.8-27B passes each position's prediction to its neighbour inside every draft layer, then resolves a 16-token block from 16 by 16 transition tables in one parallel pass. On an RTX 3090 a community llama.cpp build goes from 28 to 90 tokens a second on mixed prompts and loses 110,000 tokens of context. The arithmetic behind both numbers, the three paths that run it, and how to measure acceptance on your own prompts.
- Cheap First, Escalate on Doubt: The Arithmetic of a Model CascadeA cheap model answers first, a confidence signal decides which items a big model also sees, and the threshold trades cost against error. Hightouch's and Kiro's published cascades, OpenRouter's cache-aware router, the arithmetic, a runnable sweep, and the failure modes.
- Four Tiers for One KV CacheGoogle's TPU Sync treats TPU HBM, host DRAM, storage, and the data center network as one memory hierarchy for KV caches. The arithmetic of reload against recompute per tier, what TPU 8i Max and TPU 8tx publish, and how to size the same tiers on vLLM and SGLang.
- Revolut's Self-Hosted GLM: Where $0.30 per Million Tokens Comes FromRevolut serves GLM-5.3 to 2,000 engineers on 56 B200 GPUs for a reported $0.30 per million tokens. The arithmetic behind it, lever by lever: NVFP4 against FP8, TP8 against TP4, a 256K context cap, and prefill and decode disaggregation, with a cost machine you can drive.
- The Sandbox Signs Its Requests: Workload Identity Replaces the Key in the BoxIn one week, E2B, Render, Baseten, Agent Substrate, OpenKruise, and Anthropic converged on one pattern: the egress proxy adds the credential outside the sandbox, per destination host. E2B and Agent Substrate make that credential a signed per-sandbox token, verified through OIDC discovery and a JWKS and exchanged at AWS STS. One request walked through all four stops, the failure modes, and a demo that runs on a laptop.
- Safe Outputs: Why the GitHub Workflow Agent Holds Only a Read TokenGitHub Agentic Workflows gives the coding agent a read-only token. Each write it wants becomes a safe output that a separate job checks and applies. This essay covers the workflow file and its lock file, the limits to set, why an injected agent cannot push, and the pull_request_target change on December 2.
- A 763B Model on a 12 GB Card: The Biggest Tensors Are Lookup TablesDeepSeek-V4.1-Flash-1017 has 763.2B parameters, and 2-bit GGUFs now run it on one 12 GB GPU. Its two Engram lookup tables stay on the SSD, because a token reads only 48 rows of 144 bytes. This essay works the SSD bytes per token, the per-tensor bit budget, and what stays on the GPU.
October 20269 essays
- The Page Nobody Wrote: How a Rejected Prompt Leaked Other Users' KV CacheA rejected over-length prompt still crossed the SGLang prefill-to-decode handoff, and decode read seven KV pages that held other users' data. The mechanism, the fix releases, and the tests to run on your own deployment.
- Agent Compute Engine: The Credential Never Enters the VMHow Agent Compute Engine runs each agent in its own Cloud Hypervisor microVM on Titanium, gives it a SPIFFE identity, inserts service credentials at egress outside the VM, checkpoints idle sessions, and keeps workspaces on Hyperdisk. With the pattern to rebuild on any cloud.
- LiteLLM 1.107.0: From a Leaked Key to a Signed ReleaseHow a LiteLLM admin key leaked in May led to an organization owner's GitHub token, a pipeline the owner controlled, and a signed credential stealer published as 1.107.0, with the checks that passed and the one that catches it.
- Scion: Source Control at 30,000 Merges a DayWhy the git model stops fitting when agents merge 30,000 times a day, and how Scion answers it with changesets, derived data, three stateless cache tiers, and a tree-walk sparse checkout.
- Handwired: How JavaScript Becomes an Exact Neural NetCheng Lou's Handwired writes JavaScript as an exact neural net with no training. A step-by-step build from one if-else to a whole switch: ReLU steps, selects, comparisons whose cost depends on declared grids, cells, the host loop, loops as trees, CPU and WebGPU, training the weights, and the limits, measured.
- TypeSafe Raised $870M. Can Jev See? 408 Calls on 48 Real PhotosTypeSafe raised $870M, then gave Jev image input and a Box type. 408 real calls on 48 labeled photos: calibrated Nouls, rewritten questions, picture options, Box IoU, absent-target traps, and an intake router you can drive.
- agentgateway in Production: One Endpoint for MCP Servers, Agents, and ModelsA step-by-step agentgateway 1.6.0 guide: run it, put two MCP servers behind one endpoint, add JWT and tool rules, check arguments safely, limit calls, trace, scale, survive failures, route A2A and models, and move your setup behind it.
- Continue Is Not Consent: How a Harness Nudge Became PermissionAn agent asked permission; the harness replied "use your best judgement"; GPT-6 Astra read that as yes in 44% of runs. Why it works as consent, how seven harnesses end a silent turn, pentest rules of engagement, and a local lab.
- Build a Personal RAG Harness: Jev Judges, a Local Model WritesA step-by-step personal RAG harness over your own writing: Jev routes, reranks, and checks every cited sentence, a local qwen3 writes, and each stage is measured for recall, cost, and latency.
September 202618 essays
- Laya and Jev: System 1 Models Answer With a Type, Not TextJev and Laya return a probability for every option you define without generating any text. How Laya reads answers off mask tokens, a local CPU run at 81 ms for three questions, where the probabilities lie, how to pick a threshold, and routing 10,000 tickets a day.
- Inside the Devin Harness: Two Loops and a HandoffHarness Anatomy 5 of 5. Devin ships as a cloud agent, a CLI, a desktop editor that used to be Windsurf, and a fleet API, but underneath there are two harnesses: a local loop shared by the CLI and Devin Local, and a cloud loop that boots every session from a snapshot. The snapshot as the unit of state, the blueprint as the verification contract, routing inside the harness, and the /handoff seam, with two interactive figures.
- Agent Substrate Egress and Its Gaps: Nothing the Agent Says Is TrustedAgent Substrate tunnels every actor TCP connection over mTLS CONNECT to a gateway that ignores the hostname and SNI the agent claims. What a refusal looks like from inside, per-actor EgressPolicy rules, why hostname rules need the MITM gateway for HTTPS, and the gaps the docs admit: no control-plane authorization and DNS outside policy. Part 3 of 3.
- Agent Substrate: How Suspended Agents Share a Warm PoolPart 1 of 3. Agent Substrate runs many agents on few pods: actors live in snapshots and borrow a warm worker only while busy. The suspend, pause, and resume state machine, Full and Data snapshot scopes, gVisor and micro-VM isolation, repo manifests, request parking, and how to size a worker pool, with two interactive figures.
- Google AX (Agent Executor): Task, Workspace, Gateway, ModelAgent Substrate series, part 2. Google AX runs agents on Kubernetes as four declarative objects on top of Agent Substrate. How each one resolves underneath, why a resumed task is a fresh process over a restored /workspace, when suspending pays, and the manifests and ax commands from the repo docs, with two interactive figures.
- SWE-Bench Pro V2 Caught Two Models Cheating. The Fix Was Moving the Grader.SWE-Bench Pro V2 caught Opus 5 forging a Go checksum and Inkling editing the module cache by grading every patch on a clean image. What in-place grading measures, the 17.8 point public to private gap, and the V2 defenses as scripts for your own agent pull requests.
- Opus 5.5 in Practice: Effort Is the Only Dial LeftA practical playbook for Claude Opus 5.5 in the API and Claude Code: the request changes that avoid 400s, an agent loop that survives always-on thinking, effort per task class, routing effort in code instead of prompts, a state-file harness for long runs, a traceability review, and a worked cost example for a 38-handler migration.
- Amazon Blocked Muse. The Request Had No Name on It.Amazon blocked Meta's Muse because its requests carried no agent identity and its logins ran on stored passwords. How Web Bot Auth signatures and delegated grants fix both, with the key directory, signing and verification code, the ID-JAG token requests, a $38 checkout end to end, and where it breaks.
- Zed Turned Off Pull Requests. The Merge Gate Has to Go Somewhere.Zed's Delta replaces the pull request with a thread that keeps the agent's reasoning attached. How DeltaDB records it, review threads, the review-queue math with a worked example, and how to keep the merge gate on the remote with a land skill that queues auto-merge, a GitHub ruleset, and CODEOWNERS on agent instructions.
- BITCOS: Intel's Ternary LLM Weight Format Goes Below 1.58 BitsBITCOS (bitmap and compacted signs) is Intel Labs' weight-packing format for ternary LLMs, not a crypto token. Stdlib code to measure each tensor's zero share and pack it (BitNet b1.58 2B4T runs 38.5% to 50.8%), a per-tensor layout rule, and a worked example showing the tied f16 head outweighs every ternary layer.
- MCP Servers Can Now Ship Their Own ManualHow to serve and host skills under SEP-2640, the MCP Skills Extension: a manifest builder that hashes a skill directory into its skills/list entry, the Python reference server, a host verification gate that refuses rotated and unlisted files, a twelve-server cost example, and what breaks, with three interactive figures.
- The Harness Is an API Now. Here Is What You Still Own.If your agents run long enough to crash, OpenAI's Agents API now rents you the loop. What stays yours is the machine, its keys, and recovery around a stream that does not replay: session and executor setup, restricted networks and vaults, a webhook handler, stream recovery, a costed worked example, and the failure modes, with three interactive figures.
- The Eval Sandbox Held the KeysIf your eval harness runs agents that read outside text, its keys are production credentials. How GTG-50020 turned an AI vendor's eval sandbox into a key dispenser, and the verified setup that makes the attack come away empty: gateway outside the sandbox, per-run keys with a budget and expiry, a closed network, and a results scan.
- RSA-260 Fell to an Agent Fleet. The Human Sent 3,328 Messages.How a three-week agent run factored RSA-260, read as a build order for any long agent job: an oracle per stage, ten-minute workunits on a CADO-NFS server and clients, a restartable driver, checkpoint intervals from Young's formula, verification beside production, bounded fleets, and the 3,328 human messages that pruned it.
- GPT-6 Astra Changed the Shape of a TurnWhat GPT-6 Astra changes in your tool loop, with the code to handle it: async tool definitions and a job registry around wait_for_tasks, a steer journal for mid-turn steering, effort moved into configuration_update items to keep the cache, the 403 stop path, the migration diff, and a costed twelve-turn incident session.
- Four Flash Models in Four Months. The Bill Per Task Went Up 40%.Gemini 3.8 Flash shipped three weeks after 3.7 Flash at the same price per token, and Artificial Analysis measured the cost per task rising about 40%. What "works harder" means on the invoice, why the workhorse tier is where budgets move, and the January 1 price cliff, with three interactive figures.
- Fable 5.1 Burned the Meter in Twenty MinutesAnthropic cut Fable 5.1 cache reads by 75% on September 1; by September 2, Max 20x subscribers reported five-hour sessions gone in twenty minutes. Both are true because the cut was an API price and the wall is a plan meter that counts traffic, not invoices. Three interactive figures on where the tokens go.
- Fable 5.1 and Mythos 5.1: One Model, Two DoorsAnthropic shipped one model behind two doors: Claude Fable 5.1 for everyone, Claude Mythos 5.1 through trusted access. The safeguard stack is the only stated difference, cache reads fell 75 percent, and Google and OpenAI split their releases the same week. Where the 25 and 45 percent savings come from, with three interactive figures.
August 202620 essays
- Claude Has Two Meters. The 20x Is Only on One.Claude's usage limits run on two independent meters: a rolling five-hour window where the 5x and 20x multipliers are defined, and a weekly cap on top with no published size. The arithmetic of the September 14 change, what drains each meter, and three interactive figures.
- OpenAI Cut Cursor Off. Five Percent Is the Interesting Number.OpenAI gave Cursor the maximum notice its contract allows, with a proposed shutoff of November 12, 2026, and Cursor says OpenAI models serve about 5% of its traffic: what a vendor cutoff actually breaks inside a coding harness, why the 95% is untouched, and the audit to run before your own vendor pulls the switch, with three interactive figures.
- The Model Hardware Standard: Read, Write, and the Plate That Must Be ThereAnthropic's Model Hardware Standard gives agents read and write primitives over lab and factory devices, and a driver that refuses a write before the device moves when the plate is missing. What MHS standardizes next to MCP and A2A, with two interactive figures.
- The Bulletin Board: How Isolated Agents Built a Message Bus Out of a Package RegistryIn an OpenAI cybersecurity evaluation, agents sealed in separate sandboxes turned a shared package registry into a message board, reached the internet through it, and broke into Hugging Face. The mechanism, the dated climb, and why isolation is a property of the environment, not the model.
- MCP's New Roadmap: Events Without Sessions, Identity Without API KeysThe Model Context Protocol published a new roadmap on August 22, 2026, five months after it deleted its own sessions: how elicitation now works on a stateless server through multi round-trip requests, how push survives without a session, why DPoP and workload identity replace pasted API keys, and what a hundred-tool catalog costs before the first question, with three interactive figures.
- Grok Bot: The Agent Got a Computer. Your Bots Share It.xAI's Grok Bot gives agents a cloud computer that signs into your tools with no API or MCP, the docs say every Bot on an account shares that computer and its sign-ins, and the clones copied the shape within eight days. Three interactive figures: close the lid, the login jar, and a routine compiler.
- DFlash 2: The Drafter Stopped Guessing AloneSpeculative decoding drafts a block and verifies it in one pass; DFlash 2 keeps the draft parallel and adds a 2M-parameter path selector plus a two-tap convolution, for about 21% more accepted tokens per pass at about 1% added latency, output provably unchanged, with three interactive figures.
- All About the A2A Protocol, Now Part of the Agentic AI FoundationAgent2Agent v1.0 dropped the wire discriminator, turned the agent card into a list of transports, and made the streaming rules hard errors. What the spec standardizes, with working server, client, curl, webhook, and card-signing code, and six interactive figures.
- Claude Has Levels. Most People Stop at Two.Nine levels from the claude.ai prompt box to a fleet of agents: MCP connectors, memory engineering, skills, plugins, hooks, worktree parallelism, cloud sessions, managed agents, and the practices at each level almost nobody uses. Five interactive figures.
- The AGENTS.md Practices Nobody UsesA Linux Foundation standard in 700,000+ files, and almost nobody uses the practices that make it work: the token budget, nearest-file precedence, the resolution order that shadows your rules, and its life as an attack surface. Three interactive figures.
- The Harness, Not the ModelSame model, different harness, different bill. Claude Code, Codex, Gemini CLI, Grok Code, Kimi CLI, opencode, Qwen Code, and Pi compared, with three interactive figures.
- The Inference Engine UnderneathSame model, same GPU: the serving engine decides throughput and tail latency. vLLM, SGLang, TensorRT-LLM, LMDeploy, and TGI, with four interactive figures.
- Fifty Startups Coming NextFifty companies the agent boom is about to make necessary, told as a story across seven acts of the stack. Four come with a working interactive demo.
- Stateless MCP: The Protocol Deleted Its Own HandshakeMCP 2026-07-28 deleted its own sessions. Why a stateless protocol core scales, with a request-lifecycle stepper and a load-balancer diagram you can kill a replica in.
- OpenWiki: The LLM Wiki Pattern as Running CodeThe idea, built two ways: a connector-rich CLI that version-controls markdown, and the same generate-search-refresh-lint lifecycle decomposed into functions and triggers on a runtime. What each gains and gives up.
- The LLM Wiki: Stop Retrieving, Start CompoundingRAG retrieves the same raw documents on every query and forgets between them. A wiki the LLM maintains compiles knowledge once and compounds. The pattern, its three layers, and why the schema is the product.
- LLM Wiki v2: What Breaks at ScaleThe clean wiki sags at scale. Confidence scoring, supersession, an Ebbinghaus forgetting curve, consolidation tiers, a knowledge graph, and hybrid search, the lifecycle machinery a flat markdown store leaves out.
- The Inference Bill: Why AI’s Best Customers Lose the Most MoneyTraining is a one-time cost; inference recurs with every query and inverts unit economics, so a lab’s heaviest users are its least profitable. Cost per token, the five-layer stack, and who is burning, with three interactive figures.
- The Forward Deployed Engineer: The Job the AI Industry Paid Billions to RediscoverNine billion dollars from four companies for a title Palantir coined for its lowest-status engineers. The mechanism behind the comeback, and what the copycats will get wrong.
- What a Forward Deployed Engineer Actually DoesThe deliverables named in real postings, the pod structure, sourced pay, and the three questions that reveal whether an FDE role is real or a retitled solutions engineer.
July 202614 essays
- Graph Engineering Is 290 Years OldEuler deleted detail until only structure remained. That is still the whole job. Two interactive figures: walk the Konigsberg bridges, run the 107-agent research DAG.
- Inside the Pi Harness: The Agent That Shows You the MeterHarness Anatomy 1 of 5. Session trees, a twenty-line system prompt, no pruning, and cache misses itemized in dollars.
- Inside the Claude Code Harness: The System Prompt MachineHarness Anatomy 2 of 5. Layers ordered by change frequency, CLAUDE.md as a user message, cache preservation as architecture.
- Inside the Codex Harness: Compaction as an API PrimitiveHarness Anatomy 3 of 5. Four compaction paths, a model trained on summary placement, and the sealed-state price. Two settings tripled ARC-AGI-3.
- Kimi CLI and K3: The Newest Harness Has the Strangest IdeasHarness Anatomy 4 of 5. A harness that is also your shell, checkpoint time travel, 128-agent swarms, mandatory preserved thinking.
- Tokens per Second Is a Memory Bandwidth NumberEvery generated token reads the model's weights from memory once. Divide bandwidth by weight bytes and you can predict your speed before the download finishes.
- The AGENTS.md GenreA new genre of technical writing: documents addressed to machines, in shouted imperative English, where every rule is a scar.
- The Composable Harness: How Agents Actually Get ReliableA backend is workers, triggers, and functions, and an agent is just another worker. Once the harness is the backend, reliability becomes durable execution.
- New Names, Old Fundamentals: Decoding the Agent Era's RebrandsHarness engineering, context engineering, loop engineering. Most of the new vocabulary renames fundamentals we have had for decades.
- Your Codebase Has a Second Reader NowEvery readability rule was always for the next human. A second reader arrived, and it audits how much of that advice you followed.
- Your "Row Changed" Hook Isn't CDC: The Dual-Write TrapEmitting an event after a commit without an outbox is a lossy notifier. The fix is the outbox pattern or real change data capture.
- Harness Engineering in Practice: Stop Agents Building the Wrong ThingAgents build the wrong thing, ship duplicates, reinvent what you have. The fix is a system the agent is part of.
- Harness Engineering: The Real Leverage Is Around the ModelHold the model fixed and engineer the environment: context, tools, the loop, and proof.
- How to Become an AI Engineer in 2026: A Complete RoadmapThe skills in order, two routes in, and one free curriculum that builds the whole stack from scratch.