jacob

Design philosophy for agentic products

The one-line version

A frontier model manages. Humans hold two gates. Everything else is evidence to judge, never a verdict to pass along.

Everything below follows from that sentence.

1. The manager does no work

The most capable model in the system never writes the deliverable. Its job is to talk to the human, keep the queue of questions, hold the gates, and check what the workers claim. Once it starts doing the work itself, it can no longer judge that work, and its context, the scarcest thing in the whole system, fills up with the wrong material.

For a product, that makes the expensive model a fixed cost per run rather than per task, so price and design around it. Each worker gets the model its task deserves: the cheapest for a search, the best for a design. The choice is made per task and written into the brief.

2. Reports are decided, not forwarded

A worker’s report is evidence. The manager checks one claim from each report against the ground truth (the diff, the test output, the row in the database) and says which claim it checked. The least trustworthy thing a worker produces is its diagnosis of why something failed, because a plausible cause is the cheapest thing a model can make. On one run the manager accepted a worker’s theory that a timeout was to blame. One search of the transcript would have shown it was wrong.

So every workflow needs ground truth the manager can check against. In software you get it for free: a diff, a test, a build. In any other domain you have to build it first, whether that’s a citation check, a checklist, a reconciliation or a second source. The verification primitive is the product; the orchestration is just the plumbing around it. Without one, the whole approach falls back to “trust the agent”, which is what everyone else is already selling.

3. Finder, judge and fixer are three contexts

The reviewer that finds a problem doesn’t get to fix it. The manager judges each finding against the ground truth and against what the fix would cost, prefers the fix that removes something, and drops whatever doesn’t hold up, saying why. Then a fresh worker fixes what was kept. An author re-reading their own diff is re-reading their intentions, and a finder allowed to fix will fix what it found rather than what matters.

4. Two human gates, and a queue between them

The human decides exactly twice. First, that the thing about to be built is the right thing: they approve the spec and the design together, never a task list stripped of its reasoning. Second, that the finished thing does the job, judged on a real preview they walked through themselves. Anything else that needs a person (a credential, a fork the model shouldn’t take alone, a request to “teach me”) goes into a queue the human answers when they can. The queue is what lets the run keep going while the human is away. It’s also the part the rest of the field hasn’t picked up yet, because it’s an operating model, not a model capability.

So the human’s clock and the agents’ clock run separately, by design. A product that stops to ask has got this wrong, and so has one that never asks. Keep gate items short: the decision, the default, and what the default gives up. Put the depth behind “teach me”, because people skim a question that explains itself at length.

5. Grill before designing; a wish is not a specification

Left to itself, a model quietly picks a default for every decision the human didn’t state, and you only find out once those defaults are holding weight. So the first thing to build is the spec: numbered behaviours, each with the evidence that would show it works. After that, every open decision gets a recommended default and a named tradeoff before design starts. The human sets how deep that goes, once: every locked decision, or only the real forks. On one run the manager trimmed the list on its own judgement, and twenty-one locked decisions turned into six questions. That depth is the human’s call, not the model’s.

6. Cut vertically, gate every slice, review the whole once, subtraction first

Work is split into slices, each delivering one numbered behaviour end to end, tests first. Every slice has its own gate, and a gate is never batched with the next slice’s. The principles review runs once, on the best model, over the whole finished thing, because some problems only show up in the whole: a helper a later slice made redundant, or a seam two slices each built half of. Its first question is what the thing could do without.

7. Standing rules live in the role; the brief carries facts

A worker’s definition says how it works. Its brief carries only the facts of this task: what to do, where, from which base, which gates, which model and why, and what the report should look like. A brief that repeats the standing rules buries those facts. The strongest models also get scope discipline spelled out: one probe per claim, gates run once on the final state, and a diff size past which the worker stops and asks. Without it, the best models wander off on tangents. With it, a pass that used to take hours takes under one.

8. Context is a budget, and the agent hands itself off

Every model starts sharp and gets worse as its context fills. There’s a zone where it’s smart and a zone past that where it isn’t, and the price per token climbs as the window fills. So context gets budgeted like money. A guard warns an agent once when it crosses a first threshold. At a second threshold it tells the agent to hand off: bring the artefact up to date, write a hand-off file with everything the next context can’t work out for itself, post it where the next session will find it, and stop. The agent does this itself rather than waiting to be cut off. A fresh context reads the hand-off and carries on with nothing lost. The hand-off happens once; a second attempt is silent.

Two things follow. First, work is sized to finish inside the sharp zone. A slice that can’t reach green before two-thirds of a context is gone was cut wrong, and the answer is to re-cut it, not to push on degraded or to compact and continue. Second, large reads are asked for one at a time. A context only exists between turns, so an agent that fires off a dozen big reads in one message can go from a third full to over its limit with nothing visible in between, and get dropped mid-task with nothing saved.

9. The rules carry their own incident history

A cap on live agents, a shape for parallelism (across workstreams, never fanned out over one diff), a limit on how many children each role may spawn: each of these exists because of a dated incident, written next to the rule in the human’s own words. Anyone could copy the pattern in a week. The calibration is the asset. When something goes wrong, the fix is a new rule with its date, not a patch.

10. The canonical state is on disk, not in anyone’s context

The artefact holds the spec, the decisions, the slice table and the closing report. A hand-off file holds whatever the artefact doesn’t, and the queue holds the open threads. A session that fills its context hands off from those and loses nothing (see section 8). A product whose state lives in a conversation has no state.

11. Gate zero, the one that was missing

Gate 1 checks that the design matches the spec. Nothing checked whether the spec should exist at all. One run built every slice, a fix round and a live integration without a hitch, and a research pass afterwards showed the market had already moved on. Execution is now the cheap part, and that run proved it. So before the architect starts, answer three questions: who else does this, what did they ship this quarter, and what’s the wedge, in one sentence a customer would recognise? If you can’t write that sentence, the run doesn’t start.

What this looks like in a domain without a compiler

Take a small law practice. The interview, the two gates, the queue and evidence-not-verdict all carry over unchanged. A partner gets grilled on what “done” means for a document bundle, the slices are matters or documents, and the walkthrough is the lawyer reading the output against the source. What has to be built is the verification primitive from section 2: figures reconciled to the source documents, citations that resolve, the checklist the court’s practice direction imposes. The “never automate” list (a first conversation with someone in the worst year of their life, anything touching a child’s safety) becomes a set of gate items, and the approach already has a place for those. The orchestration is ready on day one. The ground truth is the real work, and whoever builds it owns the result.

Where the field is, as of this writing

Products sell single-task assistants with “agentic” stuck on the front, or workflow engines with an LLM step. The labs are moving towards managed multi-agent systems. An operating model that runs on its own clock, gates on the human, and judges evidence instead of forwarding it lives in blog posts and a few internal builds, not on shelves. The opening is that operating model plus a domain’s verification primitive. Either one alone is easy to copy.

Checklist for the next product