Agents · context · tooling · MCP

The Agent Harness

A model call is four lines of HTTP. Everything that makes it into a product — the loop, the tools, the context budget, the retries, the traces, the evals — is the harness, and the harness is where all the engineering lives. Built up from first principles, then spent on one concrete thing: a design tool that makes posters using models from OpenRouter.

violet = the model decides neutral = your code decides ochre = untrusted Python & React MCP 2026-07-28
00

Five words people use wrong

Most confusion about agents comes from six or seven words that get used for four or five different things. Pin them down and the architecture becomes almost boring — which is the goal.

Model
A function from a list of messages to a next message. Stateless, pure-ish, non-deterministic. It cannot do anything. It cannot remember anything. It emits text and, sometimes, a structured request that it would like something done.
Tool
A JSON Schema you show the model, plus a function you run when the model asks for it. The model never executes code. It writes down a name and some arguments; your code decides whether to honour that.
Agent
A loop: call the model, run any tools it asked for, append the results, call the model again, until it stops asking. That is the entire idea. An agent is not a model — it is a control-flow pattern wrapped around one.
Harness
Everything around the loop that makes it survivable in production: budgets, retries, timeouts, cancellation, permissioning, persistence, streaming, context assembly, tracing, and the stop conditions. This is your actual software.
Context
The exact bytes you send to the model on this call. Not “what the agent knows” — there is no such thing. Every turn you rebuild the whole input from scratch, and choosing what goes in it is the highest-leverage engineering in the system.
MCP
A wire protocol for discovering and calling tools that live outside your application. It is not how agents work; it is one way of plugging third-party capability into one. You can build a complete agent without ever touching it.
Context window
The hard token limit on that input. A budget you allocate, not a container you fill.
The one sentence to hold on to

An agent is a while-loop over a stateless function, and the hard part is deciding what to put in the argument. Everything in this document is either a way of running that loop safely or a way of deciding what goes in the argument.

MODEL stateless · no memory cannot execute anything 1 · messages[] + tool schemas rebuilt from scratch every turn 2 · text, or tool_calls[] a request, not an action THE HARNESS message list · tool dispatch budgets · retries · approvals tracing · persistence your deterministic code user / app “an A2 gig poster” plus assets, brand, budget goal artifact your tool functions validated · timed · logged 3 · dispatch 4 · result the outside world APIs · files · image models the loop ends when: the model returns no tool calls · or max turns · token budget · cost cap · wall clock · no measurable progress
Steps 1–4 repeat. Nothing else is happening. Note what the model is not connected to: it has no line to the tools or to the world — every arrow into those goes through your code, which is the only reason any of this can be made safe.
01

The atom: one model call

Before agents, tools or MCP, get completely clear on the primitive, because every later problem is a consequence of its two properties: it is stateless, and you pay for the whole input every single time.

The shape

A chat model call is a list of messages in, one message out. Four roles do all the work:

RoleWritten byPurposeNotes
systemYouStanding instructions, persona, rules, output formatGoes first, stays byte-identical across turns — that is what makes prompt caching work
userThe human, or your harnessThe request, and any injected contextHarness-injected content lives here too; the model can’t tell the difference, which is both useful and a security problem
assistantThe modelIts reply: text, and/or tool_calls[]You append its own outputs back to keep continuity
toolYouThe result of a tool the model asked forMust carry the matching tool_call_id, or the call is malformed
The two consequences of statelessness

There is no conversation. The illusion of one is you re-sending the entire history every turn. The model has no memory between calls, no variables, no open files. If something is not in this request, it does not exist.

Cost grows quadratically. A 20-turn agent run does not cost 20 model calls — it costs the sum of 20 progressively larger inputs. A run whose context reaches 100k tokens has re-read most of those tokens dozens of times. This is why §04 exists, and why prompt caching is not a micro-optimisation.

Calling it through OpenRouter

OpenRouter is a single OpenAI-compatible endpoint in front of most commercial and open models. That matters for a design tool specifically: planning, patching, critique and image generation want different models, and here they are one client, one key, one bill, and a string change.

pythonthe primitive, with nothing around it
import os, httpx

OR = "https://openrouter.ai/api/v1"
HEADERS = {
    "Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}",
    "HTTP-Referer": "https://poster.studio",   # optional: app attribution
    "X-Title": "Poster Studio",                # shows in OpenRouter rankings
}

r = httpx.post(f"{OR}/chat/completions", headers=HEADERS, timeout=120, json={
    "model": "anthropic/claude-sonnet-4.5",
    "messages": [
        {"role": "system", "content": "You are a poster art director. Be terse."},
        {"role": "user",   "content": "Three concepts for a jazz night at a Nairobi rooftop bar."},
    ],
    "temperature": 0.8,
    "max_tokens": 600,
    "usage": {"include": True},        # ask for real cost back in the response
})
d = r.json()
print(d["choices"][0]["message"]["content"])
print(d["usage"])   # prompt_tokens, completion_tokens, cost in USD
Model IDs are vendor/model. Because the surface is OpenAI-compatible you can also point the official openai SDK at base_url="https://openrouter.ai/api/v1" and use it unchanged — convenient, and it means every code sample written for OpenAI works here.

Knobs that actually matter

temperature
Randomness. Near 0 for anything you will parse or diff; 0.7–1.0 for idea generation. In the poster tool: high for concepting, low for emitting layout patches.
max_tokens
A cost ceiling and a runaway guard, not a quality setting. Always set it.
seed
Best-effort reproducibility where the provider supports it. Useful for evals; never rely on it for correctness — batching and kernel non-determinism mean identical inputs can still diverge.
response_format
Schema-constrained output: {"type":"json_schema","json_schema":{"name":...,"strict":true,"schema":{...}}}. Enforcement varies by provider — some guarantee conformance, others treat the schema as a strong suggestion. Pair with provider.require_parameters: true so you only route to endpoints that really support it.
provider
OpenRouter’s routing preferences: order, allow_fallbacks, only/ignore, sort by price / throughput / latency, plus data_collection: "deny" and zdr for privacy constraints. The shortcuts :floor (cheapest) and :nitro (fastest) appended to a model ID do the common cases in one string.
Routing, for a design tool

By default OpenRouter load-balances across providers of the same model by price. That is fine for drafting and wrong for anything you are evaluating, because the endpoint serving you can change between runs and different providers of “the same” model differ in quantisation, context limit and parameter support. For eval runs and for anything user-facing that you have tuned, pin the provider with order and set allow_fallbacks: false, or accept that your measurements have a hidden variable in them.

02

Tools: how a model acts on the world

A tool is two things that live in different places: a JSON Schema the model reads, and a function you run. The gap between them is where every safety property you have comes from — and it is also the thing people accidentally close by wiring the model straight into their database.

The four-step dance

  1. You send the message list plus an array of tool schemas.
  2. The model replies with tool_calls: a name, an id, and arguments as a JSON string (not an object — it can be malformed, and sometimes is).
  3. You validate the arguments, decide whether to run it, run it, and capture the result.
  4. You append the assistant message and a tool message carrying the matching tool_call_id, then call the model again with the longer list.
AUTHOR messages[ ] you system “You are a poster art director…” byte-identical every turn → cacheable prefix you user “Make the headline bigger and check contrast” model assistant tool_calls: [{ id: "call_7f", name: "doc.patch", … }] arguments arrive as a JSON string — parse defensively you tool tool_call_id: "call_7f" · content: "patch applied, 2 ops" untrusted input: this text goes straight into the model model assistant “Headline is now 148pt; contrast passes at 6.1:1.” ids must match, or the call is rejected all five re-sent on turn 3. and turn 4. and turn 5. this is the whole cost model. Nothing here is state held by the model. It is a list in your process that you happen to re-transmit.
The single most useful mental correction: the “conversation” is a Python list you own. You can edit it, truncate it, reorder it, replace a tool result with a summary of itself, or drop turns entirely — and often you should.
pythona complete tool round trip
import json, httpx

TOOLS = [{
  "type": "function",
  "function": {
    "name": "check_contrast",
    "description": (
        "Compute the WCAG contrast ratio between two hex colours. "
        "Use before committing a text colour over a background. "
        "Returns the ratio and whether it passes AA for the given size."
    ),
    "parameters": {
      "type": "object",
      "properties": {
        "fg":       {"type": "string", "pattern": "^#[0-9a-fA-F]{6}$"},
        "bg":       {"type": "string", "pattern": "^#[0-9a-fA-F]{6}$"},
        "pt_size":  {"type": "number", "minimum": 4},
      },
      "required": ["fg", "bg", "pt_size"],
      "additionalProperties": False,
    },
  },
}]

def check_contrast(fg, bg, pt_size):
    def lin(c):
        c = c / 255
        return c / 12.92 if c <= 0.04045 else ((c + 0.055) / 1.055) ** 2.4
    def lum(h):
        r, g, b = (int(h[i:i+2], 16) for i in (1, 3, 5))
        return 0.2126*lin(r) + 0.7152*lin(g) + 0.0722*lin(b)
    a, b = sorted([lum(fg), lum(bg)], reverse=True)
    ratio = (a + 0.05) / (b + 0.05)
    threshold = 3.0 if pt_size >= 18 else 4.5
    return {"ratio": round(ratio, 2), "passes_aa": ratio >= threshold,
            "threshold": threshold}

REGISTRY = {"check_contrast": check_contrast}

msgs = [
  {"role": "system", "content": "You are a poster art director."},
  {"role": "user",   "content": "Is #6B5B2A text OK on #2A2418 at 14pt?"},
]

while True:
    r = httpx.post(f"{OR}/chat/completions", headers=HEADERS, timeout=120, json={
        "model": "anthropic/claude-sonnet-4.5",
        "messages": msgs, "tools": TOOLS, "tool_choice": "auto",
    }).json()

    m = r["choices"][0]["message"]
    msgs.append(m)                      # append the model's own turn, verbatim

    calls = m.get("tool_calls") or []
    if not calls:
        print(m["content"]); break      # ← the loop's only natural exit

    for c in calls:                     # models may request several at once
        try:
            args = json.loads(c["function"]["arguments"])   # can be malformed
            out  = REGISTRY[c["function"]["name"]](**args)
            body = json.dumps(out)
        except Exception as e:
            body = json.dumps({"error": str(e)})            # feedback, not a crash
        msgs.append({"role": "tool", "tool_call_id": c["id"], "content": body})
That is a working agent in about forty lines. Everything in §03 is this loop plus the things that stop it destroying itself: budgets, retries, timeouts, approvals, and a stop condition that isn’t “the model felt done”.

Designing tools the model can actually use

Tool descriptions are prompt. They are read by the model on every single call, they cost tokens on every single call, and they are the main lever on whether the agent behaves. Treat them like API docs written for a capable but literal-minded new hire who cannot ask you a question.

Naming
Verb-noun, in the domain’s language

doc.patch, render.preview, image.generate. Not process, handle, doThing. The model routes largely on the name; a vague name gets called at the wrong moment.

Description
Say when to use it, not just what it does

“Use after any layout change and before export” is worth more than a paragraph on the return type. Include one concrete example call if the arguments are at all subtle.

Schema
Constrain hard

Enums over free strings, patterns on ids and colours, additionalProperties: false, sensible minimum/maximum. Every constraint is one fewer retry loop and one fewer malformed call you have to hand-handle.

Granularity
Few, powerful, composable

Thirty narrow tools blow your token budget and your accuracy — selection error rises with the number of similar options. Prefer one doc.patch that takes an operation list over twenty setters.

Errors
Errors are prompts

“Invalid layer id ‘hdln’. Valid ids: headline, subhead, date_block.” recovers on the next turn. A stack trace or a bare 500 just burns two turns and then gives up.

Return values
Budget the output

A tool that returns 30k tokens of JSON has just eaten your context and will be re-read on every subsequent turn. Return summaries and handles; let the model ask for detail. “Grep, don’t cat.”

Side effects
Idempotency and approval

Models retry. Give mutating tools idempotency keys, and route anything expensive, destructive or public through an approval gate in the harness — not through a line in the system prompt asking nicely.

Determinism
Push work out of the model

Anything computable should be a tool, not a thought. Contrast ratios, text metrics, grid maths, colour conversion, price arithmetic. Models are bad at arithmetic and excellent at deciding which arithmetic to do.

Tools vs structured output

Two different jobs that look similar. Structured output (response_format: json_schema) constrains the model’s final answer into a shape you can parse — use it when you want data and the conversation is over. Tools are for when the model needs something to happen and then wants to keep going. In the poster tool: parsing a client brief into a structured spec is structured output; generating an image mid-design is a tool.

The security fact that governs everything downstream

Tool results are attacker-controlled input that you insert directly into the model’s context. A web page you fetched, a filename in a folder you listed, an EXIF comment in an uploaded logo, the description text of an MCP server you did not write — all of it lands in the same stream as your instructions, and the model has no reliable way to tell them apart. Prompt injection is not a solved problem and will not be solved by prompting. The mitigations are structural: least privilege on tools, approval gates on consequential actions, egress allowlists, and never granting one agent both access to secrets and the ability to send data outward.

03

The harness, from scratch

The loop in §02 works and would be irresponsible to ship. It has no budget, so a confused model can spend $40 in a minute. It has no timeout, so one slow tool hangs the request. It has no approval gate, no trace, no cancellation, and no way to resume. Adding those is the harness, and the harness is your product’s actual engineering.

ONE TURN, LEFT TO RIGHT assemble context §04 lives here call model + retry the only stochastic step parse & validate args malformed JSON → error message approval policy allow / ask / deny, per tool dispatch under timeout parallel calls, bounded append + truncate cap each result before it lands check budgets turns, cost, tokens, clock next turn — unless the model asked for no tools, or a budget tripped Six of the seven boxes are ordinary deterministic code. That ratio is the point: the model is a component inside your program, not the program.
Where each concern belongs. Notice that context assembly happens inside the turn, not once at the start — on turn 12 you get to decide again what the model should see, and that is your main tuning surface.

What the harness owns

ResponsibilityWhat it means concretelySkip it and…
BudgetsMax turns, max total tokens, max USD, wall-clock deadline — checked every turna loop burns your month’s credit in ten minutes
Stop conditionsNo tool calls; budget trip; explicit finish tool; no-progress detectionagents that never end, or end early and silently
RetriesBackoff on 429/5xx/timeouts, with a cap and jitter; distinguish retryable from fatalone blip fails a five-minute run
TimeoutsPer model call and per tool call, independentlya hung tool holds a request open forever
Approval policyPer-tool allow/ask/deny, evaluated before dispatch, with the arguments shownthe model publishes something, or spends money, unsupervised
Context assemblyDeciding what goes into this request — see §04quality degrades as runs get longer and nobody knows why
Result shapingTruncating and summarising tool output before it enters the listone ls of a big folder ends the run
PersistenceDurable run state so a crash or a page reload can resumeevery failure is a total loss of an expensive run
CancellationA cooperative stop that also aborts the in-flight model call“stop” doesn’t, and users lose trust immediately
StreamingSurfacing partial text and tool intentions as they arrivea 40-second blank screen, which reads as broken
TracingA span per model call and per tool call: model, tokens, cost, latency, cache hitsyou cannot debug, cost-attribute, or evaluate anything
pythonharness.py  ·  the loop, made survivable
import asyncio, json, time, uuid
from dataclasses import dataclass, field

@dataclass
class Budget:
    max_turns: int = 24
    max_cost_usd: float = 2.00
    max_seconds: float = 300.0
    spent_usd: float = 0.0
    started: float = field(default_factory=time.monotonic)

    def check(self, turn):
        if turn >= self.max_turns:                      return "max_turns"
        if self.spent_usd >= self.max_cost_usd:         return "cost_cap"
        if time.monotonic() - self.started > self.max_seconds: return "deadline"
        return None

@dataclass
class Tool:
    name: str
    schema: dict
    fn: object
    timeout: float = 30.0
    approval: str = "allow"        # allow | ask | deny
    max_result_chars: int = 6000

class Agent:
    def __init__(self, client, tools, system, *, on_event=None, on_approve=None):
        self.client, self.system = client, system
        self.tools = {t.name: t for t in tools}
        self.on_event   = on_event   or (lambda **kw: None)
        self.on_approve = on_approve or (lambda name, args: True)

    async def run(self, goal, *, model="anthropic/claude-sonnet-4.5",
                  budget=None, cancel: asyncio.Event = None):
        budget = budget or Budget()
        cancel = cancel or asyncio.Event()
        run_id = uuid.uuid4().hex[:8]
        msgs   = [{"role": "user", "content": goal}]

        for turn in range(budget.max_turns + 1):
            if cancel.is_set():                  return self._stop(msgs, "cancelled")
            if (why := budget.check(turn)):      return self._stop(msgs, why)

            # 1. assemble — the whole request is rebuilt here, every turn (§4)
            request = self.assemble(msgs, model)

            # 2. call the model, with bounded retries
            t0 = time.monotonic()
            reply, usage = await self._call_with_retry(request, cancel)
            budget.spent_usd += usage.get("cost", 0.0)
            self.on_event(type="model", run=run_id, turn=turn, model=model,
                          ms=int((time.monotonic()-t0)*1000), usage=usage,
                          spent=round(budget.spent_usd, 4))

            msgs.append(reply)
            calls = reply.get("tool_calls") or []
            if not calls:
                return self._stop(msgs, "complete", text=reply.get("content"))

            # 3-6. dispatch every requested call concurrently, each bounded
            results = await asyncio.gather(
                *(self._dispatch(c, run_id, turn) for c in calls))
            msgs.extend(results)

        return self._stop(msgs, "max_turns")

    async def _dispatch(self, call, run_id, turn):
        cid  = call["id"]
        name = call["function"]["name"]
        tool = self.tools.get(name)

        def result(body, ok=True):
            self.on_event(type="tool", run=run_id, turn=turn, tool=name, ok=ok)
            return {"role": "tool", "tool_call_id": cid,
                    "content": body[:tool.max_result_chars] if tool else body}

        if tool is None:
            return result(f"No such tool '{name}'. Available: "
                          f"{', '.join(sorted(self.tools))}", ok=False)
        try:
            args = json.loads(call["function"]["arguments"] or "{}")
        except json.JSONDecodeError as e:
            return result(f"Arguments were not valid JSON: {e}. Re-send them "
                          f"as a JSON object matching the schema.", ok=False)

        if tool.approval == "deny":
            return result(f"'{name}' is not permitted in this run.", ok=False)
        if tool.approval == "ask" and not await _maybe_await(
                self.on_approve(name, args)):
            return result(f"The user declined '{name}'. Try another approach "
                          f"or ask them what they would prefer.", ok=False)

        try:
            out = await asyncio.wait_for(_maybe_await(tool.fn(**args)),
                                         timeout=tool.timeout)
            return result(out if isinstance(out, str) else json.dumps(out))
        except asyncio.TimeoutError:
            return result(f"'{name}' timed out after {tool.timeout}s. "
                          f"Try a narrower request.", ok=False)
        except Exception as e:
            # the model can often fix this itself — hand it back, don't raise
            return result(f"{type(e).__name__}: {e}", ok=False)
Every failure path returns a message the model can read and act on rather than raising. That single convention is most of what separates an agent that recovers from one that dies on the first bad argument — and it is why isError: true exists in MCP’s tool results rather than a JSON-RPC error.

Failure taxonomy

FailureWhere it shows upHandle it by
Rate limit / 5xx / provider hiccupModel callRetry with jittered backoff; consider an OpenRouter fallback provider
Malformed tool argumentsParse stepReturn the parse error as a tool message; tighten the schema
Tool raisesDispatchReturn the exception text as a tool result, not an HTTP 500
Tool hangsDispatchPer-tool timeout; tell the model it timed out so it narrows the request
Loops — same call, same args, repeatedlyAcross turnsHash the last N calls; on repeat, inject “you have tried this twice, it did not work” and force a different path
Context overflowAssembleCompact before you hit the wall, not after the API rejects you (§04)
Silent quality decay on long runsNowhere — that is the problemTraces plus an eval suite (§07). You will not notice this by using the product.
Persistence, and why it comes earlier than you think

Make the run resumable from the beginning: a row per run with the message list, the budget state, and a monotonically increasing turn index, written after every turn. It costs an afternoon and it buys you crash recovery, page-reload survival, background execution, a debug view of exactly what the model saw, and the ability to fork a run from turn 7 to try a different prompt. Retro-fitting it after the loop is entangled with your web request is genuinely painful.

04

Context engineering

This is the discipline that separates a demo from a product. Everything else in this document is mechanism; this is judgement. The question is always the same and you answer it fresh on every single turn: given a fixed budget, what are the most useful tokens I can put in front of this model right now?

A budget, not a container

Two independent pressures push in opposite directions. Larger context costs more money and more latency on every turn, and it grows monotonically unless you intervene. And separately — the part people underestimate — model quality degrades well before the window is full. Attention is finite and diluted; instructions given at turn 1 compete with 90k tokens of tool output by turn 20; models reliably retrieve a fact from the start and the end of a long input and much less reliably from the middle. A 1M-token window is not permission to use 900k of it.

SAME RUN, SAME TURN 30, 200K WINDOW unmanaged raw tool results · 110k history · 60k 184k used managed headroom 36k used measured quality falls off around here — long before the hard limit 0 50k 100k 150k 200k system + tool schemas (stable → cached) state projection conversation history tool results unused
The difference between the two bars is not a bigger model or a better prompt. It is that the design document lives in a store and the agent holds a handle to it, so the same work happens at a fifth of the cost, faster, and with the instructions still salient.

Four moves, in order of leverage

Move 1
Offload

Keep the artifact outside the context. The design document, the asset library, the search results, the file — live in a store; context carries a compact projection and a handle. The model asks for detail when it needs detail.

Highest leverage by a distance, and the one that changes your architecture. MCP’s own guidance now says the same thing: return a handle from a creation tool and take it as an argument later.

Move 2
Select

Retrieve only what this turn plausibly needs. Three relevant reference posters beat forty. A ranked search result beats a directory listing. If you cannot articulate why a token is in the request, take it out.

Move 3
Compress

Summarise old turns into a durable brief once history crosses a threshold; drop superseded tool results entirely (an old render preview has no value once a newer one exists). Compact before you hit the wall, on your schedule, not the API’s.

Move 4
Isolate

Give a sub-task its own fresh window and return only its conclusion. “Research five poster references and report three” costs 40k tokens in a sub-agent and adds 400 to the parent. See §06 — and note the cost: whatever the sub-agent saw is gone.

Prompt caching: the discipline that pays for itself

Providers cache the model’s internal state for a repeated prefix of your input, and charge a fraction of the normal input price for the cached portion. On an agent loop, where turn 12 re-sends everything turns 1–11 sent, this is not a micro-optimisation — it is often the majority of your bill and a large share of your latency.

The rule is simple and unforgiving: a cache hit requires a byte-identical prefix. So:

  • Order by stability. System prompt first, then tool schemas, then long-lived context, then the volatile conversation. One dynamic token near the front invalidates everything behind it.
  • Never put a timestamp, a random id, or a “current time” line in the system prompt. This is the single most common way teams silently disable their own cache. Put it in the last user message instead.
  • Serialise tool schemas deterministically — same order, same key order, every time. This matters enough that the 2026-07-28 MCP specification now says servers should return tools from tools/list in a deterministic order explicitly to improve prompt cache hit rates.
  • Append, don’t rewrite. Editing an early message to “clean up” history costs you the whole cache. If you must compact, do it rarely and deliberately, and accept the one-off miss.

Tool results are where the budget actually goes

In practice most context bloat is not conversation — it is tool output. A directory listing, a full JSON document, an un-truncated API response, a page of HTML. Rules that hold up:

Instead ofDo thisWhy
Returning the whole design document after every editReturn {"ok":true,"ops":2,"doc_version":18} and let a separate doc.summary tool fetch a projectionThe doc is the artifact, not the conversation. It can be 100k tokens.
Dumping a search result pageReturn the top 5 with title, one line, and an id to fetchSelection is your job, not the model’s
Returning a full-resolution renderReturn a downscaled preview (long edge ~1024) plus deterministic check resultsImages are expensive in tokens and the model does not need print resolution to judge composition
Truncating silently at N charactersTruncate and say so: “[showing 40 of 312 results; call again with offset]”A model that knows it was truncated will paginate; one that doesn’t will conclude there were 40
Keeping every historical tool resultReplace superseded ones with a one-line stubOnly the newest render matters; the previous six are pure cost
The pattern that makes the poster tool work at all

State lives in a store; context holds a projection. The design document — every layer, every coordinate, every colour — is a JSON structure in your database that can run to tens of thousands of tokens. It never goes into context in full. What the agent sees each turn is a compact projection: canvas size, the palette, a one-line summary per layer, the current version number, and the last render as an image. It edits by emitting patches against layer ids, and the store applies them.

This gives you the same four properties the state-handle pattern gives an MCP server: bounded context, atomic edits, a version history you can undo, and the ability for a human to edit the document directly between turns without the agent’s context going stale — because the agent re-reads the projection at the top of every turn anyway.

A worked budget for the poster agent

SlotBudgetContentsCache
System prompt~1.5kRole, design principles, hard rules, output conventionsstable
Tool schemas~3k9–12 tools, deterministically orderedstable
Brief~1kThe parsed client brief, as structured datastable per run
Doc projection~2–4kCanvas, palette, type pair, one line per layer, versionchanges each turn
Latest render~1.5kOne downscaled preview imagechanges
Check results~0.5kDeterministic lint output: contrast, overflow, margins, DPIchanges
Recent turns~6kLast few exchanges verbatim; older ones compacted to a briefrolling
Total~16–18kFlat across the run. It should not grow with turn count — if it does, one of the four moves is missing.
05

MCP, as of the 2026-07-28 revision

The Model Context Protocol solves one specific problem: N agent applications each needing bespoke integrations with M systems is N×M pieces of glue, and every one of them is written twice. MCP makes it N+M by standardising how a capability describes itself and how it is invoked. That is the whole pitch. It is a plug shape, not a theory of agents.

Read this before the rest of the section

MCP changed substantially in the 2026-07-28 revision, and a great deal of writing about it — including plenty published this year — describes a protocol that no longer exists. The headline: MCP is now stateless. There is no initialize handshake, no Mcp-Session-Id, and the list endpoints no longer vary per connection. Separately, Roots, Sampling and Logging are deprecated, with the suggested migration for Sampling being “integrate directly with LLM provider APIs instead”. If a tutorial opens with an initialize handshake or has your server calling back into the model, it is out of date.

The topology, and the misconception it fixes

the model OpenRouter / provider API not MCP — an ordinary HTTP API HOST APPLICATION · POSTER STUDIO harness · agent loop · context assembly decides what to call and whether to allow it MCP client 1 MCP client 2 MCP client 3 one client per server render-server · stdio asset-library · streamable HTTP brand-db · streamable HTTP JSON-RPC 2.0 files · GPUs SaaS APIs the internet untrusted There is no arrow from the model to a server. Every capability reaches the model only as a tool schema your harness chose to include, and every call is dispatched by your code, which can refuse.
“Connecting a model to MCP” is not a thing that happens. The host application connects to servers, decides which of their tools to expose, and mediates every call. With Sampling deprecated, the arrow that used to run the other way — a server asking the host to run an inference — is on its way out too.

The primitives

PrimitiveControlled byWhat it isStatus
ToolsThe modelCallable functions with an inputSchema, optional outputSchema, and structured or unstructured resultsthe main event
ResourcesThe applicationReadable data addressed by URI, listed and fetched by the host and injected as contextactive
PromptsThe userNamed, parameterised templates a server offers — slash commands, essentiallyactive
ElicitationThe server, mid-call“I need input from the human before I can finish”, now delivered through the MRTR pattern belowactive, reshaped
TasksThe serverLong-running work, polled via tasks/get with tasks/update for inputmoved to an official extension
SamplingThe serverServer asks the host to run an LLM completion on its behalfdeprecated — call a provider API directly
RootsThe clientTelling a server which directories it may operate indeprecated — pass paths as tool arguments
LoggingThe servernotifications/message log recordsdeprecated — use stderr or OpenTelemetry

Deprecated features stay functional for a minimum twelve-month window under the new lifecycle policy, but new implementations should not adopt them.

What changed, and why it matters to you

Stateless by default
The initialize/notifications/initialized handshake and the session header are gone. Every request carries its own protocol version, client identity and capabilities in _meta. Practically: a server is now an ordinary stateless HTTP service you can put behind a load balancer without sticky sessions.
server/discover
Servers must implement it; clients may call it up front to negotiate versions and read capabilities and identity. It replaces the handshake’s discovery role without creating a connection.
Multi Round-Trip Requests
The big conceptual change. Instead of a server initiating a request back at the client, it returns a result with resultType: "input_required" and an inputRequests map. The client gathers what was asked for and retries the original request with inputResponses and the opaque requestState the server handed back. One direction of travel, always.
resultType on every result
"complete" or "input_required". Results from older servers that omit it must be treated as "complete".
subscriptions/listen
One long-lived POST-response stream replaces the old HTTP GET endpoint and resources/subscribe. Clients opt in to specific notification types. Request-scoped notifications like notifications/progress still ride the response stream of their own request.
Cacheable lists
tools/list, prompts/list, resources/list and resources/read now return ttlMs and cacheScope, and servers should return tools in a deterministic order — explicitly to help clients cache and to improve LLM prompt cache hit rates. §04’s discipline, written into the protocol.
No stream resumability
Last-Event-ID and SSE event ids are gone from Streamable HTTP. A broken stream loses the in-flight request and the client re-issues it with a new id. Design your tools to be safely re-runnable.
Transports
stdio for local subprocesses, Streamable HTTP for everything remote. The old HTTP+SSE transport is formally deprecated.
Authorization
OAuth 2.x, with Client ID Metadata Documents now preferred over Dynamic Client Registration, and clients required to validate the iss parameter when present. Credentials are bound to the issuing authorization server and must be keyed by issuer.

Stateful work without protocol state

Because there is no session, a server that needs continuity across calls returns an explicit handle from a creation tool and accepts it as an ordinary argument later. This is exactly the “offload” pattern from §04, and it is what the poster document does:

jsonstate as a handle the model carries forward
// → tools/call
{ "name": "doc.create", "arguments": { "preset": "A2_portrait" } }

// ← result
{ "resultType": "complete",
  "content": [{ "type": "text", "text": "Created document doc_9fk2 (A2, 300dpi)" }],
  "structuredContent": { "doc_id": "doc_9fk2", "version": 1 } }

// → tools/call  — the model carries doc_id forward; the server looks it up
{ "name": "doc.patch",
  "arguments": { "doc_id": "doc_9fk2", "ops": [
      { "op": "set", "path": "/layers/headline/size_pt", "value": 148 } ] } }
The spec’s own guidance on handles is worth following: validate authorization against the handle on every call (a handle is a name, not a capability), keep it opaque, give it a bounded lifetime, state that lifetime in the tool description so the model can see it, and return an actionable error when it expires.

The wire, exactly

jsontools/list and tools/call under 2026-07-28
// → request. _meta is required and carries what the handshake used to.
{ "jsonrpc": "2.0", "id": 1, "method": "tools/list",
  "params": {},
  "_meta": {
    "io.modelcontextprotocol/protocolVersion": "2026-07-28",
    "io.modelcontextprotocol/clientInfo": { "name": "poster-studio", "version": "0.4.1" },
    "io.modelcontextprotocol/clientCapabilities": {}
  } }

// ← result
{ "jsonrpc": "2.0", "id": 1, "result": {
    "resultType": "complete",
    "tools": [{
      "name": "render.preview",
      "title": "Render preview",
      "description": "Rasterise a design document to a downscaled PNG. Call after any layout change, before judging composition.",
      "inputSchema": {
        "type": "object",
        "properties": {
          "doc_id":   { "type": "string" },
          "long_edge":{ "type": "integer", "minimum": 256, "maximum": 2048, "default": 1024 }
        },
        "required": ["doc_id"],
        "additionalProperties": false
      },
      "outputSchema": {
        "type": "object",
        "properties": { "width": {"type":"integer"}, "height": {"type":"integer"},
                        "version": {"type":"integer"} },
        "required": ["width","height","version"]
      }
    }],
    "ttlMs": 300000,
    "cacheScope": "public"
} }

// ← a tool result carrying an image the model can actually look at
{ "jsonrpc": "2.0", "id": 2, "result": {
    "resultType": "complete",
    "content": [
      { "type": "text",  "text": "Rendered doc_9fk2 v18 at 1024x1448." },
      { "type": "image", "data": "iVBORw0KGgoAAA...", "mimeType": "image/png",
        "annotations": { "audience": ["assistant"], "priority": 0.9 } }
    ],
    "structuredContent": { "width": 1024, "height": 1448, "version": 18 },
    "isError": false
} }
Two error channels, deliberately. A JSON-RPC error means the request was structurally wrong — unknown tool, malformed params — and the model probably cannot fix it. A normal result with isError: true means the tool failed in a way the model can act on, and its text is written for the model to read. Use the second one far more often.
pythona server, using the Python SDK
from mcp.server.fastmcp import FastMCP, Image

mcp = FastMCP("poster-render")

@mcp.tool()
def render_preview(doc_id: str, long_edge: int = 1024) -> list:
    """Rasterise a design document to a downscaled PNG.

    Call after any layout change and before judging composition.
    Returns the preview image plus its pixel dimensions and doc version.
    """
    doc = STORE.get(doc_id)                      # server-side state, by handle
    if doc is None:
        raise ValueError(f"Unknown doc_id {doc_id!r}. Call doc.create first.")
    png, w, h = rasterise(doc, long_edge=long_edge)
    return [
        f"Rendered {doc_id} v{doc.version} at {w}x{h}.",
        Image(data=png, format="png"),
    ]

if __name__ == "__main__":
    mcp.run()          # stdio by default; streamable HTTP for remote
The SDK generates inputSchema from the type hints and the description from the docstring, so the docstring is prompt engineering — write it for the model. Pin your SDK version and read its changelog: the surface tracks a spec that has been moving quickly.

When to reach for MCP — and when not to

Use MCP
The capability crosses an application boundary

You want your render service usable from your own app, from Claude Code, from a colleague’s agent, from a CI job. Or you are consuming someone else’s server. Or the tool is a separate process for isolation reasons, and you would otherwise be inventing an RPC protocol.

Don’t
It’s a function in your own codebase

check_contrast() does not need a JSON-RPC server, a subprocess and a schema round trip. Put it in your tool registry directly. Wrapping in-process functions in MCP for its own sake buys serialisation cost, an extra failure mode, and a deployment artifact, in exchange for nothing.

Careful
Third-party servers are third-party code

A server’s tool names, descriptions and annotations are text that lands in your model’s context, and the spec is explicit that clients must treat annotations from untrusted servers as untrusted. A malicious description is a prompt injection with a delivery mechanism. Review what you install, pin versions, and keep a server that reads your files away from one that can post to the internet.

Practical
Token cost is real

Every connected server’s tool list is re-sent on every turn. Ten servers with twelve tools each is a permanent tax on every request and a measurable drop in tool-selection accuracy. Expose a curated subset per task rather than everything you happen to have connected.

06

Multi-agent orchestration

The honest starting position: one well-built loop with good tools beats a multi-agent system almost every time, and most multi-agent architectures are a context problem being solved with an org chart. Reach for a second agent when you can name the specific property you are buying.

A · ONE LOOP agent tools one context, full history, nothing lost at a boundary. default. start here. B · ORCHESTRATOR → WORKERS orchestrator refs own window palette own window copy own window scoped brief down 40k tokens each, 400 back. parallel · independent · N× cost whatever a worker saw is gone. C · MAKER ⇆ CRITIC maker critic the artifact doc + render + checks writes reads findings a fresh critic has no sunk cost in the design. bound it: 2–3 rounds. the one that reliably pays off.
The artifact in panel C is the important detail. Maker and critic do not converse — they both operate on a shared, inspectable document, so the handoff is a file rather than a summary, and nothing is lost in translation.
PatternBuy thisPay thatUse it in the poster tool for
Single loopSimplicity, full history, one place to debugOne context window for everythingEverything, until measurement says otherwise
Orchestrator–workerParallelism; a clean window per subtaskN× tokens; workers can’t see each other; handoff lossConcept exploration — three directions researched at once, three one-paragraph reports back
PipelineA different model and prompt per stage; cheap stages stay cheapRigid; errors compound down the chainbrief → concept → layout → imagery → export
Maker–criticA genuine quality lift; the critic isn’t anchored on the maker’s reasoning2–3× cost per round; can oscillate if unboundedDesign review against the brief and the deterministic checks
RouterCheap model handles the 80% of easy requestsMisroutes; two behaviours to keep consistent“make the headline bigger” → small fast model; “redesign this” → the good one
The handoff rule

What crosses an agent boundary is a schema’d artifact, never a conversation. A worker returns {concept, rationale, palette, refs[]} validated against a schema — not a chat transcript, not “here’s what I found” prose. Two reasons: the parent’s context stays bounded and predictable, and the boundary becomes testable, so you can evaluate a worker in isolation instead of only observing that the whole system got worse.

The corollary people miss: a sub-agent that was given a vague brief produces a confident, wrong artifact, and the parent has no way to tell. Sub-agent briefs need to be more specific than what you would say to a human, including what to return and what not to bother with.

07

Evals and observability

You are shipping a system whose output changes when nothing changed. Every instinct from deterministic software — a test asserts an exact value, a green build means it works — fails here. What replaces them is measurement over distributions, and a large pile of deterministic checks around a small stochastic core.

Trace first. You cannot evaluate what you cannot see.

Before any eval work, emit a span per model call and per tool call. OpenTelemetry is the sane default, and the 2026-07-28 MCP spec even documents trace-context propagation through _meta (traceparent, tracestate, baggage) so a trace can follow a call into a server. Each span carries:

FieldWhy you will want it at 2am
run_id, turn, parent_spanReconstruct the exact sequence, including sub-agents
model, providerOpenRouter may have routed you somewhere different than yesterday
prompt_tokens, completion_tokens, cached_tokensCache hit rate is the difference between a viable and an unviable margin
cost_usdPer run, per user, per feature. Ask for it with usage: {include: true}.
latency_ms, ttft_msTime-to-first-token is what users actually feel
tool_name, ok, error_classTool error rate is the single best early-warning signal you have
Full request and response bodiesSampled, redacted, retained briefly. Reading what the model actually saw resolves most “why did it do that” questions in a minute.

Three levels of evaluation

Level 1 · deterministic
Unit tests, no model involved

check_contrast, the grid maths, the patch applier, the renderer. These are ordinary pure functions and should have ordinary fast tests with exact assertions. Most of your code is here and it should be.

Level 2 · component
Does the model make the right move?

Fixed context in, one turn out. Did it pick the right tool? Were the arguments valid against the schema? Did it call render.preview before claiming the layout was fine? Cheap, fast, run on every PR, assert on a pass rate over ~30 cases rather than on a single run.

Level 3 · end to end
Is the poster any good?

A whole run against a real brief. Slow and expensive, so it runs nightly and before releases, over a fixed golden set. This is where judgement is needed, and where the trick below matters.

Judging a poster

“Is this design good?” sounds like it needs a model to answer. Most of it does not, and the parts that do not are the parts you can trust. Split the question:

CheckHowVerdict
Text contrast against its backdropWCAG ratio computed from the rendered pixels under each text layerdeterministic
Text overflow, clipping, collisionMeasure laid-out text boxes; compare against layer bounds and each otherdeterministic
Safe margins and bleedGeometry against the print spec (3mm bleed, 5mm safe area for A2)deterministic
Minimum legible type sizePoint size at final trim dimensions, per layer roledeterministic
Effective image resolutionSource pixels ÷ placed physical size ≥ 300 dpideterministic
Brief compliance — date, venue, price, logo present and correctString and asset presence in the document, not in the imagedeterministic
Palette conformance to brandColour distance from the allowed setdeterministic
Visual hierarchy — does the eye land on the headline first?Vision model, rubric, pairwisejudged
Does it feel like “late-night jazz” rather than “corporate seminar”?Vision model, rubric, pairwisejudged
Is it good?A human, sampledunavoidable
Why the deterministic list is the valuable half

Roughly nine out of ten genuinely bad outputs from a design agent fail one of the mechanical checks: unreadable text, a headline running off the canvas, a logo at 72dpi, the wrong date. These are cheap to detect, unambiguous, and — the important part — you can hand the failures straight back to the agent as a tool result and let it fix them in the loop. Deterministic checks are not just an eval; they are the feedback signal that makes the agent good in the first place.

Making a model judge less unreliable

  • Compare, don’t score. “Which of these two better fits the brief?” is far more stable than “rate this 1–10”, which drifts between runs and clusters around 7.
  • Randomise position in pairwise comparisons and run both orders — judges have a real preference for whichever came first.
  • Give the judge the rubric and the brief, ask for a verdict per criterion with a one-line reason, and use a different model from the maker. Self-preference is measurable.
  • Calibrate against humans once. Have a designer rank 40 outputs, then check your judge’s agreement. If it is near chance, the judge is noise dressed as a number and you should say so out loud.
  • Track disagreement as a metric. Where the judge and the humans diverge is exactly where the rubric is wrong.

The golden set, and what to assert

Thirty to fifty real briefs, versioned in the repo, covering the boring middle and the edges: very long headlines, Swahili and English mixed, a client logo that is a wide horizontal lockup, a brief with no imagery budget, a brief that contradicts itself. For each, assert properties, never outputs:

pythonevals/test_golden.py
import pytest
from studio import run_brief, checks

GOLDEN = load_briefs("evals/briefs/*.yaml")

@pytest.mark.parametrize("brief", GOLDEN, ids=lambda b: b.id)
def test_brief_produces_a_shippable_poster(brief):
    result = run_brief(brief, seed=7, budget_usd=1.50)

    # hard gates — a failure here is a release blocker
    assert result.stop_reason == "complete"
    assert result.cost_usd <= 1.50
    report = checks.run_all(result.doc)
    assert report.contrast_failures == []
    assert report.overflow_failures == []
    assert report.min_effective_dpi >= 300
    for fact in brief.must_appear:            # date, venue, price, age rating
        assert fact in result.doc.text_content()

    # soft signals — recorded per run, alerted on trend, never asserted
    metrics.record(brief.id, turns=result.turns, cost=result.cost_usd,
                   seconds=result.seconds, judge=judge_vs_baseline(result))
Hard gates are things that are never acceptable and are deterministically checkable. Everything aesthetic goes into recorded metrics with a trend alert. A test that fails when a model has an off day teaches your team to ignore the test suite, which is worse than not having one.

The metric that beats all the others

Once real users are on it: the human edit rate. What fraction of agent-produced layers does a designer change before exporting, and which ones? If they always fix the type size, your system prompt or your type tool is wrong. If they always regenerate the image, your imagery prompt is wrong. It is free to collect, impossible to game, and it points at the specific broken thing rather than at a number going down.

08

The poster tool: architecture

Now spend all of it on one thing. The goal: a client types a brief, an agent designs an A2 poster, a human nudges it, and a print-ready PDF comes out. The single decision that determines whether this works is what the agent actually produces.

The fork: what does the model emit?

ApproachModel emitsText qualityEditable after?Cost per iterationPrint-ready?
Pure image generationA prompt; you get pixels mangled — typography is where image models still fall down, and a poster is typography No — a flat raster$0.03–0.15 × full regenerateNo — fixed resolution, RGB, no bleed
Pure structured layoutA document: layers, type, shapes perfect — real fonts, real kerning Yes, fullyfractions of a centYes
Hybrid  chosen A document whose image layers reference generated assets perfect — text never goes through the image model Yes, every layer independently cheap to nudge, pay only to regenerate artYes

The hybrid is how design software works and it is not a compromise — it is strictly better on every axis that matters for a poster. It also gives you the property the whole system hangs on: the design is data, so it can be diffed, versioned, undone, linted, replayed, handed to a human mid-flight, and rendered at any resolution.

client brief brief.parse structured output, temp 0 3 concept directions temp 0.9, human picks one doc.create the agent sees: projection (~3k) + preview + findings never the whole doc the design document canvas · grid · palette · type pair · layers[] versioned, undoable, human-editable ~40k tokens · never enters context doc.patch(ops) render.preview 1024px PNG, deterministic checks.run contrast · overflow · dpi · bleed the feedback loop: the agent looks at the render and reads the findings, then patches again image.generate POST /api/v1/images · base64 back asset store layers reference an asset_id, not bytes render.export PDF · 300dpi · CMYK · 3mm bleed Everything neutral is deterministic code. Everything violet is a model call. The document sits between them and is the only thing that persists.
The mechanism that makes this work is the loop at the bottom: the agent sees its own output. A render comes back as an image content block the model can actually look at, alongside machine-computed findings. Without that, an agent designing a poster is writing coordinates blind.

The design document

This schema is the product. It is what your renderer consumes, what your React canvas edits, what the agent patches, what your checks lint, and what you version. Design it first and change it rarely.

pythondoc/schema.py  ·  pydantic, because it gives you the JSON Schema for free
from pydantic import BaseModel, Field
from typing import Literal, Union, Annotated

Mm = float                            # one unit everywhere: millimetres

class Canvas(BaseModel):
    width_mm: Mm = 420.0              # A2 portrait
    height_mm: Mm = 594.0
    bleed_mm: Mm = 3.0
    safe_mm: Mm = 10.0
    dpi: int = 300
    colour_space: Literal["sRGB", "CMYK"] = "CMYK"

class Grid(BaseModel):
    columns: int = 6
    gutter_mm: Mm = 6.0
    margin_mm: Mm = 18.0
    baseline_mm: Mm = 4.0             # everything snaps to this

class TypeScale(BaseModel):
    display: str                      # font family, must exist in the font set
    text: str
    ratio: float = 1.333              # sizes are derived, not chosen ad hoc
    base_pt: float = 11.0

class Box(BaseModel):
    x_mm: Mm; y_mm: Mm; w_mm: Mm; h_mm: Mm
    rotation_deg: float = 0.0

class TextLayer(BaseModel):
    kind: Literal["text"] = "text"
    id: str                           # stable, human-readable: "headline"
    role: Literal["headline","subhead","body","credit","caption"]
    box: Box
    content: str
    step: int = 0                     # position on the type scale, not a pt size
    weight: int = 400
    tracking: float = 0.0
    leading: float = 1.15
    colour: str = Field(pattern=r"^#[0-9a-fA-F]{6}$")
    align: Literal["left","center","right"] = "left"

class ImageLayer(BaseModel):
    kind: Literal["image"] = "image"
    id: str
    box: Box
    asset_id: str                     # a handle. bytes live in the asset store.
    fit: Literal["cover","contain"] = "cover"
    opacity: float = 1.0
    blend: Literal["normal","multiply","screen"] = "normal"

class ShapeLayer(BaseModel):
    kind: Literal["shape"] = "shape"
    id: str
    box: Box
    shape: Literal["rect","ellipse","line"]
    fill: str | None = None
    stroke: str | None = None
    stroke_mm: Mm = 0.0

Layer = Annotated[Union[TextLayer, ImageLayer, ShapeLayer],
                  Field(discriminator="kind")]

class Document(BaseModel):
    doc_id: str
    version: int = 1
    canvas: Canvas = Canvas()
    grid: Grid = Grid()
    type_scale: TypeScale
    palette: list[str]                # ordered: ground, ink, accent, …
    layers: list[Layer]               # painted back to front
Three choices worth arguing about, all made deliberately. Millimetres, not pixels — the output is a physical object, and pixel coordinates make DPI and bleed unrepresentable. Type step, not size_pt — the model picks a rung on a scale rather than inventing 47.3pt, which is most of what makes generated typography look amateur. Layers reference asset_id — so a 4MB image never touches the document, the context, or a diff.

The tool catalogue

Twelve tools. Fewer than you expect, deliberately: every extra tool costs tokens on every turn and dilutes selection accuracy. Each row here carries the same six commitments a plugin extension point does — signature, timing, failure mode, budget, cost, permission.

ToolDoesReturnsCost / latencyApproval
doc.createNew document from a preset (A2, A1, 1080×1350…)doc_id, version~0 · 20msallow
doc.summaryThe projection the agent reads each turn~2–4k tokens of structure, no coordinates dumped~0 · 10msallow
doc.patchApply an op list: set / add_layer / move / remove{ok, ops_applied, version}not the document~0 · 15msallow
doc.undoRevert to a previous versionNew version number~0 · 10msallow
type.pairSuggest display + text families from the licensed set, with a rationale3 candidate pairs~0 · 5msallow
palette.buildDerive a palette from a mood, a brand colour, or a source imageOrdered hex list + contrast matrix~0 · 30msallow
layout.gridSnap layers to the column grid and baselineWhich layers moved, by how much~0 · 20msallow
render.previewRasterise at 1024px long edgePNG image content block + dimensions~0 · 300msallow
checks.runThe deterministic lint suite from §07Findings list, each with a layer id and a suggested fix~0 · 150msallow
image.generateOpenRouter /api/v1/images; stores the resultasset_id, dimensions, cost — not the bytes$0.02–0.15 · 5–25sask above a per-run count
image.editSame endpoint with input_references — variations, extend, restyleasset_id$0.02–0.15 · 5–25sask
render.exportPrint-ready PDF: 300dpi, CMYK, bleed, crop marks, fonts outlinedFile handle + a preflight report~0 · 3–8sask — this is the “done” button
Why doc.patch and not doc.write

Letting the model re-emit the whole document every time it wants to change a font size is the most expensive mistake available here. It costs thousands of output tokens per edit, it is slow, it silently loses layers the model forgot to include, it produces useless diffs, and it makes concurrent human editing impossible. A patch is small, atomic, attributable, reversible, and cheap to validate — and it lets a designer drag a layer in the canvas at the same moment the agent is adjusting a colour.

Model routing on OpenRouter

Four jobs with genuinely different requirements, one client, four model ids. This is the concrete reason to sit behind an aggregator rather than a single vendor.

JobWantsSettingsRuns
Brief parsingReliable schema conformance, cheapSmall model, temperature: 0, response_format: json_schema with strict, provider.require_parameters: trueonce
ConceptingRange and tasteStrong reasoning model, temperature: 0.9, three samples in parallelonce
Layout & patchingInstruction-following, tool use, speed — this is the hot loopMid-tier model, temperature: 0.2, prompt caching on, :nitro if latency bites10–30×
CritiqueVision, and a different lineage from the makerA vision model from another vendor, rubric in the prompt, pairwise where possible2–3×
ImageryArt direction, style control, reference following/api/v1/images, aspect_ratio matched to the layer box, seed pinned for reproducibility, input_references for variations1–5×
Two OpenRouter specifics that will bite you

Pin providers for anything you measure. The default is price-based load balancing across providers of the same model, and providers differ in quantisation, context limit and which parameters they support. Set provider.order and allow_fallbacks: false in evals, or your regression suite has a hidden variable.

Images come back as base64, not URLs. The response is data[].b64_json with a media_type; storage is yours. Decode, hash, write to object storage, and return an asset_id to the model — never let a megabyte of base64 anywhere near the message list.

The human in the loop

The designer is not an approver at the end; they are a participant. Three gates, and the third is the one that makes the product feel like a tool rather than a slot machine:

  • Concept selection. Three directions with rationales; the human picks one. Cheap, fast, and it collapses the largest source of wasted work.
  • Spend approval. Image generation and export go through the harness’s ask policy, with the prompt and estimated cost shown.
  • Direct editing, mid-run. Because the document is the shared artifact and the agent re-reads its projection each turn, a designer can drag a layer, retype a headline, or lock a layer while the agent is working. Add a locked: true flag that doc.patch refuses to override, and the collaboration works without any coordination protocol at all.
09

The poster tool: the code

Enough of it to see how the pieces fit. Everything here plugs into the Agent class from §03 and the schema from §08.

The patch applier — the tool that does most of the work

pythontools/doc_patch.py
from pydantic import ValidationError

ALLOWED_FIELDS = {"box", "content", "step", "weight", "tracking", "leading",
                  "colour", "align", "opacity", "blend", "fit", "asset_id",
                  "fill", "stroke", "stroke_mm", "palette", "type_scale"}

def doc_patch(doc_id: str, ops: list[dict]) -> dict:
    """Apply a list of edit operations to a design document.

    Each op is one of:
      {"op":"set",       "layer":"headline", "field":"step", "value":4}
      {"op":"add_layer", "layer":{...full layer object...}, "after":"headline"}
      {"op":"move",      "layer":"credit", "dx_mm":0, "dy_mm":-12}
      {"op":"remove",    "layer":"stray_rect"}
    Operations apply in order and all-or-nothing: if any op is invalid the
    document is unchanged and the error tells you which op and why.
    """
    doc = STORE.load(doc_id)
    if doc is None:
        raise ValueError(f"Unknown doc_id {doc_id!r}. Call doc.create first.")

    draft   = doc.model_copy(deep=True)
    by_id   = {l.id: l for l in draft.layers}
    applied = 0

    for i, op in enumerate(ops):
        kind = op.get("op")
        if kind in ("set", "move", "remove"):
            layer = by_id.get(op.get("layer"))
            if layer is None:
                raise ValueError(
                    f"op[{i}]: no layer {op.get('layer')!r}. "
                    f"Layers are: {', '.join(sorted(by_id))}")
            if getattr(layer, "locked", False):
                raise ValueError(
                    f"op[{i}]: layer {layer.id!r} is locked by the designer. "
                    f"Leave it alone and adjust something else.")

        if kind == "set":
            if op["field"] not in ALLOWED_FIELDS:
                raise ValueError(f"op[{i}]: field {op['field']!r} is not "
                                 f"editable. Editable: {sorted(ALLOWED_FIELDS)}")
            setattr(by_id[op["layer"]], op["field"], op["value"])
        elif kind == "move":
            b = by_id[op["layer"]].box
            b.x_mm += op.get("dx_mm", 0.0); b.y_mm += op.get("dy_mm", 0.0)
        elif kind == "remove":
            draft.layers = [l for l in draft.layers if l.id != op["layer"]]
        elif kind == "add_layer":
            new = parse_layer(op["layer"])                 # validates the union
            if new.id in by_id:
                raise ValueError(f"op[{i}]: layer id {new.id!r} already exists")
            idx = next((n for n, l in enumerate(draft.layers)
                        if l.id == op.get("after")), len(draft.layers) - 1)
            draft.layers.insert(idx + 1, new)
        else:
            raise ValueError(f"op[{i}]: unknown op {kind!r}")
        applied += 1

    try:
        Document.model_validate(draft.model_dump())        # whole-doc invariants
    except ValidationError as e:
        raise ValueError(f"Result would be invalid: {e.errors()[0]['msg']}")

    draft.version = doc.version + 1
    STORE.save(draft)                       # append-only: every version kept
    return {"ok": True, "ops_applied": applied, "version": draft.version}
Every raised message is written to be read by a model, and every one of them names the fix. “Layers are: headline, subhead, date_block” turns a dead end into a corrected retry on the next turn. All-or-nothing application means a half-applied op list can never leave a document in a state nobody designed.

Image generation through OpenRouter

pythontools/image_generate.py
import base64, hashlib, httpx

ASPECTS = {(420, 594): "3:4", (594, 420): "4:3", (1080, 1350): "4:5"}

async def image_generate(doc_id: str, layer_id: str, prompt: str,
                         style: str = "", seed: int | None = None) -> dict:
    """Generate artwork for one image layer and store it as an asset.

    The prompt should describe imagery only — never text, headlines or logos.
    Type is set by the layout engine; asking an image model for words is how
    posters end up with beautiful gibberish on them.
    """
    doc   = STORE.load(doc_id)
    layer = doc.layer(layer_id)
    ratio = nearest_aspect(layer.box.w_mm, layer.box.h_mm)

    r = await httpx.AsyncClient().post(
        f"{OR}/images", headers=HEADERS, timeout=120,
        json={
            "model": "google/gemini-2.5-flash-image",
            "prompt": f"{prompt}. {style}. No text, no lettering, no logos.",
            "aspect_ratio": ratio,
            "resolution": "2K",              # enough for 300dpi at this box size
            "output_format": "png",
            "n": 1,
            **({"seed": seed} if seed is not None else {}),
        })
    r.raise_for_status()
    body = r.json()

    img   = body["data"][0]
    raw   = base64.b64decode(img["b64_json"])          # base64, never a URL
    aid   = "img_" + hashlib.sha256(raw).hexdigest()[:16]
    w, h  = ASSETS.put(aid, raw, img.get("media_type", "image/png"))

    # check print viability NOW, not at export time
    dpi = min(w / (layer.box.w_mm / 25.4), h / (layer.box.h_mm / 25.4))

    return {                        # a handle and facts. never the bytes.
        "asset_id": aid, "width": w, "height": h,
        "effective_dpi": round(dpi),
        "print_ok": dpi >= 300,
        "cost_usd": body.get("usage", {}).get("cost"),
        "next": f"call doc.patch to set layer {layer_id!r} asset_id to {aid}",
    }
Two habits worth copying. The tool tells the model what to do next — generating an asset and wiring it into a layer are separate steps, and saying so prevents a dangling asset. And it computes effective DPI immediately, so a too-small image is caught while it is still cheap to regenerate rather than at export.

Render and check

pythontools/render.py  ·  one document, two renderers, same geometry
MM_PER_IN = 25.4

def to_svg(doc: Document, *, embed_assets=True) -> str:
    """The single source of truth for geometry. Preview and export both
    go through here, so what the agent judged is what gets printed."""
    W, H, bl = doc.canvas.width_mm, doc.canvas.height_mm, doc.canvas.bleed_mm
    out = [f'<svg xmlns="http://www.w3.org/2000/svg" '
           f'width="{W+2*bl}mm" height="{H+2*bl}mm" '
           f'viewBox="{-bl} {-bl} {W+2*bl} {H+2*bl}">']
    out.append(f'<rect x="{-bl}" y="{-bl}" width="{W+2*bl}" '
               f'height="{H+2*bl}" fill="{doc.palette[0]}"/>')
    for layer in doc.layers:                        # back to front
        out.append(render_layer(doc, layer, embed_assets))
    out.append("</svg>")
    return "".join(out)

def render_preview(doc_id: str, long_edge: int = 1024) -> list:
    """Rasterise to a downscaled PNG. Call after any layout change and
    before judging composition or claiming something looks right."""
    doc = STORE.load(doc_id)
    png = rasterise(to_svg(doc), long_edge=long_edge)   # resvg / cairosvg
    return [f"Rendered {doc_id} v{doc.version}.",
            ImageContent(data=png, mime="image/png")]   # ← the model sees this

def checks_run(doc_id: str) -> dict:
    """Deterministic pre-flight. Every finding names a layer and a fix."""
    doc, findings = STORE.load(doc_id), []
    pixels = rasterise(to_svg(doc), long_edge=1024)     # for real contrast

    for layer in doc.layers:
        if layer.kind == "text":
            behind = dominant_colour_under(pixels, doc, layer.box)
            ratio  = contrast(layer.colour, behind)
            pt     = doc.type_scale.base_pt * doc.type_scale.ratio ** layer.step
            need   = 3.0 if pt >= 18 else 4.5
            if ratio < need:
                findings.append({
                    "layer": layer.id, "check": "contrast", "severity": "error",
                    "detail": f"{ratio:.1f}:1 against {behind}, needs {need}:1",
                    "fix": "lighten the text, or add a scrim behind it"})
            w, h = measure_text(layer, doc.type_scale)
            if w > layer.box.w_mm or h > layer.box.h_mm:
                findings.append({
                    "layer": layer.id, "check": "overflow", "severity": "error",
                    "detail": f"text needs {w:.0f}x{h:.0f}mm, box is "
                              f"{layer.box.w_mm:.0f}x{layer.box.h_mm:.0f}mm",
                    "fix": "drop one type step, or widen the box"})
            if outside_safe_area(layer.box, doc.canvas):
                findings.append({"layer": layer.id, "check": "safe_area",
                    "severity": "error", "detail": "inside the trim margin",
                    "fix": f"keep text {doc.canvas.safe_mm}mm from every edge"})
        if layer.kind == "image":
            a = ASSETS.meta(layer.asset_id)
            dpi = min(a.w / (layer.box.w_mm / MM_PER_IN),
                      a.h / (layer.box.h_mm / MM_PER_IN))
            if dpi < 300:
                findings.append({"layer": layer.id, "check": "dpi",
                    "severity": "error", "detail": f"{dpi:.0f} dpi at this size",
                    "fix": "regenerate at higher resolution or shrink the box"})

    return {"version": doc.version, "errors":
            sum(f["severity"] == "error" for f in findings), "findings": findings}
Contrast is measured against the rendered pixels under the text, not against a declared background colour — because the thing behind a headline is usually a generated image, and that is exactly where legibility fails. This is the sort of check a model cannot do by looking and a computer does perfectly.

Wiring it together

pythonstudio.py  ·  the system prompt is an artifact, version it like code
SYSTEM = """You are a poster art director working in a structured design tool.

You never draw pixels. You edit a design document made of typed layers, and a
deterministic renderer turns it into a poster. Type is always set by the layout
engine — never ask an image model for words.

Method, every time:
1. doc.summary to see the current state.
2. Make one coherent change with doc.patch. Prefer few, decisive ops.
3. render.preview, then LOOK at it.
4. checks.run. Fix every error before doing anything else.
5. Repeat until checks are clean and the brief is satisfied. Then stop.

Rules:
- Type sizes are steps on the scale. Never invent a point size.
- Everything snaps to the grid and the baseline. Use layout.grid.
- Contrast failures and overflow are never acceptable, even if it looks fine.
- A locked layer belongs to the designer. Work around it.
- Say what you changed and why in one sentence. No commentary on your process.
"""

TOOLS = [                       # deterministic order: cache-friendly (§4)
    Tool("doc.summary",     S.doc_summary,   doc_summary,   timeout=5),
    Tool("doc.patch",       S.doc_patch,     doc_patch,     timeout=10),
    Tool("doc.undo",        S.doc_undo,      doc_undo,      timeout=5),
    Tool("type.pair",       S.type_pair,     type_pair,     timeout=5),
    Tool("palette.build",   S.palette_build, palette_build, timeout=5),
    Tool("layout.grid",     S.layout_grid,   layout_grid,   timeout=10),
    Tool("render.preview",  S.render_prev,   render_preview,timeout=20),
    Tool("checks.run",      S.checks_run,    checks_run,    timeout=20),
    Tool("image.generate",  S.image_gen,     image_generate,timeout=120,
         approval="ask"),
    Tool("render.export",   S.render_export, render_export, timeout=60,
         approval="ask"),
]

async def design(brief_text: str, on_event):
    brief = await parse_brief(brief_text)          # structured output, temp 0
    doc   = doc_create(preset=brief.format)

    agent = Agent(client, TOOLS, SYSTEM, on_event=on_event,
                  on_approve=ui_approval_prompt)

    result = await agent.run(
        goal=f"{brief.as_prompt()}\n\nWorking on document {doc['doc_id']}.",
        model="anthropic/claude-sonnet-4.5",       # the hot loop model
        budget=Budget(max_turns=28, max_cost_usd=3.00, max_seconds=420),
    )

    for _ in range(2):                             # bounded maker–critic (§6)
        notes = await critique(doc["doc_id"], brief) # vision model, other vendor
        if not notes.blocking:
            break
        result = await agent.run(goal=notes.as_instructions(),
                                 budget=Budget(max_turns=10, max_cost_usd=1.00))
    return result

The React canvas

The frontend renders the same document the agent edits. Because the document is plain JSON and the geometry lives in one to_svg-shaped function, the canvas is a pure render of state — no separate drawing code to drift out of sync with the print output.

javascriptCanvas.jsx
import { useEffect, useReducer } from "react";

// The agent streams { type:"patch", version, ops } events. We apply them
// locally so the canvas updates live, mid-turn, instead of after the run.
function reduce(doc, ev) {
  switch (ev.type) {
    case "doc":     return ev.doc;                       // full state on load
    case "patch":   return applyOps(doc, ev.ops, ev.version);
    case "locked":  return setLayer(doc, ev.layer, { locked: ev.locked });
    default:        return doc;
  }
}

export function Canvas({ docId, onSelect }) {
  const [doc, dispatch] = useReducer(reduce, null);

  useEffect(() => {
    const es = new EventSource(`/api/docs/${docId}/stream`);
    es.onmessage = (e) => dispatch(JSON.parse(e.data));
    return () => es.close();
  }, [docId]);

  if (!doc) return <div className="canvas-skeleton" />;
  const { width_mm: W, height_mm: H, bleed_mm: B, safe_mm: S } = doc.canvas;

  return (
    <svg viewBox={`${-B} ${-B} ${W + 2 * B} ${H + 2 * B}`} className="canvas">
      <rect x={-B} y={-B} width={W + 2 * B} height={H + 2 * B}
            fill={doc.palette[0]} />

      {doc.layers.map((l) => (
        <Layer key={l.id} layer={l} doc={doc}
               onClick={() => onSelect(l.id)}
               style={{ cursor: l.locked ? "not-allowed" : "pointer" }} />
      ))}

      {/* print furniture — drawn last, never exported */}
      <rect x={0} y={0} width={W} height={H}
            fill="none" stroke="#A62834" strokeWidth={0.3} strokeDasharray="2 2" />
      <rect x={S} y={S} width={W - 2 * S} height={H - 2 * S}
            fill="none" stroke="#12695B" strokeWidth={0.2} strokeDasharray="1 3" />
    </svg>
  );
}
Millimetres as the SVG user unit means the canvas coordinate system is the document coordinate system: no conversion layer, no rounding drift, and a bug in placement is visible in both the browser and the PDF in exactly the same place.
10

What frameworks add to the loop you just wrote

Having built §03 by hand, the frameworks become legible: each one is that loop plus a specific opinion. Knowing which opinion you are buying is the whole decision.

CategoryExamplesThe opinion you’re buyingWhat it costs
Provider agent SDKs OpenAI Agents SDK, Claude Agent SDK A well-tested loop, tool decorators, handoffs, sessions, built-in tracing. Fastest path from zero to working. Provider gravity. Several accept an OpenAI-compatible base_url, so they can point at OpenRouter — but the vendor’s own features are the happy path.
Graph / state machine LangGraph Your agent is an explicit graph with typed state, checkpointing at every node, human-in-the-loop interrupts, and time-travel replay. Deterministic control flow around stochastic nodes. Real conceptual overhead. Worth it when the flow genuinely branches; heavy when it is one loop.
Typed / schema-first Pydantic AI, Instructor Validated structured output, typed dependencies injected into tools, model-agnostic by design. Closest in spirit to §03 with the boilerplate removed. Little. This is often the right default for a Python service.
Role / crew abstractions CrewAI, AutoGen Multi-agent as a first-class concept: roles, delegation, conversation between agents. Encourages multi-agent before you have proven you need it, and hides the token cost of doing so. See §06.
Durable execution underrated Temporal, Restate, DBOS Your run is a workflow: it survives process death, retries individual steps with their own policies, and resumes exactly where it stopped. Nothing agent-specific — which is the point. Infrastructure. But a 6-minute poster run that dies at minute 5 is exactly the problem these were built for.
A defensible position

Write the loop yourself once — it is a day, and afterwards no framework is mysterious. Then adopt something for the parts that are genuinely hard infrastructure: durability, checkpointing and tracing. Keep in your own code the parts that are your product: context assembly, the tool definitions, the approval policy, and the evals. Those encode your domain, they are where your quality comes from, and no framework has an opinion about posters.

11

Build order, and the pitfalls

The order matters more than usual here, because the agent is the last thing you should build, not the first. Every week below produces something a person can use.

  1. The document, the renderer, the checks — with no model anywhere. A schema, an SVG renderer, a PDF exporter, and the deterministic lint suite. Hand-write three posters as JSON. If the renderer cannot produce something you would print, no agent will fix that. This is also, quietly, a shippable product on its own.
  2. One model call. Brief text in, a validated BriefSpec out, via structured output. Fill a hand-made template with it. Still no loop, still no tools. You now have a working generator and a feel for how the model reads briefs.
  3. The loop, with four tools: doc.summary, doc.patch, render.preview, checks.run. This is the moment it becomes an agent, and the moment tracing has to exist. Do not add a fifth tool until this loop reliably fixes its own check failures.
  4. Imagery. image.generate, the asset store, DPI validation, and the approval gate. This is the first tool that costs real money and takes real time, which is why it comes after the budget machinery works.
  5. The golden set and the eval harness. Thirty briefs, hard gates on the deterministic checks, cost and latency recorded per run. Wire it into CI before you start tuning prompts, or you will tune against vibes.
  6. Then, only if measurement says so: a bounded critic pass, parallel concepting, a router for cheap edits, and an MCP server if other applications want your renderer.
PitfallHow it presentsThe fix
The model emits the artifact directlyIt rewrites the whole document, or the whole SVG, every turnPatches against a stored document. §08.
Context grows with turn countTurn 20 costs 8× turn 3 and the output is worseOffload, project, compact. If your context is not roughly flat across a run, one of §04’s four moves is missing.
Cache silently disabledCosts 5–10× higher than expected, no obvious causeCheck for a timestamp or a random id in the system prompt, and for non-deterministic tool ordering
Tool sprawlTwenty-eight tools; the model picks the wrong one; every turn costs 6k tokens before you say anythingConsolidate to a dozen. Expose a task-scoped subset.
Exceptions instead of feedbackRuns die on the first bad argumentEvery tool failure returns a message that names the fix
No budgetA confused loop spends $40 in ten minutesTurns, tokens, cost and wall-clock, checked every turn
Asking the image model for textBeautiful posters with gibberish headlinesImagery from the image model, type from the layout engine, always. Put it in the system prompt and in the tool description.
Trusting tool outputA fetched page or an uploaded asset’s metadata steers the agentLeast privilege, approval gates on consequential tools, egress allowlists. Never one agent with both secrets and an outbound channel.
Multi-agent too earlyFive agents, 5× the cost, no measurable improvement, five times the debuggingOne loop until an eval says otherwise. §06.
Evaluating by using it“Feels better” after a prompt change; a regression shipsA golden set with hard gates on the deterministic checks. §07.
The through-line

Every good decision in this document is the same decision: push work out of the model and into deterministic code, and let the model do only the part that genuinely requires judgement. The renderer is deterministic. The checks are deterministic. The patch applier is deterministic. The grid maths, the contrast maths, the DPI maths, the type scale — all deterministic. What is left for the model is choosing a direction, choosing what to change, and looking at the result — which is exactly what it is good at, and exactly what your code cannot do.

12

Glossary

Agent
A loop that calls a model, executes any tools it requested, appends the results, and repeats until it stops asking or a budget trips.
Context assembly
Building the exact message list for this turn. The main tuning surface of an agent system.
Context rot
The observed decline in quality as an input grows, well before the hard window limit. Attention is finite and diluted.
Elicitation
An MCP server asking for human input mid-call. Since 2026-07-28 it is delivered through MRTR rather than a server-initiated request.
Golden set
A fixed, versioned collection of real inputs used to detect regressions. Assert properties, never exact outputs.
Handle
An opaque server-minted id returned by one tool and passed to the next. How stateful work happens now that MCP is stateless.
Harness
Everything around the loop: budgets, retries, timeouts, approvals, persistence, streaming, tracing, stop conditions.
MRTR
Multi Round-Trip Requests. A server returns resultType: "input_required"; the client gathers the inputs and retries the original request with inputResponses.
Projection
A compact, context-sized view of a large piece of state. The agent reads the projection; the state stays in the store.
Prompt caching
Provider-side reuse of a repeated input prefix, charged at a fraction of the normal rate. Requires a byte-identical prefix.
Prompt injection
Instructions smuggled into a model’s context through tool results or fetched content. Not solvable by prompting; mitigated structurally.
Streamable HTTP
MCP’s remote transport. The older HTTP+SSE transport is deprecated.
Structured output
Schema-constrained generation via response_format: json_schema. Distinct from MCP’s structuredContent, which is server-produced result data.
Tool
A JSON Schema shown to the model plus a function your code runs when the model asks. The model never executes anything itself.