Agent Engine

Every run remembers where it’s been

The agent formula and ReAct loop already covered it: the core of an agent is the “reason → act → observe” loop, repeated. The agent engine is where that loop actually turns — the model reasons about what to do, acts through a single tool call, observes the result, and returns to reasoning, step after step, until the task is done.

It behaves nothing like an ordinary chat completion. A chat completion is one-shot and stateless: you hand it a span of text, it hands one back, and once the exchange ends nothing is kept. A run of the engine spans many tool-call rounds and remembers the whole way — where it has got to, what it has tried, what each attempt returned; that steadily accumulating context is what lets it carry a long task coherently. But context only grows, and sooner or later it grows large enough to overflow the window, so the engine compacts it automatically as it goes. Meanwhile the intermediate results don’t wait for the whole run to finish — they stream to the client live, so the user watches the task move forward step by step.

The engine is responsible for the loop itself, and nothing more. The DB, the per-task lock, the credit ledger, and whether the transport is SSE, WebSocket, or IPC all live in the Task Orchestrator around it. And because it couples to none of them, the loop can be abstracted into a single runAgentLoop() (@zapvol/backend/src/agent/): the server’s web service and the desktop app each assemble their own params outside it and consume the stream it returns — both calling the same engine function.

Entry point: runAgentLoop

Now that the engine is a stateful ReAct loop, let’s see how one run gets going. The engine has a single entry point: runAgentLoop(params) — one task run (one turn) starts here. The caller — the server’s Task Orchestrator or the desktop agent handler — assembles the parameters first (message history, sandbox, MCP tools, …), hands them in, and consumes the result. The body is just four steps, and their order is locked by data dependency — they can’t be reordered:

1. Build inputs (buildAgentInputs) — first gather what this turn feeds the model: which model, the 4-layer system prompt, and the tool set available this turn. Instructions and tools depend on neither the other, so they build in parallel.
2. Wire compaction, render the turn-entry message (buildCompactionSetup) — to compact accurately you first have to know how much budget this turn actually uses, so this step measures first: it counts the tokens of the actual instructions + tools being sent, and builds the compaction engine from that. Then engine.buildTurnInput renders the stored history into this turn’s opening message — [user: Task Context] + raw window + tail — and seeds the cross-turn calibration anchor from the persisted meta:anchor, so this turn’s compaction lines up with the last. Finally it creates the per-step compactor.
3. Snapshot the budget (saveBudgetSnapshot) — record the budget just measured, for admin observability. It’s fire-and-forget (server-only, skipped on isolated paths), and whether it writes or not it never blocks the turn — observability yields to the real work.
4. Assemble and stream (new ToolLoopAgent + agent.stream) — everything ready, wire the prepareStep / onStepEnd / onToolExecutionEnd callbacks and the loop turns: LLM generates → tool executes → result appended → generation continues, until the model calls complete itself or hits the step limit. One thing to note: the moment runAgentLoop returns AgentLoopResult { agentStream, stepUsages }, the stream is not yet consumed — the engine just hands it over for the caller to drive; once consumed and the reference dropped, the underlying StreamTextResult can be GC’d.

What prepareStep does each step

The four-step assembly is a one-time opening; prepareStep is the part that re-runs on every step. It runs before each LLM call and does four things in a fixed order:

First it runs the in-loop step compaction (replay → gate → reduce → project), squeezing the context back within budget before anything is actually sent. Then it sets a prefix cache boundary (markPrefixCacheBoundary), so the stable prefix before it can be served from cache and save re-billing. Then it appends transient reminders outside that cache prefix — they change every step, and folding them into the cached region would dirty the cache. Finally it narrows the MCP tool surface actually exposed to the model this step via activeTools. As each step closes, onStepEnd accumulates one StepUsageData, recording what that step cost.

What crosses turns

The engine wipes itself clean once a run finishes; it keeps no end-of-turn state. So how does the next turn pick up? On two things kept on purpose.

One is the calibration anchor: it is written only after this turn’s messages are durable, by saveTurnAnchor into the snapshot store’s meta:anchor, and re-seeded at the next turn’s buildTurnInput. The other is the content-addressed snapshot store — per-segment summaries, per-part reduced forms — which is written during the loop and so likewise survives into the next turn.

AI SDK internals: prepareStep’s message reference semantics, two-layer onStepEnd / onEnd firing, stopWhen default fallback — for the counter-intuitive details, see AI SDK → Runtime Lifecycle and Message Reference Model.

The state machine

A run often takes several minutes, and you can’t leave the user staring at a blank screen. So the engine broadcasts its state as it goes, letting the client UI render progress live: state events go over the agent-state data part via context.writeTransient() (transient — a live broadcast only, never persisted into the message history).

The generating ⇄ executing cycle is the ReAct core — the LLM alternates between producing text and invoking tools until it signals completion or reaches the step limit. compacting surfaces only when a step actually crosses the compaction trigger, so the operator sees “compacting context” rather than mistaking it for a stall.

Note: The setup states (agent_building / context_building / mcp_connecting / agent_running) are emitted by the Task Orchestrator’s stream-setup phase, not by the engine itself. The full key set lives in AgentStateKey (@zapvol/common).

The subsystems the loop drives

The engine is the seam that wires the subsystems below into the ReAct cycle — inputs to the model (prompt, tools, MCP, skills), state management (memory, compaction), and output (streaming). Each has its own page; this table is the map.

Subsystem	Role in the loop	→
Prompt System	4-layer prompt assembly (policy → tools → memory → environment), built each turn	Prompt System
Tool Registry	Self-describing tool configs with capability filtering and compaction hooks	Tools
MCP Integration	Bridges runtime-discovered MCP tools into the tool set, with credential scoping	MCP
Skill Loading	view_skill(name, path?) loads SKILL.md / L3 resources on demand	Skill Loading
Memory System	Cross-session persistent memory — explicit save + background auto-extraction	Memory System
Context Compaction	Append-only raw stream + part-addressed checkpoint, reduced per step under token pressure	Compaction
Streaming	Data-part protocol + dual transport (SSE / WebSocket) + Redis-backed SSE resume	Streaming
Planning · Subagents	write_todos planning; task spawning isolated subagents for delegation	Planning · Subagents

Orchestrator-level, not engine: the Background Job Queue (post-stream billing, summaries, memory extraction) is driven by the Task Orchestrator, not the loop.

Related reading

• Task Orchestration — the server-only host that runs the engine over HTTP: locks, credits, persistence, transport.
• Overview — the agent and the shape of one task’s run, one level up.
• AI SDK → Runtime Lifecycle · Message Reference Model — the ToolLoopAgent / prepareStep mechanics the loop rides on.