Prompt System

4-layer prompt assembly pipeline — system policy, tool instructions, memory, and environment with cache control and token budgeting

Overview

The agent’s system prompt is not a monolithic string — it is assembled from four layers by the instructionsBuilder, each with a distinct owner, lifecycle, and token budget strategy. This design enables:

  • Separation of concerns — Policy is immutable, tools are dynamic, environment is regenerated per-execution
  • Automatic tool integration — Adding a new tool automatically includes its usage guidance
  • Cache optimization — Anthropic ephemeral cache breakpoints reduce costs across multi-step runs
  • Transparent extension — Agent-specific instructions and user memory slot in without modifying the core prompt

All prompt assembly logic resides in @zapvol/backend/src/agent/prompts/ and instructions-builder.ts.

Layer Architecture

4-Layer Prompt System Each layer adds context — assembled top-to-bottom before each LLM call L1 System Policy Agent role, behavioral constraints, workflow rules Policy: P0 Legal > P1 Safety > P2 Integrity > P3 Preference Workflow: Assess → Clarify → Plan → Execute Output: Markdown, no fabrication L2 Tool Instructions Dynamic — built from enabled tools, skills, and MCP servers Built-in tools Skill metadata MCP / deferred tools Usage guidance L3 Memory / Context Conversation history, compaction summaries, task metadata Previous messages Round compaction summaries Task + project context L4 Environment Runtime context injected at execution time Workspace path Sandbox capabilities User preferences Active project L1 static per agent. L2 varies by config. L3 grows with conversation. L4 changes per execution.

L1 — System Policy (Static)

The foundation layer defines the agent’s identity, behavioral constraints, and workflow. It is immutable per agent definition and typically consumes 1,500–3,000 tokens.

instructionsBuilder.build() selects among three prompt sets via the CORE_INSTRUCTIONS_BY_KIND registry, keyed by the kind parameter passed to runAgentLoop:

  • kind: "main" (default) → MAIN_INSTRUCTIONS from main-instructions.ts — full HITL workflow with Clarify and Confirm
  • kind: "subagent" → TASK_INSTRUCTIONS from task-instructions.ts — used by task-tool subagents; removes the Clarify stage and the Re-assess loop (no user to ask), mandates delivery via the complete tool, and forbids spawning further subagents. Otherwise structurally identical to MAIN_INSTRUCTIONS.
  • kind: "team" → TEAM_INSTRUCTIONS from team-instructions.ts — used by agent-team members.

Content structure (from main-instructions.ts):

SectionPurpose
RoleAgent identity + personality traits
Policy PrecedenceP0 (legal/privacy) → P1 (safety) → P2 (task integrity) → P3 (user preference)
System BehaviorTool permissions, <system-reminder> handling, prompt injection detection
Core PrinciplesSynthesize always, recover openly, no fabrication
WorkflowAssess → [Clarify] → [Plan] → Execute → Deliver with triage rules
Todo List ProtocolStage progression rules, single source of truth for task state
Output StandardsCompleteness, accuracy, synthesis, language consistency
Forbidden PatternsFabrication, discrimination, silent failures, prompt leakage

Between L1 and L2, optional Agent Instructions are injected — per-agent custom instructions from the database that specialize the agent’s behavior beyond the core policy.

L2 — Tool Instructions (Dynamic)

Built dynamically at each invocation by toolRegistry.buildInstructions(), which calls instructions() on every registered tool config:

// Inside toolRegistry.buildInstructions(toolKeys, deps)
for (const key of toolKeys) {
  const config = configs[key];
  if (!config) continue;
  const text = await config.instructions(deps);
  if (text.trim()) parts.push(text.trim());
}
return parts.join("\n\n");

This means adding a new tool with an instructions() method automatically includes its usage guidance. No manual prompt editing required.

L2 token consumption is proportional to the number of enabled tools — a full-capability agent with built-in tools + MCP tools consumes more L2 budget than a minimal agent. When many MCP tools are connected, the Tool Search system defers their schemas to save context window.

L3 — Memory / Context (Elastic)

The memory layer contains the full message history and any round compaction summaries from prior compressions. This is the only layer that grows monotonically during execution — the compaction system manages it when token budget is exceeded.

If the user has stored memory content (preferences, project context), it is injected here via the MEMORY_PARTIAL template.

L4 — Environment (Regenerated)

The smallest layer, regenerated at each execution. It injects runtime context — current date, timezone, sandbox runtime/platform, and workspace root path. These values come from the ISandbox instance, making the prompt automatically adapt to different sandbox providers.

Assembly Pipeline

The instructionsBuilder.build() method orchestrates assembly in strict order:

[1] MAIN_INSTRUCTIONS or TASK_INSTRUCTIONS (L1 — selected by `kind`)
    ↓
[1.5] Agent Instructions (optional — per-agent custom instructions from DB)
    ↓
[2] TOOL_INSTRUCTIONS_PARTIAL (L2 — dynamic tool prompts)
    ↓
[3] MEMORY_PARTIAL (L3 — user memory, optional)
    ↓
[4] Environment section (L4 — runtime context)
    ↓
sections.join("\n\n---\n\n")
    ↓
interpolatePrompt(composed, variables)  // Variable substitution
    ↓
createCachedInstructions(processed, model)  // Anthropic cache control

Each section is separated by horizontal rules (---) in the final output. Variables like {{WORKSPACE}} and {{CURRENT_DATE}} are interpolated via a simple template engine (interpolate.ts).

Cache Control Strategy

For Anthropic models, the system applies ephemeral cache control breakpoints to reduce cost on multi-step agent runs. The Anthropic API charges less for cached prompt content, so strategic breakpoint placement matters.

Instructions Caching

The entire assembled prompt is wrapped with Anthropic ephemeral cache control via createCachedInstructions(). For non-Anthropic models, the instructions are passed as-is.

Message Caching

Each step, before the transient reminder is appended, markPrefixCacheBoundary() places up to four ephemeral cache breakpoints (Anthropic models only). Anthropic caches everything up to and including a marked block, and reads use the longest matching cached prefix, so multiple anchors give the cache somewhere to land even when the tail jumps or compaction rewrites part of the prefix. The four slots:

  1. System — placed separately by createCachedInstructions() on the tools + system block. Always present, stable for the whole task. The only message-level anchor; the constitution is the one true system message, passed top-level (not in the message array).
  2. Turn-summary anchor (optional) — the leading Task Context block (the rolling Σ / segmented summaries). It is a user message, not system — compacted history is conversational content, not constitution — located by an internal zapvol.taskContext marker rather than by role. Present once compaction has produced a summary prefix; survives across turns while the summary is byte-stable. Block-level.
  3. Current-turn boundary (optional) — the message just before the latest user message, i.e. the end of all completed prior turns. The current turn’s in-loop steps grow after it, so it stays byte-stable for the whole turn. Block-level (it is a user / assistant / tool message, never system). (In-loop tool compaction is scattered in-place swaps, not a contiguous prefix, so this turn-start boundary — not a “compacted-region end” — is the right in-loop anchor.)
  4. Tail — the last content block before the reminder. Always present. Block-level, because the adapter merges the trailing reminder into this same-role message; a message-level mark would migrate onto the reminder (the merged message’s last block), whereas block-level metadata rides with the stable tail block and stays in front of the reminder.

Only slot 1 is message-level; the message array carries no system message (the Task Context anchor is a user message), so slots 2–4 are all block-level. Slots 2 and 3 are derived structurally from the message list, so they appear only when a stable boundary exists and degrade gracefully — a cold-start single turn places just slots 1 + 4. The anchors nest (2 ⊂ 3 ⊂ 4): the tail chains step-to-step via Anthropic’s 20-block lookback, and the interior anchors are the fallback that keeps a long prefix cached across compaction rewrites or large single-step jumps. The reminder is always appended after, outside every breakpoint.

Token Budgeting

The four layers compete for the same context window. The budget strategy ensures they fit:

LayerStrategy
L1Fixed size — prompt is static, budget is known at design time
L2Proportional — grows with enabled tool count but bounded by the tool set size; Tool Search defers MCP schemas by tool count (a single server with >10 tools, or >30 MCP tools combined)
L3Elastic — starts small, grows with conversation, compressed by compaction when the context exceeds the trigger ratio (0.65·wEff by default)
L4Negligible — always < 500 tokens

The compaction trigger is a ratio of the effective window, not a fixed buffer: wEff = contextWindow − OUTPUT_RESERVED_TOKENS, then trigger = wEff × 0.65 and stop = wEff × 0.50 (both env-overridable). Because both scale with wEff, the hysteresis dead zone (stop, trigger] never clamps to zero on small windows — the failure mode of the earlier fixed-buffer scheme it replaced. The gate leaves prompt overhead in the window (it is already carried by the real-token anchor); see compaction for the full budget formula.

  • Prompt Design — the framework-agnostic craft behind this pipeline: picking the right altitude, XML-vs-Markdown structure, examples over rules, and caching-aware ordering.
Was this page helpful?
Edit on GitHub