Perception, Execution, Collaboration

Perception Tools

Perception tools are the primary channel for Agents to obtain external information.

Designing an excellent perception tool system requires careful trade-offs across multiple dimensions, including granularity, organization, and output format.

Perception tools often face the challenge of returning far more information than the Agent can process: a single search might return tens of thousands of characters, a PDF might be hundreds of pages long. Dumping everything into the context fills the context window and drowns key content in noise. The general response is to integrate context-aware compression (introduced in Chapter 2) at the tool level—when the output exceeds a threshold (e.g., 10,000 characters), automatically compress it based on the Agent’s current query intent (the principle and compression effectiveness are detailed in Chapter 2 and not repeated here). Beyond this general mechanism, several common types of perception tools have their own unique design issues.

Return format and pagination for search tools. The return value of a search tool should be a structured list of candidates (title, location, summary snippet), not a concatenation of full text—let the Agent browse candidates first, then decide which one to read in depth. When there are many results, provide pagination or cursor parameters: return only the first few by default, and note the total number of results and how to get the next page in the return value, letting the Agent decide whether to continue paging, rather than dumping all results at once.

Offset/limit and truncation strategy for read tools. Read tools should support offset/limit parameters to read specific segments of large files on demand. When content must be truncated because it exceeds a threshold, the truncation should be explicitly visible: note how much content was omitted and how to read the rest (e.g., “Displayed lines 1-200 of 5000; use the offset parameter to continue reading”). Silent truncation is dangerous—the Agent mistakenly believes it has seen everything and makes incorrect judgments based on incomplete information.

Engineering benefits of read-only nature. Perception tools do not change the external world. This read-only characteristic brings two natural advantages: results can be safely cached (identical queries reuse results, saving time and cost), and multiple perception calls can be safely executed in parallel (e.g., reading five files simultaneously, launching three searches concurrently) without worrying about interference. Execution tools do not have this freedom—call order and side effects must be strictly controlled.

Output form for multimodal perception. For multimodal inputs like screenshots, charts, or scanned documents, the tool needs to decide what form to present to the model: return the image directly to a model with vision capabilities, or first convert it to text using OCR, chart parsing, etc.? The former preserves layout and visual details but consumes more tokens; the latter is concise and efficient but may lose critical spatial structure (e.g., row-column relationships in a table). In practice, the choice is often based on content type: pure text content uses text extraction; layout-sensitive content (UI interfaces, complex tables, design drafts) retains the image.

Execution Tools

If perception tools are the Agent’s “senses,” execution tools are its “hands and feet.” But unlike perception tools, execution tools can fail expensively: a file deleted by mistake is gone for good, a bad system command can take down a service, an ill-judged API call can cost real money. Their design must therefore strike a delicate balance between capability openness and security constraints.

Hierarchical Design of Security Mechanisms.

The security of execution tools should not rely on a single mechanism but should be built as a multi-layered defense system.

The first layer is input validation — before executing any operation, check the validity of all parameters: whether file paths contain path traversal attacks (e.g., ../../etc/passwd — attackers use ../ in the path to make the tool escape the designated directory and access system files it shouldn’t), whether command parameters have injection risks (e.g., using semicolons or pipe characters to append additional commands), and whether the data types and formats of API parameters are correct. The key is to fail fast — immediately reject anomalous inputs without attempting “smart” corrections.

Above this is permission control. File operations are restricted to accessing only specific working directories; command execution maintains a blacklist of prohibited commands (e.g., rm -rf /, dd if=/dev/zero); external APIs check quotas and rate limits. Different deployment scenarios can customize permission policies through configuration files. Note that blacklists are only the most basic layer of defense and should not be the sole safeguard — attackers can bypass simple string matching with obfuscated commands. A more robust approach combines semantic parsing to understand the actual intent of a command rather than just matching its surface form. Chapter 5 will discuss this direction in detail.

Proposer-Reviewer: Security Review by an Independent Model.

Beyond input validation and permission control, irreversible critical operations call for a smarter layer of review. Applied to security, the Proposer-Reviewer paradigm introduced in the Introduction—an independent reviewer examining the proposer’s output—takes two typical forms: pre-approval and post-validation.

The first mechanism is pre-approval: before a tool is executed, one model is responsible for proposing the action (Proposer), and another independent model is responsible for reviewing and approving it (Reviewer) — similar to the dual-signature system in banking where a transfer instruction requires two signatures to take effect.

An efficient implementation hinges on three points. First, model selection: the proposing and approving models should come from different families (e.g., the GPT series and the Claude Sonnet series) but sit at a similar capability level. Different origins bring cognitive diversity—like having two engineers trained at different schools review the same plan: their backgrounds and habits of mind differ, so they are unlikely to make the same mistake in the same place. Two models from the same family (say, both GPTs) share training data and preferences, and tend to fail in the same scenarios. Similar capability, meanwhile, ensures the approver can follow the proposer’s reasoning; too wide a gap (Haiku reviewing Opus’s output) makes review unreliable—the reviewer cannot keep up. The ideal pairing is two models of similar capability but different training preferences, such as Claude Opus and GPT-5 reviewing each other.

In prompt design, the underlying rules and constraints for both models must be completely consistent (otherwise, they will argue and deadlock), but their focus should differ — the proposing model emphasizes action orientation and task completion, while the approving model emphasizes risk control and rule adherence.

After a rejection, the system should not simply retry. Instead, the rejection reason should be added to the Agent’s trajectory as a tool call result. From the proposing model’s perspective, a rejection by the approver is like a failed tool call that returns an error message and correction suggestions — the Agent already has the capability to handle tool failures, and the review mechanism is just a new input source.

Pre-approval essentially introduces an independent review perspective into the decision-making chain to reduce the error rate of a single model’s decisions. In practice, various optimizations can be applied: risk-graded approval (high-risk operations always require approval, low-risk ones are executed directly), human-supervised approval escalation (when the approving model is uncertain, it escalates to a human). Any irreversible, high-impact operation can benefit from pre-approval: charging fees, sending notifications and emails, modifying critical configurations, creating external resources, etc. Their common characteristic is that the consequences of the operation are persistent and the cost of error is high, making it worthwhile to invest additional computational resources for review.

The second mechanism is post-validation: after the operation is completed, a review perspective checks the correctness of the result. The key to post-validation is modality switching — not simply having a second model re-read the same content and review it again, but checking the result in a different modality. For example, after an Agent generates a document represented as code, it renders it as visual output to check if the layout is correct; after an Agent modifies a configuration file, it actually runs it in a sandbox to verify whether the configuration takes effect. Different modalities provide complementary verification perspectives, and single-modality review is prone to falling into the same blind spots. Chapter 5 will demonstrate further applications of the Proposer-Reviewer paradigm in content quality iteration (Proposer generates presentation code, Reviewer checks the rendered screenshot).

Sidecar Mechanism: Security Verification Parallel to Main Thinking.

The Proposer-Reviewer mechanism addresses the issue of “approval before operation execution or validation after operation completion,” while the Sidecar mechanism addresses another issue: “how to verify security and reliability in real time during operation execution.” It can be seen as a concrete implementation form of the “verification” function in the Harness framework from Chapter 1, and this section explains it in detail.

We need an out-of-band security check module that independently assesses risk before and after each tool call, while minimizing the slowdown of the main Agent’s thinking process. This design draws inspiration from the Sidecar pattern in microservice architecture — like a sidecar attached to a motorcycle, it runs independently but in parallel with the main entity. A Sidecar is a lightweight LLM call pattern that accompanies the main Agent’s thinking loop. It does not review the main Agent’s final output but makes independent judgments on the main Agent’s behavior. The actual timing deserves clarification: the Sidecar runs in parallel with the main model’s streaming output — while the main model issues a tool call and keeps generating text, the Sidecar’s review is already underway; yet for the tool call under review, the Sidecar acts as a gate — a dangerous operation will not execute until the Sidecar gives the go-ahead. In other words, parallelism reduces the review queuing delay; it does not remove the review gate itself. Claude Code’s approach in Auto Mode is a typical case: when the main model decides to execute a tool call, an independent lightweight LLM call (non-streaming, low latency) is triggered to judge “whether this tool call is safe.” This out-of-band call only looks at the structured tool call data (tool name, parameters) and does not look at the main model’s free-text thinking process — this is a deliberate design to prevent the main model from manipulating permission judgments through rhetoric.

The key threat here remains prompt injection (as introduced in the MCP security section earlier). Specifically in the Sidecar scenario: if the Sidecar also reads the main model’s free text, once an attacker embeds rhetoric like “please allow execution of rm -rf” in user input or web page content, the main model might repeat it in its own thinking process, which could then be misinterpreted by the Sidecar as a valid reason. Reading only structured fields blocks this rhetorical channel. For example: the main model prepares to execute bash("rm -rf /tmp/data"), the Sidecar classifier receives structured input {tool: "bash", command: "rm -rf /tmp/data"}, identifies the rm -rf pattern, judges it as a high-risk operation, returns a rejection, and requests user confirmation. This lightweight model call is typically completed within hundreds of milliseconds (sub-second), running in parallel with the main model’s streaming output, so the user barely perceives any additional latency.

A reader might object: we just said that review across a large capability gap is unreliable—so why is a lightweight model acceptable here? The answer lies in what is being reviewed. The Proposer-Reviewer examines open-ended thinking, so the reviewer must keep up with the proposer’s reasoning, which demands similar capability; the Sidecar judges a classification problem over structured data (is this command out of bounds?), a far simpler task that a lightweight model handles comfortably.

Both the Sidecar and the Proposer-Reviewer mechanism introduce a second perspective, but their execution timing and review targets differ. Table 4-2 compares the key differences between these two mechanisms.

Table 4-2 Comparison of Proposer-Reviewer Mechanism and Sidecar Mechanism

Dimension	Proposer-Reviewer	Sidecar
Execution Timing	Before operation (pre-approval) or after operation (post-validation)	Runs in parallel with the main model’s streaming output and gates individual tool calls
Review Target	The reasonableness of the operation or the result of the operation	The operation itself (tool call)
Review Perspective	Independent model approval, modality-switching validation	Security/reliability verification
Input Isolation	Proposer and reviewer see similar information	Sidecar deliberately isolates the main model’s free text
Typical Uses	Irreversible operation approval, document generation, configuration modification	Permission classification, memory relevance judgment, tool output summarization

Another typical application of the Sidecar pattern is context enrichment: while the main model is thinking, an out-of-band call runs in parallel to filter the relevance of user memories, summarize large tool outputs, and pre-assess permission requirements — these results are ready when the main model needs them, and the user perceives no additional latency.

A security Sidecar also needs a rejection circuit breaker: when the classifier rejects operation after operation, the system should not retry indefinitely—that wastes resources and can trap the user in a loop—but fall back to asking the user to judge manually. This is a typical instance of the Harness “correction” function from Chapter 1.

Automated Validation and Feedback Loop.

Another important design principle for execution tools is: if the result of an operation can be verified, it should be verified automatically. Taking code writing as an example: when an Agent calls write_file to create or modify a code file, the tool should not just write the content and return “success.” Instead, it should immediately perform a syntax check after writing: call the appropriate linter (a static code analysis tool) based on the file type, parse its output into a structured list of errors, and return this as part of the tool’s return value to the Agent.

This creates an “execute-validate-feedback” loop. If the code has syntax errors, the Agent will see specific error messages in the next thinking round (e.g., “Line 10: undefined variable result”), allowing it to make immediate corrections.

Truncation and Persistence of Long Outputs.

Execution tools often produce complex, lengthy outputs. When the output is detected to exceed a threshold (e.g., 200 lines or 10,000 characters), the tool only returns the first and last few lines to the context, while saving the complete result to a temporary file:

• Head retention: The first 50 lines, usually containing initial output or error context
• Tail retention: The last 50 lines, usually containing the final error message or success indicator
• Omission notice: e.g., “... [8523 lines omitted, full output saved to /tmp/execution_output.txt]...”
• File guidance: “To view the full output, use the read_file tool to read this file”

Isolation and Sandboxing of Execution Environments.

General-purpose execution tools (e.g., Python interpreter, Shell terminal) essentially allow the Agent to execute arbitrary code and require special security considerations. The ideal implementation is to run them in a sandboxed environment, isolated from the host machine — like conducting a chemistry experiment in a sealed laboratory; even if an accident occurs, it won’t affect the outside. A common misconception needs clarification here: a Python virtual environment (venv) is not a sandbox — it only isolates package dependencies and has no security constraints on the file system, network, or processes. Code running in a venv can still delete arbitrary files and access any network. True isolation relies on the operating system and lower-level mechanisms, arranged in order of increasing isolation strength:

• OS-level isolation: Uses the operating system’s security mechanisms to constrain process behavior, such as macOS’s Seatbelt (sandbox-exec), Linux’s seccomp and namespaces. It can restrict file access scope, disable networking, and block dangerous system calls. This is the preferred lightweight local solution.
• Container isolation: Docker and other containers provide an independent file system view and network stack, offering more complete isolation, but they share the kernel with the host machine. Kernel vulnerabilities could still be exploited for escape.
• microVM/Virtual Machine: Firecracker and other microVMs provide hardware-level isolation with an independent kernel. This is the strongest level for running completely untrusted code.
• Resource Quotas: At any isolation level, limits on CPU, memory, disk, and network usage should be set to prevent malicious or runaway code from consuming all resources.

The isolation level should be chosen based on the deployment environment and security requirements — OS-level mechanisms are sufficient for local development, while production environments or scenarios handling untrusted input require container or even microVM-level isolation.

Observability of Tool Execution.

Execution tools also require observability (the ability to infer a system’s internal state from its external outputs) — for monitoring, auditing, and debugging the Agent’s execution behavior. Good execution tools should provide: detailed logs (time, parameters, results, duration of each call), audit trails (who performed what operation in what context and why), performance metrics (call frequency, success rate, average duration), and alerting mechanisms (notify administrators of frequent failures, timeouts, resource overruns).

Idempotency and Cancellation Semantics.

Execution tools change the external world, so they must answer a question that perception tools don’t need to consider: when a call is cancelled or times out, did its side effects actually happen or not? A transfer call that returns an error after a network timeout might have already transferred the money, or it might not have — if the Agent retries without checking, it could duplicate the transfer. This problem is particularly prominent in asynchronous architectures, where interruptions and timeouts are common.

The core approach to handling this is idempotency: executing the same operation once and executing it multiple times has exactly the same effect on the external world, allowing safe retries. There are two common design methods: first, have the operation carry a unique identifier (e.g., a client-generated idempotency key), which the server uses for deduplication, returning the first result for duplicate requests instead of executing again; second, query before mutation — before retrying, query the current state of the target resource (whether the order has been created, whether the file has been written), and only execute if the operation has not already completed. Operations with idempotency make handling timeouts and interruptions much simpler.

But not all operations can be made idempotent. Operations like sending an email, making a phone call, or transferring money each produce an irreversible real-world event every time they are executed. Furthermore, the server is often outside your control, making it impossible to deduplicate using a unique identifier. For such non-idempotent operations, a “pre-check then confirm” two-phase approach should be used: the first phase only performs validation and a dry run (checking the balance, confirming the recipient, generating the content to be sent), returning the result along with a confirmation token; the second phase uses the token to actually execute, and if execution fails, it should not retry blindly in the same phase, but should hand control back to the upper layer to repeat the pre-check. This is of a piece with the Proposer-Reviewer pre-approval discussed earlier, and with the “initiate/complete” decoupling of asynchronous tool interfaces discussed later.

Collaboration Tools

When a task exceeds the capability boundary of a single Agent, collaboration tools allow it to delegate subtasks to other Agents or humans, then integrate the results from all parties.

Design Philosophy of Sub-Agents.

The core value of sub-agents lies in specialization through division of labor—rather than building one do-everything Agent, build a group of specialists that solve problems by collaborating. Each sub-agent can optimize its prompt, toolset, and knowledge base independently, without worrying about conflicts with the others.

Key Elements of Sub-Agent Prompts.

Role definition must be clear. State upfront, “You are an assistant Agent specifically responsible for XXX.”

Context sources must be clearly labeled. A sub-agent may receive information from multiple sources. The prompt should clearly distinguish each source: “[FROM_MAIN_AGENT] is the task instruction from the main coordinating agent; [FROM_USER] is information provided directly by the user; [TOOL_RESULT] is the result returned after you call a tool.” This labeling prevents the sub-agent from confusing information sources and avoids prompt injection attacks (introduced in the Sidecar section earlier).

Task boundaries must be clearly defined. Define what falls within the scope of responsibility and what needs to be handed off or escalated.

Output format must be standardized. A uniform JSON structure reduces the parsing burden on the main Agent and makes error handling more reliable.

Collaboration Mechanisms Between Agents.

The interfaces of collaboration tools can be distilled into three groups of primitives. First, spawning and canceling: spawn_subagent creates a sub-agent and assigns it a task; cancel_subagent terminates it promptly once the task has lost its purpose (the user changed their mind, another sub-agent already found the answer), avoiding further token waste. Second, message passing: send_message_to_subagent sends supplementary instructions or follow-up questions to a sub-agent while it is running, and the sub-agent can send messages back to the main Agent to report progress or request clarification. Third, discovery: in a system running multiple Agents at once, list_agents enumerates the currently available Agents along with their responsibility descriptions and running status, letting an Agent find potential collaborators—the same idea as MCP using tools/list to enumerate available tools, except what is enumerated here are Agents.

Built on top of these primitives, various collaboration modes can be supported: Synchronous Call (wait for the sub-agent to return, suitable for quick tasks), Asynchronous Call (receive a task ID immediately and an event notification upon completion), Streaming Collaboration (the sub-agent continuously sends incremental messages, suitable for scenarios where the process itself is valuable), and Multi-turn Interaction (a conversational collaboration where the sub-agent proactively asks questions and the main Agent responds). This chapter focuses on the shared tool interfaces for these modes; what context to pass when calling a sub-agent, which collaboration mode to choose, and how to organize the topology and division of labor among multiple Agents fall under the scope of multi-agent collaboration architecture, detailed in Chapter 10.

The Art of Human Intervention.

Although AI Agents are becoming increasingly powerful, human intervention remains necessary at certain critical decision points—some judgments inherently require human values, common sense, or domain expertise.

Timeout and Fallback Strategies. An HITL (Human-In-The-Loop—inserting a human review step into the Agent’s decision flow) request may not get an immediate response, so set timeout thresholds and default behaviors: “If no response within 5 minutes, adopt the conservative strategy.” Priority queues help too: urgent requests notify across multiple channels; routine requests get an email.

Establishing a Feedback Loop. HITL should not be a one-off interaction but should form a learning loop. Human approvals, rejections, and their reasons first constitute evidence-backed feedback data: generalizable principles of judgment can be incorporated into experiential knowledge or a Skill, while high-dimensional and implicit preferences can form post-training data. Chapter 8 discusses how to evaluate such trajectories and select an update carrier. Whichever method is used, a single human judgment must not be generalized directly into a universal rule without prior synthesis.

Engineering Practice

The three kinds share one ServerToolConfig contract in Zapvol, each kind’s design focus landing on a different hook:

• Perception tools’ “control the return volume” lands on the compact hook — grep / read tools shrink deterministically (top-N slice, keep head and tail, mark the elided middle); context-aware compression of large output is compaction. The knowledge base’s grep_docs / read_section are exactly paged-cursor perception tools.
• Execution tools’ isolation lands on ISandbox — constructed through the createSandbox() factory dispatching on SANDBOX_TYPE (only NodeSandbox is implemented today; Daytona / E2B are config + env placeholders). Credentials are isolated behind the KeyEncryption port and never enter the sandbox; HITL confirm / ask results are normalized into one user message by the normalize hook.
• Collaboration tools land on sub-agents and Agent Team.

Related reading

• Tool Design — the five-kind classification and universal design principles
• Event-Driven Async — the user-communication and event-trigger kinds