Guardrails and Safety

Guardrails are the core landing of the Harness's constrain, verify, and correct layers — a single guardrail is unlikely to provide enough protection, but layering several specialized ones across the input, execution, and output sides builds a far more resilient agent system.

Guardrails are the core means by which the Harness’s “constrain, verify, and correct” layers are realized — they form the layered defense that keeps an agent’s behavior safe and controllable. Well-designed guardrails help manage data-privacy risk (preventing system-prompt leakage) and reputational risk (keeping behavior consistent with the brand). You can start with guardrails for the risks you have already identified, then add more as new vulnerabilities surface. Think of them as layered defense: a single guardrail is unlikely to provide enough protection, but combining several specialized ones builds a more resilient system.

Three Kinds of Guardrail, by Position

By where they sit, guardrails fall into three kinds — input side, execution side, and output side.

Input-side guardrails intercept before a request reaches the agent, usually via four mechanisms. A relevance classifier flags off-topic queries (a coding assistant asked “how tall is the Empire State Building?”). A safety classifier detects jailbreaks and prompt injection — the key difference being that a jailbreak is the user trying to bypass the model’s own safety limits, while prompt injection is an attacker manipulating the model indirectly through external data (web content, documents). Content moderation flags harmful or inappropriate input. Rule-based protection applies deterministic measures — blocklists, input length limits, regex filters — to guard against known threats like SQL injection.

Execution-side guardrails verify at the moment a tool is called. Their core is a tool risk rating: based on whether an operation is reversible, its permission level, and its financial impact, each tool is labeled low / medium / high risk, and high-risk operations require extra review or human confirmation.

Output-side guardrails check before a response returns to the user. A PII filter screens the output for personally identifiable information (ID numbers, phone numbers) to prevent unnecessary exposure; output validation checks that the reply is consistent with brand values.

Some mechanisms (rule-based regex filtering, for one) can sit on either the input or the output side; the grouping above is by most common placement. A representative industrial practice for classifier guardrails is Anthropic’s Constitutional Classifiers, whose core has three parts: rule-driven (a natural-language “constitution” generates synthetic training data to train input and output classifiers), joint-context judgment (checking the user’s question and the model’s answer together, because some answers look harmless alone and only reveal a code word against the question), and two-tier screening (a very lightweight probe checks all conversations, escalating anything suspicious to a stronger classifier — the first tier can afford false positives without hurting user experience, and cost drops sharply).

Human in the Loop

Human intervention (human in the loop) is a key protection that lets an agent improve real-world performance without hurting user experience. It matters most in early deployment, helping identify failure modes, surface edge cases, and build a robust evaluation cycle. Implementing it lets the agent hand off control gracefully when it cannot finish a task: in customer service, that means escalating to a human agent; for a coding agent, handing control back to the developer. Two situations typically trigger it:

  • Exceeding a failure threshold. Set an upper bound on retries or operations. If the agent exceeds it (repeatedly failing to understand the customer’s intent, say), escalate to a human.
  • High-risk operations. Sensitive, irreversible, or high-stakes operations should trigger human oversight, at least until the team has built enough confidence in the agent’s reliability. Typical examples include canceling an order or authorizing a large refund or payment.

Guardrails are the first line of a layered defense and cannot be foolproof. Their concrete implementations unfold across later chapters: prompt-injection defense in Prompt Engineering, tool permission control and tool risk rating in the tools chapter, and code-execution safety in the coding-agent chapter.

Was this page helpful?
Edit on GitHub