Guardrails

Guardrails are project-scoped policies that sit between your application and the upstream LLM provider. They evaluate every request (input) and/or every response (output) and can block, redact, or silently log content that matches your rules — without any changes to your API calls.

Your App → [input rules] → LLM Provider → [output rules] → Your App

Why Guardrails?

Compliance — prevent sensitive data (PII, card numbers, secrets) from leaving your system.
Content moderation — reject harmful or off-topic prompts before they cost you tokens.
Output enforcement — ensure model responses match a required JSON schema.
Observability — record every rule hit for auditing and debugging without blocking traffic.

Creating a Rule

Rules are created in the Guardrails section of your project dashboard or via the management API. Each rule requires:

Field	Description
`title`	Human-readable label
`direction`	`input` (check the request) or `output` (check the response)
`mode`	Matching strategy — see Matching Modes
`action`	What to do on a match — see Actions
`config`	Mode-specific configuration object

INFO

Rules are cached and applied in under 1 ms on the hot path. When a project has no enabled rules the evaluator is skipped entirely.

Matching Modes

`regexp` — Regular Expression

Matches when the payload contains a substring matching a RE2 regular expression.

{
  "mode": "regexp",
  "config": {
    "pattern": "\\b\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}\\b"
  }
}

The pattern is validated at rule-creation time — an invalid regex returns 400 GUARDRAIL_RULE_INVALID.

`string_presence` — Keyword List

Matches when the payload contains any of the listed literal terms (case-sensitive).

{
  "mode": "string_presence",
  "config": {
    "terms": ["confidential", "internal only", "do not share"]
  }
}

`json_validator` — JSON Schema

Fires when the model's response is valid JSON but does not conform to the provided JSON Schema. Use this on direction: output to enforce structured-output shapes.

{
  "mode": "json_validator",
  "direction": "output",
  "config": {
    "schema": {
      "type": "object",
      "required": ["customer_id", "status"],
      "properties": {
        "customer_id": { "type": "string" },
        "status": { "type": "string", "enum": ["active", "inactive"] }
      }
    }
  }
}

WARNING

json_validator is silently skipped on streaming endpoints. Use it only with non-streaming requests.

`llm_evaluation` — AI Judge

Sends the payload to a configured judge model. The judge's response is checked for a denial keyword to determine pass/fail.

{
  "mode": "llm_evaluation",
  "config": {
    "judge_model": "gpt-4o-mini",
    "prompt": "Does the following message discuss competitor pricing? Answer YES or NO only.",
    "deny_response_substring": "YES"
  }
}

INFO

Judge requests carry an internal bypass marker so they are never themselves evaluated by guardrails, preventing infinite recursion.

WARNING

llm_evaluation is silently skipped on streaming endpoints.

Actions

When a rule matches, one of three actions is applied. Rules are evaluated in three ordered passes — reject first, then mask, then log-only:

`reject`

The request is immediately blocked. On direction: input the upstream LLM is never called. On direction: output the upstream response is discarded. The client receives:

{
  "error": {
    "code": "GUARDRAIL_REJECTED",
    "message": "Request blocked by guardrail rule",
    "rule_id": "b3d1e2a0-...",
    "rule_title": "Block credit card numbers",
    "direction": "input"
  }
}

`mask`

Matching substrings are replaced with mask_template before the request is forwarded (or the response is returned). The call proceeds normally with the rewritten payload.

Only regexp and string_presence modes support masking.

{
  "action": "mask",
  "mask_template": "[REDACTED]"
}

`log_only`

The match is recorded as an event but the payload is never modified and the request is never blocked. This is the recommended mode for dry-running a new rule before enforcing it.

Precedence when multiple rules match

Active actions	User sees	Events
`reject` only	403 blocked	One per matched reject rule
`mask` only	Normal response (rewritten)	One per matched mask rule
`log_only` only	Normal response (unchanged)	One per matched log_only rule
`reject` + others	403 blocked	All matched rules

Optional Rule Settings

Field	Default	Description
`enabled`	`true`	Toggle a rule on/off without deleting it
`on_failure`	`reject`	What to do when evaluation itself fails (e.g. judge timeout). `allow` lets the request through; `reject` blocks it.
`event_sample_rate`	`1.0`	Fraction of `mask`/`log_only` hits to persist as events (0.0–1.0). Reject hits are always stored.
`event_redaction`	`partial`	How much matched text to store: `full` (none), `partial` (truncated preview), `none` (raw — requires PII acknowledgement).
`route_scope`	`[]`	Restrict to specific routes. Empty = all routes. Known values: `chat_completion`, `chat_completion_stream`.

Event Sampling

High-traffic log_only rules can produce a large number of event rows. Use event_sample_rate to cap writes while still capturing trend data:

{
  "action": "log_only",
  "event_sample_rate": 0.1
}

This stores ~10 % of hits. reject events are always stored regardless of this setting.

Streaming Behaviour

Mode	Works on streaming?
`regexp`	✅ Evaluated on a sliding window over accumulated chunks
`string_presence`	✅ Evaluated on a sliding window over accumulated chunks
`json_validator`	❌ Silently skipped
`llm_evaluation`	❌ Silently skipped

For reject on streaming: the stream is terminated early and the client receives a GUARDRAIL_REJECTED error event. Chunks already flushed to the client are not recalled — handle this error event in your UI to indicate that the response was truncated.

Failure Modes

When evaluation itself fails (Redis unavailable, judge timeout, etc.) the on_failure field controls the outcome:

`on_failure`	Behaviour
`reject` (default)	Block the request — favours compliance over availability
`allow`	Let the request through — favours availability over enforcement

Use on_failure: allow for llm_evaluation rules where a judge timeout should not interrupt production traffic.

Audit Log

Every change to a guardrail rule (create, update, enable, disable, delete) is recorded as an immutable audit entry. Entries include the full before/after rule snapshot so you can diff exactly what changed and who made the change.

Access audit history from Guardrails → Audit Log in your project dashboard.

Examples

Block credit card numbers on input

const rule = await fetch("https://api.llmapi.ai/projects/{projectId}/guardrails", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LLM_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    title: "Block credit card numbers",
    direction: "input",
    mode: "regexp",
    action: "reject",
    config: {
      pattern: "\\b\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}[\\s-]?\\d{4}\\b",
    },
  }),
});

Mask internal keywords before forwarding

const rule = await fetch("https://api.llmapi.ai/projects/{projectId}/guardrails", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LLM_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    title: "Mask internal project names",
    direction: "input",
    mode: "string_presence",
    action: "mask",
    mask_template: "[REDACTED]",
    config: {
      terms: ["Project Atlas", "Operation Sunrise"],
    },
  }),
});

Dry-run before enforcing

Start with log_only, review events in the dashboard, then switch to reject:

// Step 1 — observe
await fetch("https://api.llmapi.ai/projects/{projectId}/guardrails", {
  method: "POST",
  body: JSON.stringify({
    title: "Detect competitor mentions",
    direction: "output",
    mode: "string_presence",
    action: "log_only",
    config: { terms: ["CompetitorX", "CompetitorY"] },
  }),
});

// Step 2 — enforce (after reviewing events)
await fetch("https://api.llmapi.ai/projects/{projectId}/guardrails/{ruleId}", {
  method: "PATCH",
  body: JSON.stringify({ action: "reject" }),
});

Handle a rejected request in your app

const response = await fetch("https://api.llmapi.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LLM_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "gpt-4o",
    messages: [{ role: "user", content: userMessage }],
  }),
});

if (response.status === 403) {
  const error = await response.json();
  if (error.error?.code === "GUARDRAIL_REJECTED") {
    // Show a user-friendly message instead of crashing
    console.warn("Blocked by rule:", error.error.rule_title);
    return { blocked: true, reason: error.error.rule_title };
  }
}