Guardrails for LLM Outputs

9 min read

Learn

Quick Reading

Estimated 9 mins

Prereq

Intermediate

Basic ML concepts helpful

Interactive

Static Playbook

Static guide & reference tables

Why guardrails exist as a separate layer

An LLM's output is probabilistic and shaped by training, not guaranteed to satisfy any hard constraint — it can produce unsafe content, leak sensitive information, break a required output format, or confidently state something false. Guardrails are the system-level checks, filters, and structural constraints wrapped around a model's input and output to catch these failures before they reach a user or downstream system, treating the model itself as a component that can and will sometimes misbehave, rather than a fully trustworthy black box.

Input-side and output-side guardrails

Guardrails apply on both sides of a model call. Input guardrails screen the prompt before it reaches the model: detecting prompt injection attempts, PII in user input that shouldn't be forwarded, or requests that should be refused outright before incurring the cost of a model call. Output guardrails screen the model's response before it's returned: checking for policy violations (toxicity, disallowed topics), schema conformance (does the JSON match the expected structure), factual consistency against provided context, and leakage of system prompts or credentials.

Guardrail types and what they catch

Guardrail type	Applied to	Catches
Schema validation	Output	Malformed JSON, missing required fields, wrong types
Content moderation classifier	Input and output	Toxicity, harassment, disallowed categories
PII detection/redaction	Input and output	Leaked emails, SSNs, phone numbers
Prompt injection detection	Input	Attempts to override system instructions
Groundedness/faithfulness check	Output	Claims unsupported by retrieved context
Rate/cost limiting	Both	Runaway loops, abuse, excessive spend

output_guardrail_pipeline.py

python

import json

def validate_schema(output_text, schema):
    try:
        data = json.loads(output_text)
    except json.JSONDecodeError:
        return False, "not valid JSON"
    for field in schema["required"]:
        if field not in data:
            return False, f"missing required field: {field}"
    return True, data

def run_guarded(llm_call, prompt, schema, moderation_check, max_retries=2):
    for attempt in range(max_retries + 1):
        raw = llm_call(prompt)
        ok, result = validate_schema(raw, schema)
        if not ok:
            prompt = prompt + f"\n\nYour last output was invalid ({result}). Return valid JSON only."
            continue
        flagged, reason = moderation_check(raw)
        if flagged:
            return {"error": f"blocked by moderation: {reason}"}
        return result
    return {"error": "failed schema validation after retries"}

Guardrails as a second model call, not just regex

Simple guardrails use regex or keyword lists (fast, cheap, but brittle and easy to evade). More robust guardrails use a smaller, fast classifier model or a constrained LLM call dedicated purely to judging the primary model's output — e.g. 'does this response contain any claim not supported by the following context?' This LLM-as-guardrail pattern is more flexible and catches subtler failures, at the cost of added latency and a second model call's worth of expense, which is why many production systems layer cheap rule-based checks first and only escalate to a model-based check for ambiguous cases.

Structural guardrails: constrained decoding

Beyond post-hoc validation, some guardrails operate during generation itself. Constrained decoding (grammar-based sampling, JSON-schema-constrained generation) restricts which tokens the model is even allowed to sample at each step, so output is guaranteed to match a grammar or schema by construction rather than checked afterward. This eliminates an entire class of retries and failures for structured output tasks, at some cost in flexibility and occasionally quality, since the model is prevented from exploring token sequences outside the allowed grammar even when they might have been a better answer.

Guardrails are not a substitute for good prompting or fine-tuning

Teams sometimes treat guardrails as a catch-all fix for a poorly specified or poorly aligned model, retrying endlessly until output passes checks. This masks root causes, burns latency and cost on retries, and still lets subtle failures through if the guardrail itself has blind spots. Guardrails work best as a safety net around a model that's already reasonably well-prompted and well-suited to the task — not as the primary mechanism for achieving correctness.

What's Next?

Catching bad outputs after the fact only works if you can actually see what the model did and why — which is the role of observability and tracing infrastructure in a production LLM system.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →