Guardrails, permissions and human oversight

Production AI systems fail in rare, costly ways; layered guardrails turn those failures into blocked actions instead of incidents.

Guardrails are checks around the model that do not depend on the model behaving. No single layer is enough; combine several so each catches what others miss.

Input side:

Validate and classify input: block or flag jailbreak attempts (messages crafted to talk the model out of its rules), off-topic requests, or personal data you should not process. A small, fast model or a dedicated moderation service can do this before the main call.
Redact sensitive data (card numbers, IDs) before it reaches the model or logs.

Output side:

Validate structure with a schema; reject and retry on failure.
Check content: policy filters, "does every claim cite a source", links pointing only to allowed domains, numbers matching the data.
Fall back safely: on failure, return "I can't help with that" or hand over to a person rather than a best guess.

Actions (where the real risk is):

Scoped permissions: separate credentials per agent, read-only by default, access only to the needed folders, tables or accounts. Enforce this in the tool code or service, not just in the prompt.
Allow-lists of tools, commands and destinations rather than block-lists.
Approval tiers: let reading run freely, ask a human before writing, and always ask before anything irreversible or external (payments, deletes, sending to customers). Show the approver exactly what will happen.
Budgets and limits: maximum steps, spend, time and rate per task and per user.
Sandboxes: run code-executing agents in isolated environments with no access to production secrets.

Oversight: log every prompt, tool call and decision; review samples regularly; have an off switch; and feed incidents back into your evals. Most automation platforms and agent SDKs provide approval steps and guardrail hooks; use them rather than relying on instructions alone.

Example

POLICY = {
    "read_file":   {"allowed": True,  "approval": False},
    "write_file":  {"allowed": True,  "approval": True},
    "send_email":  {"allowed": True,  "approval": True},
    "delete_file": {"allowed": False, "approval": True},
}
ALLOWED_DIR = "/data/sales/"
MAX_STEPS = 20

def guard(tool: str, args: dict, step: int, approve) -> bool:
    rule = POLICY.get(tool)
    if rule is None or not rule["allowed"] or step >= MAX_STEPS:
        return False                                     # unknown, forbidden or over budget
    path = args.get("path", ALLOWED_DIR)
    if not path.startswith(ALLOWED_DIR) or ".." in path:
        return False                                     # outside the permitted folder
    return approve(tool, args) if rule["approval"] else True

print(guard("read_file", {"path": "/data/sales/q3.csv"}, 1, lambda t, a: False))  # True
print(guard("read_file", {"path": "/etc/passwd"}, 2, lambda t, a: True))          # False

Remember

Guardrails are checks the model cannot talk its way past
Enforce permissions in code and services, not only in prompts
Tiered approval, limits, sandboxes and audit logs around actions

Check yourself

Where should 'this agent may only read the Sales folder' be enforced?

Official docs: developers.openai.com/api/docs/guides/agent-builder-safety

Example

Remember

Check yourself

Builds on

Unlocks