Resources / Architecture
The operational cost of failing closed
Protect the boundary without hiding the availability cost. Design for known failure states, bounded retries and a recovery path that does not become a bypass.
- Failing closed is the correct default and it is not free — it trades availability for safety.
- The cost lands as blocked legitimate work when the policy path is unavailable.
- Making it practical is an engineering problem: scope, cache, and a path back.
The Default Nobody Questions
When software systems encounter an error, they have two choices: continue operating without the failed component, or stop. The industry default for most web applications is to continue. If the authentication cache goes down, serve the request anyway and log a warning. If the rate limiter is unreachable, let the traffic through and alert the ops team. This is fail-open behavior, and for most web applications, it is the right choice. Uptime matters. Users expect availability. A brief gap in rate limiting is less costly than downtime.
AI agents are not web applications. When an agent’s governance layer fails, continuing to operate means the agent is executing actions with no policy evaluation, no audit trail, no kill switch enforcement, and no approval workflow. Every tool call during that window is ungoverned. The agent is operating outside the boundaries the organization defined for it.
For autonomous systems that execute real-world actions — processing payments, modifying databases, sending emails, calling external APIs — fail-open is not a calculated tradeoff. It is an uncontrolled risk.
Fail-Open vs. Fail-Closed
The distinction is straightforward but its implications are often underestimated.
Fail-open: If the governance system cannot evaluate an action, the action is allowed to proceed. The assumption is that most actions are legitimate, and the cost of blocking a legitimate action outweighs the cost of allowing an unauthorized one.
Fail-closed: If the governance system cannot evaluate an action, the action is denied. The assumption is that an ungoverned action is an unacceptable risk, and the cost of allowing an unauthorized action outweighs the cost of temporarily blocking a legitimate one.
In network security, this distinction was settled long ago. Firewalls fail closed. If the firewall cannot evaluate a packet, the packet is dropped. The reasoning is simple: the cost of allowing a malicious packet through is greater than the cost of dropping a benign one. The same logic applies to AI agent governance.
Why It Matters for Autonomous Agents
The fail-open risk is amplified for autonomous agents because of three properties that distinguish them from human-driven systems.
Speed of Execution
An agent can make dozens of tool calls per second. A human operator encountering a governance outage might make one or two requests before noticing something is wrong. An agent encountering the same outage will continue executing at full speed. In the time it takes a monitoring system to detect the governance failure and page an engineer, an ungoverned agent can execute hundreds of actions.
The blast radius of a five-minute governance outage with a human-driven system is a handful of unchecked requests. The blast radius with an autonomous agent is potentially thousands.
No Self-Awareness of Governance State
Human users can recognize when a system is behaving unexpectedly. “The approval workflow did not trigger — something might be wrong.” Agents have no such awareness. An agent does not know whether its tool calls are being evaluated against policy. It submits a tool call, receives a response, and continues. If the governance layer silently fails open, the agent has no way to detect that it is operating outside its intended boundaries.
This is why governance failure detection must be external to the agent. The agent cannot be trusted to report its own governance state because it has no visibility into it.
Irreversibility of Actions
Many actions that agents take are difficult or impossible to reverse. A processed payment, a sent email, a deleted record, a deployed infrastructure change — these have real-world consequences that persist after the governance system recovers. Fail-open during a governance outage does not create a temporary gap in coverage. It creates permanent consequences that the organization must deal with long after the outage is resolved.
Where GovernorAI fails closed, and where it does not
- The gateway adapters. The Envoy ext_proc, MCP proxy and forward-auth adapters treat a transport error, timeout, non-2xx response or unreadable verdict from the GovernorAI gateway as a deny, and mark the denial as fail-closed so it is not mistaken for a policy deny.
- No matching policy. When no policy matches an action, the gateway denies it.
- Kill switches on a customer-managed gateway. A gateway that takes its kill switches from a signed policy bundle denies every call when that bundle is missing or expired, because it cannot confirm which switches are active.
Some paths can still let a call through: embedded OPA before it is initialised, hybrid mode after an OPA error (it falls back to the built-in verdict), a policy with fail_closed off for the tools it does not list, and some framework integrations. A customer-managed gateway keeps serving its last tenant snapshot while the control plane is unreachable, so a control-plane outage does not stop it. Check the path you deploy.
The Operational Cost of Fail-Closed
Fail-closed is not free. It means that governance system outages directly impact agent availability. When the governance layer is down, agents stop. This creates operational pressure to keep the governance system highly available, which is the right incentive structure.
The alternative — a governance system where outages do not impact agent availability because agents continue operating ungoverned — creates the wrong incentive. It means governance reliability is not a priority because failures are invisible. The governance system becomes the component that nobody notices when it is down, which means nobody notices when agents are running without policy enforcement.
Fail-closed makes governance failures visible. It turns them into incidents that get the same attention as any other infrastructure outage. This visibility is what drives investment in governance reliability.
Making Fail-Closed Practical
The objection to fail-closed is always the same: “We cannot afford to have our agents stop because of a governance outage.” This objection is valid. Two choices reduce how often an outage stops work.
Local evaluation. A customer-managed gateway evaluates policy in its own process, from its own policy mount, and keeps serving its last tenant snapshot while the control plane is unreachable. A control-plane outage therefore does not stop the agents it serves.
Several gateways. Run more than one gateway instance behind a load balancer, so the adapters still get a verdict when one instance is down.
The Principle
Fail-closed is not a feature. It is a design principle that flows through every architectural decision. It answers a fundamental question: when the governance system itself fails, what happens to the agents it governs?
If the answer is “they keep running,” then governance is advisory. It is a best-effort service that agents can operate without. If the answer is “they stop,” then governance is mandatory. It is infrastructure that agents depend on, the same way they depend on the network, the compute, and the model.
GovernorAI’s gateway adapters are built around the second answer: when an adapter cannot get a verdict, the call stops. The paths that can still let a call through are listed above, so you can choose the ones that match your answer.