Circuit Breaker

A single slow dependency should not be able to take down your entire service. Yet it happens constantly, and the mechanism is almost always the same: one downstream system degrades, callers pile up waiting on timeouts, threads and connections exhaust, and the failure propagates upward until the whole graph is dark. The Circuit Breaker pattern is the standard defense. It is a small state machine that sits between a caller and a remote dependency, watches for failures, and — when a dependency is clearly unhealthy — stops sending traffic to it and fails immediately instead.

The name is borrowed from electrical engineering, and the analogy is exact. A house circuit breaker trips to protect the wiring from a fault downstream; it does not repair the fault, it isolates it. The software version does the same thing: it protects the caller from a broken callee.

The problem

Consider a checkout service that calls a pricing service to compute taxes. Under normal load the pricing call returns in 20 ms. One afternoon the pricing service's database begins to struggle — a bad query plan, a lock contention spike, whatever — and pricing calls start taking 10 seconds before eventually timing out.

Here is what happens next, in order:

1. Latency inflates. Every checkout now blocks for 10 seconds on the pricing call. Response times that were budgeted in milliseconds are now measured in tens of seconds.
2. Threads exhaust. The checkout service runs a bounded thread pool (or connection pool, or event-loop concurrency limit). Each in-flight request holds a thread while it waits. At 200 requests/second and a 10-second wait, you need 2,000 concurrent threads to keep up. You do not have 2,000 threads. The pool saturates.
3. The caller stops serving everything. Once the pool is exhausted, the checkout service can no longer accept any request — not just pricing-dependent ones. Health checks time out. The load balancer marks the instance unhealthy and shifts traffic to its siblings, which are drinking from the same poisoned well and fall over in turn.
4. Retries make it worse. Well-meaning retry logic now multiplies the load on the already-drowning pricing service by 2x or 3x. This is a retry storm: the dependency is failing because it is overloaded, and the callers respond by sending more traffic. The system converges on total collapse.

This is a cascading failure, and its defining feature is that the blast radius is far larger than the original fault. The pricing database hiccup should have degraded one feature — tax calculation — for a few minutes. Instead it took down checkout entirely, and possibly every service that depends on checkout.

The root cause is that waiting is not free. A timeout is a promise to hold resources for its full duration before giving up. When a dependency is reliably failing, waiting for the timeout on every single call is the worst possible strategy: you pay maximum cost for zero information you did not already have. The circuit breaker's core insight is that once you know a dependency is down, you should stop paying the timeout tax and fail instantly.

When to use it

Reach for a circuit breaker when all of the following hold:

• The call crosses a process or network boundary. Remote calls to other services, databases, caches, message brokers, or third-party APIs are the natural home for breakers, because those are the calls that can hang, time out, and exhaust pools.
• The dependency can fail independently and stay failed for a while. Transient blips are handled by retries; breakers earn their keep against sustained degradation lasting seconds to minutes.
• The caller has a bounded resource it must protect. Thread pools, connection pools, or concurrency limits are what get exhausted. If saturating the caller has systemic consequences, a breaker is protecting something real.
• You have a sensible fallback or a graceful failure. Failing fast is only useful if the caller can do something reasonable with the failure: serve a cached value, degrade the feature, queue the work, or return a clean error instead of a hung request.

The classic fit is a service-to-service call behind an API gateway, or any fan-out call in a microservices topology where one slow leaf can back-pressure the whole tree.

When NOT to use it

Breakers are not free, and applying them reflexively adds fragility of a different kind. Skip the breaker when:

• The call is in-process. A function call within the same process does not exhaust a network pool and cannot hang the way a remote call can. Wrapping local logic in a breaker adds state and failure modes for no protective benefit.
• Retry plus a tight timeout already suffices. For idempotent operations that fail fast — a quick 400 from a validation endpoint, a fast connection-refused — the failure is already cheap. You are not accumulating held resources, so there is nothing for the breaker to protect. A bounded retry with a strict timeout is simpler and sufficient.
• The added state complexity outweighs the benefit. A breaker is stateful: it remembers recent outcomes. In a serverless or heavily horizontally-scaled deployment, each instance holds its own breaker state, so a fleet of 500 short-lived Lambdas each learns "pricing is down" independently and slowly. In-memory breakers on ephemeral compute give you weak, inconsistent protection; you often want a shared health signal (a service mesh, or the gateway) instead.
• You cannot define "failure." If every response looks like success at the transport layer and you have no way to classify a call as failed, the breaker has nothing to count. Fix your failure detection first.

A good rule: a breaker protects a caller from a slow or failing remote callee. If either half of that sentence is not true, you probably do not need one.

Architecture

A circuit breaker is a three-state machine. In Closed state, calls flow through normally and outcomes are recorded. When the failure rate crosses a threshold, the breaker trips to Open: calls are rejected immediately without touching the dependency. After a cooldown, the breaker moves to Half-Open and admits a single probe call to test whether the dependency has recovered. Success closes the breaker; failure re-opens it and restarts the cooldown.

stateDiagram-v2
    [*] --> Closed
    Closed --> Open: failure threshold exceeded
    Open --> HalfOpen: reset timeout elapsed
    HalfOpen --> Closed: probe succeeds
    HalfOpen --> Open: probe fails
    Closed --> Closed: success (reset counters)

The runtime request path differs sharply depending on state. When Closed, the caller pays the full cost of a real request. When Open, the caller pays almost nothing — the breaker short-circuits and returns immediately, either raising an error or invoking a fallback.

flowchart TD
    A[Caller] --> B{Breaker state?}
    B -->|Closed / Half-Open| C[Call downstream]
    B -->|Open| D[Reject immediately / fallback]
    C --> E{Success?}
    E -->|Yes| F[Record success]
    E -->|No| G[Record failure]
    F --> A
    G --> A
    D --> A

The essential property visible in these diagrams is that the Open path never reaches the downstream box. That is the whole point: while the dependency is unhealthy, the breaker spends the caller's resources on failing fast rather than on waiting.

Code walkthrough

Here is a correct, dependency-free circuit breaker in TypeScript. It uses a simple consecutive-failure count for clarity; I discuss the rolling-window refinement afterward. Start with the state model and configuration.

type State = "closed" | "open" | "half-open";

interface BreakerOptions {
  failureThreshold: number; // consecutive failures before opening
  resetTimeoutMs: number;   // how long to stay open before probing
}

class CircuitOpenError extends Error {
  constructor() {
    super("Circuit breaker is open");
    this.name = "CircuitOpenError";
  }
}

The three states map directly to the state diagram. CircuitOpenError is what callers catch to trigger a fallback — it is distinct from a downstream error, so callers can tell "the dependency failed" apart from "we refused to call the dependency."

Next, the breaker itself. The core method, execute, wraps any async operation. Before running, it decides whether the call is even allowed based on the current state and the clock.

class CircuitBreaker {
  private state: State = "closed";
  private failures = 0;
  private nextAttempt = 0; // epoch ms; when Open may transition to Half-Open

  constructor(private readonly opts: BreakerOptions) {}

  async execute<T>(operation: () => Promise<T>): Promise<T> {
    if (this.state === "open") {
      if (Date.now() < this.nextAttempt) {
        throw new CircuitOpenError(); // still cooling down: fail fast
      }
      this.state = "half-open"; // cooldown elapsed: allow one probe
    }

    try {
      const result = await operation();
      this.onSuccess();
      return result;
    } catch (err) {
      this.onFailure();
      throw err;
    }
  }

The gate logic is the heart of the pattern. When Open, the breaker compares the clock against nextAttempt. If the cooldown has not elapsed, it throws CircuitOpenError in microseconds without ever awaiting operation() — this is the fail-fast behavior that protects the caller's pool. Once the cooldown passes, it flips to Half-Open and lets exactly one call through as a probe.

Finally, the outcome handlers that drive the transitions:

  private onSuccess(): void {
    // A successful probe (or normal call) fully resets the breaker.
    this.failures = 0;
    this.state = "closed";
  }

  private onFailure(): void {
    this.failures++;
    // A failed probe in half-open re-opens immediately.
    // In closed, we open once we cross the threshold.
    if (this.state === "half-open" || this.failures >= this.opts.failureThreshold) {
      this.state = "open";
      this.nextAttempt = Date.now() + this.opts.resetTimeoutMs;
    }
  }

  get currentState(): State {
    return this.state;
  }
}

onSuccess is deliberately absolute: any success closes the breaker and clears the counter. onFailure encodes both trip conditions — a failure while Half-Open re-opens instantly (the probe told us the dependency is still sick), and a failure in Closed opens only after the threshold is crossed. Both paths set nextAttempt to schedule the next probe. Usage is a single line:

const breaker = new CircuitBreaker({ failureThreshold: 5, resetTimeoutMs: 10_000 });

try {
  const price = await breaker.execute(() => pricingClient.getPrice(sku));
  return price;
} catch (err) {
  if (err instanceof CircuitOpenError) return cachedPrice(sku); // fast fallback
  throw err; // a real downstream error
}

Two refinements matter for production. First, replace the consecutive-failure counter with a rolling window (count failures over the last N requests or the last T seconds) so a breaker under mixed traffic trips on a rate, not on an unlucky streak. Second, wrap operation() in its own timeout, because a breaker that never times out its calls can still let every Closed-state request hang — the breaker limits how many slow calls you make, but the timeout limits how long each one waits. Breaker and timeout are complementary, not alternatives.

Performance characteristics

The metrics in the panel above are indicative — sensible defaults and order-of-magnitude figures, not measurements from a specific benchmark. They exist to build intuition about the trade-offs.

Fail-fast latency vs. timeout waits. This is the entire value proposition. A rejected call in the Open state returns in well under a millisecond — it is a state check and a clock comparison. A call that instead waits on a hung dependency pays the full timeout, commonly 1 to 30 seconds. When a dependency is down, the breaker converts every one of those multi-second waits into a sub-millisecond rejection. That is a four-to-five order-of-magnitude reduction in resource hold time per failed call, which is precisely what keeps the caller's thread pool from saturating.

Memory per breaker. A single breaker instance is tiny: a state enum, a couple of counters, and a timestamp — on the order of a couple hundred bytes. Even a rolling-window implementation storing recent outcomes is measured in kilobytes. You can afford one breaker per downstream dependency (and you should — a shared breaker across unrelated dependencies causes false trips), so a service talking to a dozen downstreams carries a negligible memory footprint for its breakers.

Threshold tuning. The two knobs that matter are the failure threshold and the reset timeout, and both are genuine trade-offs. A low threshold trips eagerly: it protects the caller quickly but risks opening on transient noise and denying service that would have succeeded. A high threshold tolerates blips but lets more slow calls through before protecting you. A short reset timeout probes for recovery aggressively, restoring service fast but hammering a dependency that is still down; a long reset timeout is gentler on the dependency but keeps the feature dark longer after it recovers. Sensible starting points are a 50% failure rate over a 20-request window and a 10-30 second reset, then tune against real incident data. There is no universal correct value — the right numbers depend on your traffic volume, your dependency's failure signature, and how costly a false trip is to your users.

Live demo

An interactive breaker playground is coming soon. You will be able to drive a live breaker through Closed, Open, and Half-Open by dialing up the downstream failure rate and latency, watch the state transitions in real time, and see the fail-fast latency win against a plain timeout. Check back shortly.

Related patterns

Retry is the pattern most often confused with the circuit breaker, and the distinction is worth stating plainly: a breaker is not a retry. Retry assumes the failure is transient and tries the same call again, hoping it succeeds this time. A breaker assumes the failure is sustained and refuses to call at all until a cooldown passes. They operate at different timescales and, crucially, in different directions — retry adds load, a breaker removes it. Used together, the breaker must sit outside the retry, or a retry storm will happily exhaust your pool inside a Closed breaker that never gets a chance to trip. They compose well when layered correctly and fight each other when they are not.

Bulkhead is the natural companion. Where a breaker stops calls to an unhealthy dependency, a bulkhead partitions resources so that one saturated dependency cannot consume the pool that serves the others — separate connection pools or concurrency limits per downstream. Breaker plus bulkhead is a strong combination: the bulkhead contains the blast radius even before the breaker trips, and the breaker stops the bleeding once the dependency is clearly down.

API Gateway is where breakers often live in practice. Centralizing breakers at the gateway (or in a service mesh sidecar) gives you consistent policy, shared health signal across instances, and observability in one place — which also addresses the serverless weakness noted earlier, where per-instance in-memory breakers learn too slowly. See the API Gateway pattern for how breaker policy composes with routing and rate limiting.

A note on the ecosystem, because the history is instructive. Netflix Hystrix popularized the pattern in the JVM world around 2012 and made circuit breaking a default expectation for microservices. Netflix put Hystrix into maintenance mode in 2018 — not because the pattern was wrong, but because a heavy, per-call-wrapping library was the wrong shape; the industry moved toward lighter libraries like resilience4j (functional, composable, no thread-pool isolation baggage) and toward pushing resilience concerns out of application code and into the service mesh. The pattern is more relevant than ever; the implementation just got lighter.