Saga

A saga is how you get "all or nothing" semantics for a business process that spans several services, when you have deliberately given up the shared transaction that would make "all or nothing" easy. Instead of one atomic commit, you run a sequence of local transactions, and for every step that can move the system forward you also define a step that can move it back. When something fails halfway, you do not roll back — you run the backward steps for everything that already succeeded. That is the whole idea. Everything else is about doing it without corrupting data or losing your mind.

The problem

Consider the most ordinary flow in commerce: a customer places an order. Behind that single click, four services have to agree.

• Order service records the order.
• Payment service authorizes and captures money.
• Inventory service decrements stock and reserves units.
• Shipping service books a carrier and schedules a pickup.

In a monolith with one database, this is a single BEGIN … COMMIT. If shipping fails, the whole transaction rolls back, the money is never captured, and the stock is never decremented. The database gives you atomicity for free.

Split those four responsibilities into four services with four databases and that guarantee evaporates. Payment succeeds, inventory succeeds, and then shipping fails because the carrier's API is down. Now you have charged a customer for goods you will never ship. There is no ROLLBACK that reaches across four separate databases.

The textbook answer is two-phase commit (2PC): a coordinator asks every participant to prepare, and only if all vote yes does it tell them to commit. It is correct on paper and a liability in production:

• It blocks. Between prepare and commit, every participant holds locks. A slow or partitioned participant stalls the others, and lock contention becomes a throughput ceiling under load.
• The coordinator is a single point of failure. If it dies after some participants have prepared but before the commit decision propagates, those participants are stuck holding locks, unable to safely commit or abort. This is the classic in-doubt state.
• Nothing modern supports it. Kafka, most managed queues, HTTP APIs, and horizontally-sharded or NoSQL stores do not expose an XA prepare phase. You cannot enlist a REST call or a Kafka publish into a distributed transaction. The infrastructure the industry actually runs on simply does not offer the primitive 2PC needs.

So you drop the fantasy of a global atomic commit and accept a weaker, more honest guarantee: the system will be eventually consistent, passing through visible intermediate states, and you will design explicitly for the moments when it is only partway done. That design is the saga.

When to use it

Reach for a saga when all of these hold:

• A business operation genuinely spans multiple services that each own their data, and you cannot (or should not) merge those data stores.
• Every step can be semantically undone. Not bit-for-bit reversed — semantically compensated. A captured payment is undone by a refund, a reserved seat by a release, an email by... well, that is the hard case (see below).
• You can tolerate brief inconsistency windows. For a few hundred milliseconds to a few seconds, an order may be "paid but not yet shipped." If the business can live with that intermediate reality, a saga fits.
• Throughput matters enough that blocking locks are unacceptable. Sagas hold no cross-service locks, so they scale where 2PC chokes.

Order fulfillment, travel booking (flight + hotel + car), account opening across KYC/funding/provisioning, and money movement between ledgers are the canonical fits.

When NOT to use it

Sagas are machinery, and machinery has a carrying cost. Do not build one when:

• The data can live in one service and one database. This is the most common mistake. Teams split an aggregate across services for organizational reasons, then reach for a saga to stitch it back together. If order and payment intent can share a boundary, put them in one transactional store and use a plain local transaction. A saga you did not need is pure liability.
• Compensations are impossible. Some actions have no meaningful inverse. You cannot un-send a physical letter, un-launch a missile, or un-email a customer their password. If a step has an irreversible external side effect, either move it to the very end (so nothing after it can fail) or reconsider the design. A "compensation" that only pretends to undo is a lie in your data model.
• Volume is low and reconciliation is cheap. If you process a few dozen of these flows a day, a nightly reconciliation job and a human who fixes exceptions is cheaper, simpler, and easier to audit than a saga engine. Do not build a state machine to handle what an operator can clear over coffee.
• You actually need strict serializable isolation. Sagas expose intermediate states. If your domain cannot tolerate another actor observing "reserved but not paid," you may need real locking, and a saga is the wrong tool.

Architecture

There are two ways to coordinate the steps, and choosing between them is one of the more consequential decisions you will make.

Choreography — no central brain. Each service reacts to events and emits its own. The workflow is emergent.

sequenceDiagram
    participant O as Order
    participant P as Payment
    participant I as Inventory
    participant S as Shipping
    O->>P: OrderCreated (event)
    P->>I: PaymentCaptured (event)
    I->>S: StockReserved (event)
    S-->>I: ShippingFailed (event)
    Note over I,P: failure ripples backward
    I->>P: StockReleased (compensation)
    P->>O: PaymentRefunded (compensation)

Orchestration — a central coordinator (the saga orchestrator) issues commands and reacts to replies, driving the flow explicitly. The workflow lives in one place.

flowchart TD
    Start([Order placed]) --> C{Orchestrator}
    C -->|1. capture| P[Payment]
    C -->|2. reserve| I[Inventory]
    C -->|3. book| S[Shipping]
    S -->|fail| X{{Failure}}
    X -->|compensate 2| IC[Inventory: release]
    X -->|compensate 1| PC[Payment: refund]
    IC --> End([Saga: aborted, consistent])
    PC --> End

The trade-off is real and it is mostly about teams and debuggability, not raw performance:

• Choreography couples services only through events, so it fits autonomous teams and avoids a central bottleneck. The cost is that no single artifact describes the workflow. To answer "what happens when an order is placed?" you read the code of four services and trace event subscriptions. Cyclic event dependencies creep in. Debugging a stuck flow means correlating logs across services. It works beautifully for two or three steps and becomes a maze at seven.
• Orchestration puts the entire workflow in one readable place. You can look at the orchestrator and know the sequence, the compensations, and the current state of any in-flight saga. That clarity is worth a great deal operationally. The cost is a component that knows about all participants — a mild coupling — and one more service to run. For anything non-trivial, prefer orchestration; the debuggability alone pays for it.

Two ideas make either style safe. First, semantic locks: because you hold no database locks across steps, you protect intermediate state at the application level. Inventory that is "reserved" is a pending state — visible, but flagged so other sagas do not double-book it. The reservation is your semantic lock, released by either confirmation or compensation. Second, and most important: compensations are business decisions, not technical rollbacks. A rollback restores the previous bytes. A compensation issues a new business fact — a refund, a credit, an apology, a release — that counteracts the original. The refund is itself a real event with real accounting consequences. Model it as such.

Code walkthrough

We will build a small orchestrated saga in TypeScript. Start with the vocabulary: a step is an action plus its compensation.

interface SagaStep<Ctx> {
  name: string;
  // Move the saga forward. May mutate ctx with results (e.g. paymentId).
  action: (ctx: Ctx) => Promise<void>;
  // Undo the effect of `action`. Must be idempotent and safe to retry.
  compensate: (ctx: Ctx) => Promise<void>;
}

interface SagaState {
  id: string;
  completedSteps: string[]; // names of steps whose action succeeded
  status: 'running' | 'completed' | 'compensating' | 'aborted';
}

The completedSteps list is the spine of recovery: it records exactly which actions succeeded, so we know precisely which compensations to run and in what order if we later crash and resume.

Next, persistence. Saga state must survive a process crash, so every transition is written to durable storage before we act on it. This turns the orchestrator into a resumable state machine rather than a fragile in-memory loop.

interface SagaStore {
  save(state: SagaState): Promise<void>;
  load(id: string): Promise<SagaState | null>;
}

// A record is written twice per step: once when we mark completion of an
// action, and once when we mark completion of its compensation. On restart,
// a resumer loads any state whose status is 'running' or 'compensating'
// and continues from where the log says we stopped.

Now the executor. It runs actions forward; on any failure it stops and unwinds the compensations in reverse order.

async function runSaga<Ctx>(
  steps: SagaStep<Ctx>[],
  ctx: Ctx,
  state: SagaState,
  store: SagaStore,
): Promise<SagaState> {
  try {
    for (const step of steps) {
      if (state.completedSteps.includes(step.name)) continue; // idempotent resume
      await step.action(ctx);
      state.completedSteps.push(step.name);
      await store.save(state); // durable point: this action is now "done"
    }
    state.status = 'completed';
    await store.save(state);
    return state;
  } catch (err) {
    state.status = 'compensating';
    await store.save(state);
    // Unwind in reverse. Compensate only what actually succeeded.
    for (const step of [...steps].reverse()) {
      if (!state.completedSteps.includes(step.name)) continue;
      await step.compensate(ctx);
      state.completedSteps = state.completedSteps.filter((n) => n !== step.name);
      await store.save(state);
    }
    state.status = 'aborted';
    await store.save(state);
    return state;
  }
}

Two subtleties carry the correctness of the whole thing. Resume idempotency: the if (state.completedSteps.includes(step.name)) continue guard means that if the process crashes after store.save but before the next step, a resumer replays the loop and skips already-done work — the action never fires twice. Reverse-order compensation with completion tracking: we only compensate steps whose actions succeeded, and we do it in the reverse of the order they ran, mirroring how a stack unwinds.

Finally, the actions and compensations themselves must be idempotent, because at-least-once delivery and crash-resume both mean they can be invoked more than once. The standard tool is an idempotency key.

const paymentStep: SagaStep<OrderCtx> = {
  name: 'capture-payment',
  action: async (ctx) => {
    // The gateway dedupes on the key: a retry returns the original capture
    // instead of charging the customer twice.
    const res = await payments.capture({
      idempotencyKey: `${ctx.sagaId}:capture`,
      amount: ctx.total,
      customerId: ctx.customerId,
    });
    ctx.paymentId = res.paymentId;
  },
  compensate: async (ctx) => {
    if (!ctx.paymentId) return; // action never took effect; nothing to undo
    await payments.refund({
      idempotencyKey: `${ctx.sagaId}:refund`,
      paymentId: ctx.paymentId,
    });
  },
};

Note that the refund is a first-class business operation with its own idempotency key — not a rollback, but a compensating fact recorded in the ledger. That is the insight made concrete.

Performance characteristics

The figures in the panel above are indicative, not measured — order-of-magnitude reasoning to set expectations, to be replaced by real benchmarks later.

• End-to-end latency. A saga replaces one local transaction with a chain of N local transactions, each preceded by a network hop and a state-persistence write. Expect end-to-end latency to be roughly the sum of the per-step latencies, commonly 5-50x a single local commit depending on N and whether steps run sequentially. Steps with no data dependency can sometimes run in parallel, which cuts wall-clock time at the cost of a more complex compensation graph.
• Compensation rate. The cost that matters is not the happy path — it is how often you unwind. A compensation roughly doubles the work for the steps involved (you did them, now you undo them) and touches external systems again (a refund, a release). If your compensation rate is a fraction of a percent, the overhead is noise. If a downstream service is flaky and you are compensating 20% of sagas, that instability, not the saga pattern, is your real problem. Watch this metric; it is a health signal for your dependencies.
• Saga-state storage. Every step transition is a durable write, so a saga of N steps produces on the order of 2N+1 rows or events over its life (two transitions per step plus a terminal record). At scale this is a meaningful write load and a growing table. Archive or compact terminal sagas, and index on status so the resumer's "find stuck sagas" query stays cheap.

Live demo

An interactive saga walkthrough — driving the order flow through success and forced-failure paths, with the compensation unwind visualized step by step — is coming soon. It will be published as a live implementation via stop.procrastin.ar and linked here once available.

Related patterns

• Pub/Sub is the natural transport for a choreographed saga: services publish domain events and subscribe to the ones that drive their next step. The event bus is the connective tissue that lets the workflow emerge without a coordinator.
• Event Sourcing pairs cleanly with sagas for state management. If your saga's progress is itself an event stream (StepCompleted, CompensationStarted), you get a durable, auditable history of every transition for free, and recovery becomes a replay.
• The outbox pattern solves the atomicity gap inside a single step. When a step must both commit local data and publish an event, write the event to an "outbox" table in the same local transaction, then relay it asynchronously. This guarantees the event is published if and only if the local change committed — closing the dual-write hole that would otherwise let a saga lose a step's message.
• Two-phase commit is the alternative a saga replaces. 2PC buys you strict atomicity and isolation at the price of blocking locks and a coordinator that can leave participants in doubt. The saga trades that strictness for availability and throughput, accepting eventual consistency and visible intermediate states as the cost of doing business at scale.