API Gateway

You split a monolith into a dozen services and shipped it. Two weeks later the mobile team is filing tickets: they hard-coded six hostnames, three of the services rotated their auth scheme, and a load test just took down the billing service because nothing was throttling a retry storm. None of these are new problems. They are the same cross-cutting concerns the monolith solved implicitly, now scattered across a dozen deployables and one very confused client. The API Gateway pattern puts them back in one place — at the edge, where the outside world meets your system.

The problem

Without a gateway, every client talks to every service directly. That sounds clean until you count what it actually costs.

Client coupling. Each client — web, mobile, partner integrations — needs to know the address of every service it calls. When you rename a service, split one into two, or move it to a new cluster, every client ships a coordinated release. Mobile clients are the worst case: a bad hostname baked into a released binary can linger in the wild for months.

N×M cross-cutting concerns. Authentication, TLS termination, rate limiting, request logging, CORS, and API versioning are not business logic, but every service needs all of them. With N services and M concerns, you either implement M concerns N times or copy a shared library into N codebases and pray they stay in sync. They don't. One service validates JWTs with a five-minute clock skew, another with none; one enforces rate limits, another forgot. The gaps become your incident reports.

Duplication that drifts into risk. The dangerous version of duplication isn't the wasted keystrokes — it's the silent divergence. A security patch to your token-validation code has to land in every service. Miss one and you have an authentication bypass sitting in production that no single team owns. Centralizing auth, TLS, and throttling at the edge turns "N places to get right" into "one place to get right, N places to trust."

No coordinated back-pressure. When a downstream service degrades, clients retry. Retries amplify load. With no chokepoint, a slow dependency becomes a self-inflicted denial of service. You need somewhere to enforce quotas and shed load before it reaches fragile internals.

An API Gateway is a dedicated service that sits between clients and your backend, presenting one stable address and one consistent policy layer. Routing, auth, TLS, rate limiting, and request shaping live there. Services behind it get to assume they're being called by a trusted, well-behaved caller — because the gateway made it so.

When to use it

Reach for a gateway when the shape of your system matches these conditions:

• You have multiple services and multiple client types. The value scales with N services × M clients. Three services and one first-party web app can wait; a dozen services feeding web, mobile, and partner APIs cannot.
• Cross-cutting concerns are real and shared. If auth, rate limiting, TLS, and observability apply uniformly, a gateway lets you write and audit each once. This is the single strongest reason to adopt one.
• You want to decouple public API shape from internal topology. A gateway lets you refactor services — split, merge, rename, relocate — without breaking published contracts. The URL clients depend on stays put while everything behind it moves.
• You need a Backend-for-Frontend (BFF). Mobile wants a lean, aggregated payload; the web wants something richer. A gateway (or a BFF variant per client) can compose several backend calls into one response, cutting client round-trips over slow networks.
• You have an organization to serve, not just code. At team scale, a gateway is where platform engineering enforces standards — API versioning, deprecation headers, quota tiers — without negotiating with every product team. This is as much an org tool as a technical one.

The honest sweet spot: a growing engineering org (say, 4+ teams) running 8+ services behind multiple clients, where consistency and safe evolution matter more than shaving the last millisecond off internal calls.

When NOT to use it

A gateway is infrastructure you now own, monitor, scale, and can take down your entire product with. Don't add one reflexively.

• You have a single service or a tiny system. One service behind a gateway is just a proxy adding a hop, a failure mode, and an on-call rotation for no benefit. Point the client at the service. Add the gateway when you have a second and third service to unify.
• Latency-critical internal service-to-service calls. A gateway is an edge concern. Routing east-west traffic between internal services through a central gateway adds a hop and a chokepoint to your hottest paths. For internal calls, prefer direct addressing or a service mesh (sidecar proxies) where the policy lives next to the workload, not in a central box. The gateway is for north-south (client-to-system) traffic.
• You'd turn the gateway into a monolith. The failure mode that sinks gateway projects is scope creep: business logic, request orchestration, data transformation, and per-feature special cases all migrate into the gateway until it's the monolith you escaped — except now it's a single point of failure that every team must change and no team owns cleanly. Keep the gateway policy-only: routing, auth, throttling, shaping. The moment domain logic appears in gateway config, stop and push it back into a service.
• You can't staff it. A gateway is a tier-0 dependency. If it's down, everything is down. If you don't have the operational maturity to run it with redundancy, health checks, and a tested failover, a premature gateway lowers your availability instead of raising it.

Rule of thumb: adopt a gateway when the pain of not having one (duplication, coupling, no back-pressure) exceeds the cost of running one more critical tier. Below that line, resist.

Architecture

Clients hit one endpoint. The gateway resolves cross-cutting concerns, then routes to the right backend, discovering instances via service discovery and protecting fragile downstreams with circuit breakers.

flowchart LR
  web[Web app]
  mobile[Mobile app]
  partner[Partner API]

  subgraph edge[API Gateway]
    tls[TLS termination]
    authz[AuthN / AuthZ]
    rate[Rate limiting]
    route[Routing table]
  end

  web --> edge
  mobile --> edge
  partner --> edge

  tls --> authz --> rate --> route

  route --> orders[Orders service]
  route --> billing[Billing service]
  route --> catalog[Catalog service]

  disc[(Service discovery)] -.-> route
  route -. via circuit breaker .-> billing

The pipeline is ordered deliberately: terminate TLS first, authenticate and authorize next, enforce rate limits before any backend is touched, then route. Cheap rejections (bad token, over quota) happen before expensive work. Service discovery feeds the routing table so instances can come and go without config edits, and calls to failure-prone dependencies pass through a circuit breaker so a sick backend can't drag the edge down with it.

Code walkthrough

A minimal gateway in TypeScript using Hono, a small, fast web framework that runs on Node and edge runtimes. This is a sketch of the concerns, not a drop-in product — but the APIs are real.

Start with a declarative routing table. Keeping routes as data (not scattered if statements) is what keeps a gateway from rotting into a monolith.

// routes.ts — routing as data, not logic
export interface Route {
  prefix: string;        // public path prefix
  upstream: string;      // resolved base URL of the backend
  public?: boolean;      // if true, skip auth (e.g. health, login)
}

export const routes: Route[] = [
  { prefix: "/v1/orders", upstream: "http://orders.internal:8080" },
  { prefix: "/v1/billing", upstream: "http://billing.internal:8080" },
  { prefix: "/v1/catalog", upstream: "http://catalog.internal:8080", public: true },
  { prefix: "/healthz", upstream: "http://localhost:8080", public: true },
];

export function matchRoute(path: string): Route | undefined {
  // longest-prefix wins, so /v1/orders/exports beats /v1
  return routes
    .filter((r) => path.startsWith(r.prefix))
    .sort((a, b) => b.prefix.length - a.prefix.length)[0];
}

Next, authentication as middleware that runs before routing. Here we verify a JWT with jose and attach the claims to the request context so downstream logging and forwarding can use them. In production the upstream URL would come from service discovery rather than a static string — that's the seam where the service discovery pattern plugs in.

// auth.ts — one place to get token validation right
import { createMiddleware } from "hono/factory";
import { jwtVerify, createRemoteJWKSet } from "jose";
import { matchRoute } from "./routes";

const JWKS = createRemoteJWKSet(new URL(process.env.JWKS_URL!));

export const auth = createMiddleware(async (c, next) => {
  const route = matchRoute(c.req.path);
  if (route?.public) return next(); // public routes skip auth

  const token = c.req.header("authorization")?.replace(/^Bearer\s+/i, "");
  if (!token) return c.json({ error: "missing bearer token" }, 401);

  try {
    const { payload } = await jwtVerify(token, JWKS, {
      issuer: process.env.JWT_ISSUER,
      audience: process.env.JWT_AUDIENCE,
    });
    c.set("subject", payload.sub); // available to later middleware/handlers
    await next();
  } catch {
    return c.json({ error: "invalid token" }, 401);
  }
});

Rate limiting comes next — a fixed-window counter keyed by the authenticated subject (or client IP for public routes). This sketch uses an in-memory map for clarity; a real deployment uses Redis so the limit is shared across gateway replicas rather than per-instance.

// rate-limit.ts — cheap rejection before expensive backend work
import { createMiddleware } from "hono/factory";

const WINDOW_MS = 60_000;
const MAX = 100; // requests per subject per window
const hits = new Map<string, { count: number; resetAt: number }>();

export const rateLimit = createMiddleware(async (c, next) => {
  const key = c.get("subject") ?? c.req.header("x-forwarded-for") ?? "anon";
  const now = Date.now();
  const entry = hits.get(key);

  if (!entry || now > entry.resetAt) {
    hits.set(key, { count: 1, resetAt: now + WINDOW_MS });
  } else if (entry.count >= MAX) {
    const retry = Math.ceil((entry.resetAt - now) / 1000);
    return c.json({ error: "rate limit exceeded" }, 429, { "retry-after": String(retry) });
  } else {
    entry.count++;
  }
  await next();
});

Finally, wire the pipeline and forward matched requests upstream. Note the ordering — auth and rate limiting run before the proxy handler ever touches a backend.

// index.ts — the pipeline: TLS (at the LB) -> auth -> rate limit -> route
import { Hono } from "hono";
import { auth } from "./auth";
import { rateLimit } from "./rate-limit";
import { matchRoute } from "./routes";

const app = new Hono();
app.use("*", auth);
app.use("*", rateLimit);

app.all("*", async (c) => {
  const route = matchRoute(c.req.path);
  if (!route) return c.json({ error: "no route" }, 404);

  const target = route.upstream + c.req.path.slice(route.prefix.length);
  // A wrapping circuit breaker (see Related patterns) belongs around this fetch.
  const upstream = await fetch(target, {
    method: c.req.method,
    headers: c.req.raw.headers,
    body: c.req.raw.body,
  });
  return new Response(upstream.body, {
    status: upstream.status,
    headers: upstream.headers,
  });
});

export default app;

That's the whole idea: routing is data, each concern is a small ordered middleware, and business logic stays out. When someone proposes adding a currency conversion or an order-total calculation "just here in the gateway," this is the structure that makes the wrongness obvious — it has no place to go.

Performance characteristics

The metrics in the frontmatter are indicative, drawn from well-known characteristics of reverse proxies rather than measured benchmarks of this implementation. Real numbers come later, once the live demo is instrumented.

Added latency (~1-10 ms p50, indicative). A gateway inserts one extra network hop. For in-region traffic the cost is dominated by that hop and TLS work, not gateway CPU — a well-built proxy adds low single-digit milliseconds at p50. Where it bites is the tail: if the gateway is a chokepoint and saturates, p99 latency climbs sharply. Size it with headroom and watch the tail, not just the median. This latency tax is exactly why you keep the gateway on north-south traffic and off hot internal paths.

Centralized auth (1 place vs N, indicative). The operational win isn't a benchmark, it's a count. Token validation logic lives and gets patched in one place instead of N services. The security value of "one place to audit" is hard to overstate and impossible to get from copied libraries that drift.

TLS termination (1 edge, indicative). Certificates, cipher policy, and rotation are managed at the edge. Backends can speak plain HTTP inside a trusted network (or mTLS if you need zero-trust internally), and you renew and harden TLS in one config surface.

Client round-trips (1 vs many, indicative). With BFF-style aggregation, the gateway collapses several backend calls into one client request. Over high-latency mobile links, cutting three round-trips to one often dwarfs the millisecond the gateway added — the net user-perceived latency drops.

The through-line: a gateway trades a small, predictable latency cost for large gains in consistency, safety, and evolvability. That trade is worth it at the edge and a mistake in the interior.

Live demo

A live, deployed API Gateway demo is coming soon. Rather than a canned animation, it will run as a real gateway in front of real backend services — you'll be able to watch routing, auth rejections, and rate-limit responses happen against live instances, with the latency figures above replaced by measured numbers from the running system.

Related patterns

An API Gateway rarely ships alone; it's the composition point for two patterns in particular.

Circuit Breaker wraps the gateway's calls to fragile downstreams. When billing starts timing out, the breaker trips and the gateway fails fast — returning a fast error or cached fallback — instead of piling up connections and dragging the whole edge into the same failure. The gateway is the natural home for this because it already sees and mediates every downstream call.

Service Discovery feeds the routing table. Instead of hard-coding upstream hostnames (as the walkthrough does for clarity), the gateway asks a registry where the healthy instances of a service currently live and load-balances across them. New instances register on startup and drop out on failure, so the gateway routes around dead backends without a config change or redeploy. Together, discovery tells the gateway where to send traffic and the circuit breaker decides when to stop — the gateway is where both decisions are enforced on every request.