API Design in Production: 15 Lessons From the Gateway
Nine years of Golang on AWS taught me one discipline: name the promise, price the boundary, and retire versions before they retire you. Each lesson starts from a real production failure, states its assumptions up front, and ends in math you can re-run. The bill hides in the boundary.
TL;DR: Fifteen forensic API lessons with live diagrams, from contracts to idempotency and rate limits and deprecation math. Built from production postmortems with calculators for the math.
By Mukul Kumar Mishra · Backend and SRE course · Updated October 23, 2026
What you will be able to do
- Write contracts clients can code against, version them without casualties, and retire old versions on published dates.
- Make retries safe with idempotency keys, cap floods with token buckets, and fail with errors clients can act on.
- Place policy correctly across gateway and mesh, prove identity at the edge, and deliver webhooks that survive the night.
- Page through millions at constant cost, run pact drills per commit, and price the gateway month as a modeled invoice.
- Answer API interviews with clarified scope, out-loud bucket math, and three ways you would break your own contract.
How interviews test this course: clarify scope first, estimate out loud second, break your own design third. Lesson 14 rehearses the bench.
The Contract Is the Product
Clients outlive every implementation behind them. Stripe reported $1.4 trillion in 2024 total payment volume; that dollar-volume figure is not a transaction count or proof that every payment succeeded exactly once. Its API documentation describes idempotency keys for safely retrying eligible requests. Design the promise before the handler.
- Name resources as nouns and actions as verbs. Verbs in URLs age into lies the moment a second client arrives.
- One contract, many handlers. Keep the client-visible promise stable while implementation details change.
- Deep dive: Stripe's $1.4T 2024 Volume: Idempotency and a Modeled Scale Teardown, with documented API behavior and explicitly modeled rates.
Q1. Design checkout for a store doing 40,000 RPS. What is your first endpoint?
Q2. A partner stored your v1 response shape in their warehouse. How do you ship v2?
Promise named. Now keep it across versions. Lesson 02: Versioning →
Version Without Breakage
Every breaking change is a migration you force on strangers. Salesforce once queued logins behind one shared path on conference day two, which is what a forced migration feels like at scale. Ship v2 beside v1. Measure adoption. Sunset with dates and headers, never with surprises.
- Prefer URL versions for public APIs and header versions for partners who negotiate. Either works when the sunset date is contractual.
- Emit Sunset and Deprecation headers on every old response. Clients that can see the deadline meet the deadline.
- Deep dive: Salesforce Dreamforce Login Stall, with a reported service symptom and an explicitly modeled queue explanation.
Q1. Your mobile app has a 6-week review lag. How do you version?
Q2. When is header versioning better than URL versioning?
Versions live side by side. Now make retries safe. Lesson 03: Idempotency →
Idempotency Keys End Double Charges
Networks retry. The API contract must distinguish a retry of one intent from a new operation. Stripe's documentation says an eligible request with the same idempotency key can reuse its saved result instead of repeating the operation. This is an API behavior, not a claim of global exactly-once processing across every downstream system.
- Keys ride in headers, never in bodies. The key identifies intent while the body carries details.
- Store key and response for at least 24 hours. A retry inside the window replays. Outside it starts fresh.
- Deep dive: Stripe's $1.4T 2024 Volume: Idempotency and a Modeled Scale Teardown, the API guarantee and its scenario limits.
Q1. Design fund transfer with at-least-once delivery. Where does the key live?
Q2. A client reuses one key for two different payments. What breaks?
Retries made safe. Now cap the flood. Lesson 04: Limits →
Rate Limits Are Capacity
Azure West US taught the industry that retries multiply a 75 percent capacity loss into a 6 times overload at the surviving edge. Limits at the gateway turn that flood into a queue with fast errors instead of slow timeouts. Scale without limits replays the incident faster. Limits without scale prolongs the pain. Together they converge.
- Gate per key and per route and per region. One bucket per dimension, each with its own refill math.
- Return 429 with Retry-After and a limit-policy header. Clients that can compute backoff will back off.
- Deep dive: Azure West US Route Withdrawal, admission math at 1.4M offered RPS.
Q1. Size a token bucket for 90k burst RPS against 12k steady. Show the refill.
Q2. When do you shed with 429 versus queue with 202?
Flood gated. Now write the contract down. Lesson 05: OpenAPI →
OpenAPI Turns Promises Into Artifacts
Thirteen thousand postings in ninety days ask for REST API skill, and five thousand ask for OpenAPI by name. The spec is the contract clients code against, mocks run from, and gateways enforce. A spec reviewed like code prevents the drift that becomes the midnight page.
- Lint specs in CI: no verb URLs, no unpaged lists, no undocumented errors. Red spec blocks the merge.
- Generate server stubs and client SDKs and mocks from one source. Three artifacts, zero drift.
- Deep dive: Razorpay Kafka, what happens when client contracts meet partition limits.
Q1. Your spec has 200 endpoints. How do you review it without drowning?
Q2. Codegen versus hand-written SDKs. When does each win?
Contract written. Now place the enforcement. Lesson 06: Topology →
Gateway Versus Mesh
A gateway enforces at the border. A mesh enforces between every pair of services. Use the gateway for client-facing policy such as auth and limits and versioning. Use the mesh for service-to-service policy such as mTLS and per-route budgets. One border, many interiors. Confusing the two buys two bills and one gap.
- Terminate TLS and validate auth and apply limits at the gateway. Fail fast before interior spend.
- Run mTLS and retries and timeouts in the mesh. Interior policy travels with the call, not the docs.
- Deep dive: Cell-Based Control Planes, bulkheads that hold when maps fail.
Q1. Draw a request path for a checkout call through a gateway and mesh. Label every policy point.
Q2. When does a mesh add more risk than it removes?
Policy placed. Now prove identity. Lesson 07: Auth →
Auth at the Edge
None of your handlers should parse credentials. The edge validates tokens, checks scopes, and forwards identity downstream as headers services can trust. OAuth scopes carry permission. mTLS carries workload identity. IAM carries cloud permission. Mix them deliberately or attackers will mix them for you.
- Validate and cache JWKS with short TTLs. Google's us-west1 report lists IAM among services affected by regional network degradation; it does not identify token validation as the cause.
- Scope per action, not per service. Broad scopes turn one leaked token into a skeleton key.
- Deep dive: GCP us-west1, where a fiber-maintenance capacity loss reached IAM and other dependent products.
Q1. Design a scope model for a marketplace with buyers, sellers and admins.
Q2. Your JWKS endpoint slows to 4 seconds. What breaks first?
Identity settled. Now survive the callback. Lesson 08: Webhooks →
Webhooks at Scale
Webhook endpoints receive asynchronous events and can fail independently of the original request. Stripe's documentation says live-mode delivery can retry for up to three days and does not guarantee event order. Sign each payload, deduplicate processed event IDs, and isolate slow destinations in your own design.
- HMAC-sign payloads with rotating secrets and versioned signatures. Unsigned callbacks are rumors.
- Set a retry policy that fits your service. Stripe documents retries for up to three days in live mode; its public docs do not specify an eight-attempt schedule.
- Deep dive: Stripe's $1.4T 2024 Volume: Idempotency and a Modeled Scale Teardown, including webhook delivery limits and modeled load.
Q1. Design webhook delivery for 1M events with 5 percent failing endpoints.
Q2. How do you rotate signing secrets without dropping trust?
Callbacks tamed. Now page through millions. Lesson 09: Paging →
Pagination Discipline
Offset pagination scans deeper with every page, which means page 10,000 costs a full table walk. Cursor pagination walks an index forward at constant cost. At Discord trillions-of-messages scale the difference is the cluster. Default to cursors. Reserve offsets for admin UIs with small result sets.
- Encode cursor as opaque token with sort key and tiebreaker. Clients iterate, never compute.
- Cap page size and enforce max window. A client asking for a million rows is asking for your outage.
- Deep dive: Discord message storage, hot partitions and quorum fan-in.
Q1. Price offset page 50,000 against cursor page 50,000 at 1M rows. Show the gap.
Q2. When is offset pagination still the right call?
Lists flattened. Now fail usefully. Lesson 10: Errors →
Errors Clients Can Handle
Every error must answer three questions: what happened, whether retry helps, and what to do next. Machine-readable codes, retryable flags, and request IDs make failures easier to diagnose. Salesforce Trust reported that some customers could not create support cases during its September 16 disruption while public status updates continued; its record does not say all support channels were unavailable.
- Mark every error retryable or terminal. Ambiguous errors breed the retry storms of Lesson 04.
- Echo a request ID in every response and every log line. Correlation without IDs is archaeology.
- Deep dive: Salesforce Dreamforce Login Stall: 16 Instance IDs, 7h36m, with the status record separated from the queue model.
Q1. Design an error catalog for payments with 12 failure modes.
Q2. Your 500s spike. How does a client tell retryable from fatal?
Failures legible. Now retire the old. Lesson 11: Sunsets →
Deprecation Economics
Every live version bills forever: test matrices and docs and support and incident surface. Deprecation converts that annuity into a project with a date. Announce early, header every response, offer a migration sandbox, then delete. Versions are inventory. Inventory rots.
- Price each live version per quarter: CI minutes and support tickets and incident share. Publish the number internally.
- Give migration sandboxes with diff tooling. The version that is easy to leave is the version clients actually leave.
- Deep dive: Figma Multiplayer, per-document isolation as versioning discipline.
Q1. Two versions cost how much per quarter? Model it for your fleet.
Q2. A top customer refuses migration. What is your playbook?
Old retired. Now price the whole gateway. Lesson 12: Billing →
The Gateway Bill
Assume 1M RPS through gateway and auth and limits at a modeled $0.0000012 per origin request. That is about $1.20 per second, or about $103,700 per day, before data and support (1M x 86,400 x $0.0000012; illustrative scenario, not a measured invoice). Shedding 65 percent of a modeled ~5M RPS storm for five hours saves about $70,000 in origin compute at the same unit rate (0.65 x 5M x 18,000s x $0.0000012). Run the RPS Envelope calculator with your own numbers. The bill hides in the boundary. Here the boundary has a rate limit.
- Model per-request cost as gateway and auth and origin. Three lines, no mysteries.
- Hold headroom as insurance with a published price. Unpriced headroom gets cut the quarter before the storm.
- Deep dive: Azure West US, the full invoice model.
Q1. Price your gateway at current RPS with modeled unit cost. What dominates?
Q2. A storm triples offered load. What does the gated invoice read?
Invoice priced. Now rehearse breakage. Lesson 13: Contracts →
Breaking-Change Drills
Consumer-driven contracts flip testing: consumers publish what they need, providers verify every build against those pacts. A pact suite that runs per commit catches the renamed field before it ships. Pact tests are cheaper than apologies by roughly three orders of magnitude. Run them like billing depends on it.
- Collect pacts from top consumers first. Ten pacts catch more breakage than ten thousand unit tests of shape.
- Verify pacts in CI and canary. A pact that only runs nightly is a history book.
- Deep dive: Kafka Rebalance Explained, contract discipline under protocol change.
Q1. Design a pact suite for 3 consumers sharing one checkout API.
Q2. A pact breaks at 2 AM. Roll forward or roll back?
Breakage rehearsed. Now face the panel. Lesson 14: Interviews →
The Interview Bench
API interviews test three moves: clarify scope first, estimate out loud second, break your own design third. Rate-limiting questions want bucket math. Webhook questions want retry horizons. Versioning questions want sunset plans. Answer with numbers and tradeoffs and one story from these lessons. Panels hire engineers who price their own designs.
- Open with throughput and latency and consistency asks. Five minutes of questions saves forty of redesign.
- Close every answer with failure modes and cost. Designs without prices are sketches.
- Deep dive: System Design Course, Lesson 01 starts from outages.
Q1. Design rate limiting for 100M users in 45 minutes. Talk while you draw.
Q2. Tell me about an API breakage you caused and fixed.
Bench warmed. Now ship for real. Lesson 15: Capstone →
Ship One API
Start small on purpose. One resource, three clients, one gateway. Write the OpenAPI spec, generate mocks, gate with buckets, sign webhooks, publish pacts, run a breaking-change drill, price the month. An API you have broken on purpose is the only kind you understand.
- Run the fourteen drills from lessons 1 through 13 against one real contract. Gaps show at a glance.
- Kill a version mid-week and measure migration. Untested sunsets are rumors with headers.
- Close with the invoice. Requests per month times modeled unit price, labeled as estimates.
Q1. Ship the spec and gateway config to one reviewer. What do they flag?
Q2. Your webhook endpoint goes dark Friday night. Walk the runbook.
Course complete. Interview the whole system next: System Design: 15 Lessons From Real Outages
Sources and method
This course teaches from published Buildopsy postmortems linked under each lesson: Stripe idempotency, Razorpay client contracts, Salesforce queues, Azure admission math, Discord pagination, Figma isolation, Kafka protocol discipline, and the gateway invoice model. Incident details follow those accounts. Cost framing is modeled from public list prices.