API Design in Production: 15 Lessons From the Gateway
Nine years of Golang on AWS taught me one discipline: name the promise, price the boundary, plus retire versions before they retire you. Each lesson starts from a real production failure, states its assumptions up front, plus ends in math you can re-run. The bill hides in the boundary.
TL;DR: Fifteen forensic API lessons with live diagrams, from contracts to idempotency plus rate limits plus deprecation math. Built from production postmortems with calculators for the math.
By Mukul Kumar Mishra · Backend plus SRE course · Updated September 26, 2026
What you will be able to do
- Write contracts clients can code against, version them without casualties, plus retire old versions on published dates.
- Make retries safe with idempotency keys, cap floods with token buckets, plus fail with errors clients can act on.
- Place policy correctly across gateway plus mesh, prove identity at the edge, plus deliver webhooks that survive the night.
- Page through millions at constant cost, run pact drills per commit, plus price the gateway month as a modeled invoice.
- Answer API interviews with clarified scope, out-loud bucket math, plus three ways you would break your own contract.
How interviews test this course: clarify scope first, estimate out loud second, break your own design third. Lesson 14 rehearses the bench.
The Contract Is the Product
Assumption first: clients outlive every implementation behind them. Stripe processed $1.4 trillion without double charging anything because the API promise held while ledgers plus regions plus vendors moved underneath. Design the promise before the handler. Version the promise like money depends on it, because it does.
- Name resources as nouns plus actions as verbs. Verbs in URLs age into lies the moment a second client arrives.
- One contract, many handlers. Stripe plus Razorpay both kept promises while swapping queues plus regions underneath.
- Deep dive: The Ledger That Cannot Blink, idempotency keys plus double-entry discipline at $1.4 trillion.
Q1. Design checkout for a store doing 40,000 RPS. What is your first endpoint? Seen at: backend plus API loops.
Q2. A partner stored your v1 response shape in their warehouse. How do you ship v2? Seen at: gateway plus platform loops.
Promise named. Now keep it across versions. Lesson 02: Versioning →
Version Without Breakage
Every breaking change is a migration you force on strangers. Salesforce once queued logins behind one shared path on conference day two, which is what a forced migration feels like at scale. Ship v2 beside v1. Measure adoption. Sunset with dates plus headers, never with surprises.
- Prefer URL versions for public APIs plus header versions for partners who negotiate. Either works when the sunset date is contractual.
- Emit Sunset plus Deprecation headers on every old response. Clients that can see the deadline meet the deadline.
- Deep dive: The Login Queue That Ate Dreamforce, shared paths plus forced marches.
Q1. Your mobile app has a 6-week review lag. How do you version? Seen at: backend plus API loops.
Q2. When is header versioning better than URL versioning? Seen at: gateway plus platform loops.
Versions live side by side. Now make retries safe. Lesson 03: Idempotency →
Idempotency Keys End Double Charges
Networks retry. That is not a bug, that is physics. The bug is charging twice for one intent. Stripe solved it with idempotency keys plus a double-entry ledger, so a retried request replays the stored result instead of repeating the side effect. Every mutating endpoint needs the same discipline.
- Keys ride in headers, never in bodies. The key identifies intent while the body carries details.
- Store key plus response for at least 24 hours. A retry inside the window replays. Outside it starts fresh.
- Deep dive: The Ledger That Cannot Blink, exactly the machinery, priced.
Q1. Design fund transfer with at-least-once delivery. Where does the key live? Seen at: backend plus API loops.
Q2. A client reuses one key for two different payments. What breaks? Seen at: gateway plus platform loops.
Retries made safe. Now cap the flood. Lesson 04: Limits →
Rate Limits Are Capacity
Azure West US taught the industry that retries multiply a 75 percent capacity loss into a 6 times overload at the surviving edge. Limits at the gateway turn that flood into a queue with fast errors instead of slow timeouts. Scale without limits replays the incident faster. Limits without scale prolongs the pain. Together they converge.
- Gate per key plus per route plus per region. One bucket per dimension, each with its own refill math.
- Return 429 with Retry-After plus a limit-policy header. Clients that can compute backoff will back off.
- Deep dive: Azure West US Route Withdrawal, admission math at 1.4M offered RPS.
Q1. Size a token bucket for 90k burst RPS against 12k steady. Show the refill. Seen at: backend plus API loops.
Q2. When do you shed with 429 versus queue with 202? Seen at: gateway plus platform loops.
Flood gated. Now write the contract down. Lesson 05: OpenAPI →
OpenAPI Turns Promises Into Artifacts
Thirteen thousand postings in ninety days ask for REST API skill, plus five thousand ask for OpenAPI by name. The spec is the contract clients code against, mocks run from, plus gateways enforce. A spec reviewed like code prevents the drift that becomes the midnight page.
- Lint specs in CI: no verb URLs, no unpaged lists, no undocumented errors. Red spec blocks the merge.
- Generate server stubs plus client SDKs plus mocks from one source. Three artifacts, zero drift.
- Deep dive: Razorpay Kafka, what happens when client contracts meet partition limits.
Q1. Your spec has 200 endpoints. How do you review it without drowning? Seen at: backend plus API loops.
Q2. Codegen versus hand-written SDKs. When does each win? Seen at: gateway plus platform loops.
Contract written. Now place the enforcement. Lesson 06: Topology →
Gateway Versus Mesh
A gateway enforces at the border. A mesh enforces between every pair of services. Use the gateway for client-facing policy such as auth plus limits plus versioning. Use the mesh for service-to-service policy such as mTLS plus per-route budgets. One border, many interiors. Confusing the two buys two bills plus one gap.
- Terminate TLS plus validate auth plus apply limits at the gateway. Fail fast before interior spend.
- Run mTLS plus retries plus timeouts in the mesh. Interior policy travels with the call, not the docs.
- Deep dive: Cell-Based Control Planes, bulkheads that hold when maps fail.
Q1. Draw request path for a checkout call through gateway plus mesh. Label every policy point. Seen at: backend plus API loops.
Q2. When does a mesh add more risk than it removes? Seen at: gateway plus platform loops.
Policy placed. Now prove identity. Lesson 07: Auth →
Auth at the Edge
None of your handlers should parse credentials. The edge validates tokens, checks scopes, plus forwards identity downstream as headers services can trust. OAuth scopes carry permission. mTLS carries workload identity. IAM carries cloud permission. Mix them deliberately or attackers will mix them for you.
- Validate plus cache JWKS with short TTLs. A slow validator becomes the outage, as us-west1 proved for IAM.
- Scope per action, not per service. Broad scopes turn one leaked token into a skeleton key.
- Deep dive: GCP us-west1, when IAM validation queues behind disk.
Q1. Design scope model for marketplace with buyers plus sellers plus admins. Seen at: backend plus API loops.
Q2. Your JWKS endpoint slows to 4 seconds. What breaks first? Seen at: gateway plus platform loops.
Identity settled. Now survive the callback. Lesson 08: Webhooks →
Webhooks at Scale
Stripe-scale webhooks mean millions of outbound callbacks into customer endpoints of unknown quality. Sign every payload. Retry with backoff plus jitter across hours. Keep per-destination queues so one slow customer cannot stall the rest. The callback is your API in reverse, with all the same discipline owed.
- HMAC-sign payloads with rotating secrets plus versioned signatures. Unsigned callbacks are rumors.
- Retry 8 times over 3 days with jitter. Dead-letter the rest to a dashboard someone actually reads.
- Deep dive: The Ledger That Cannot Blink, webhook storms at 1M RPS.
Q1. Design webhook delivery for 1M events with 5 percent failing endpoints. Seen at: backend plus API loops.
Q2. How do you rotate signing secrets without dropping trust? Seen at: gateway plus platform loops.
Callbacks tamed. Now page through millions. Lesson 09: Paging →
Pagination Discipline
Offset pagination scans deeper with every page, which means page 10,000 costs a full table walk. Cursor pagination walks an index forward at constant cost. At Discord trillions-of-messages scale the difference is the cluster. Default to cursors. Reserve offsets for admin UIs with small result sets.
- Encode cursor as opaque token with sort key plus tiebreaker. Clients iterate, never compute.
- Cap page size plus enforce max window. A client asking for a million rows is asking for your outage.
- Deep dive: Discord Trillions of Messages, hot partitions plus quorum fan-in.
Q1. Price offset page 50,000 against cursor page 50,000 at 1M rows. Show the gap. Seen at: backend plus API loops.
Q2. When is offset pagination still the right call? Seen at: gateway plus platform loops.
Lists flattened. Now fail usefully. Lesson 10: Errors →
Errors Clients Can Handle
Every error must answer three questions: what happened, whether retry helps, plus what to do next. Machine-readable codes plus retryable flags plus request IDs turn a 3 AM mystery into a 3 PM ticket. Salesforce darkness taught that support goes dark with the product unless errors carry their own flashlight.
- Mark every error retryable or terminal. Ambiguous errors breed the retry storms of Lesson 04.
- Echo a request ID in every response plus every log line. Correlation without IDs is archaeology.
- Deep dive: The Login Queue, darkness with no flashlight.
Q1. Design an error catalog for payments with 12 failure modes. Seen at: backend plus API loops.
Q2. Your 500s spike. How does a client tell retryable from fatal? Seen at: gateway plus platform loops.
Failures legible. Now retire the old. Lesson 11: Sunsets →
Deprecation Economics
Every live version bills forever: test matrices plus docs plus support plus incident surface. Deprecation converts that annuity into a project with a date. Announce early, header every response, offer a migration sandbox, then delete. Versions are inventory. Inventory rots.
- Price each live version per quarter: CI minutes plus support tickets plus incident share. Publish the number internally.
- Give migration sandboxes with diff tooling. The version that is easy to leave is the version clients actually leave.
- Deep dive: Figma Multiplayer, per-document isolation as versioning discipline.
Q1. Two versions cost how much per quarter? Model it for your fleet. Seen at: backend plus API loops.
Q2. A top customer refuses migration. What is your playbook? Seen at: gateway plus platform loops.
Old retired. Now price the whole gateway. Lesson 12: Billing →
The Gateway Bill
Assume 1M RPS through gateway plus auth plus limits at a modeled $0.0000012 per origin request. That is about $3,100 per day before data plus support. Token buckets that shed 65 percent of storm load save $70,000 per five-hour storm at that scale. Run the RPS Envelope calculator with your own numbers. The bill hides in the boundary. Here the boundary has a rate limit.
- Model per-request cost as gateway plus auth plus origin. Three lines, no mysteries.
- Hold headroom as insurance with a published price. Unpriced headroom gets cut the quarter before the storm.
- Deep dive: Azure West US, the full invoice model.
Q1. Price your gateway at current RPS with modeled unit cost. What dominates? Seen at: backend plus API loops.
Q2. A storm triples offered load. What does the gated invoice read? Seen at: gateway plus platform loops.
Invoice priced. Now rehearse breakage. Lesson 13: Contracts →
Breaking-Change Drills
Consumer-driven contracts flip testing: consumers publish what they need, providers verify every build against those pacts. A pact suite that runs per commit catches the renamed field before it ships. Pact tests are cheaper than apologies by roughly three orders of magnitude. Run them like billing depends on it.
- Collect pacts from top consumers first. Ten pacts catch more breakage than ten thousand unit tests of shape.
- Verify pacts in CI plus canary. A pact that only runs nightly is a history book.
- Deep dive: Kafka Rebalance Explained, contract discipline under protocol change.
Q1. Design a pact suite for 3 consumers sharing one checkout API. Seen at: backend plus API loops.
Q2. A pact breaks at 2 AM. Roll forward or roll back? Seen at: gateway plus platform loops.
Breakage rehearsed. Now face the panel. Lesson 14: Interviews →
The Interview Bench
API interviews test three moves: clarify scope first, estimate out loud second, break your own design third. Rate-limiting questions want bucket math. Webhook questions want retry horizons. Versioning questions want sunset plans. Answer with numbers plus tradeoffs plus one story from these lessons. Panels hire engineers who price their own designs.
- Open with throughput plus latency plus consistency asks. Five minutes of questions saves forty of redesign.
- Close every answer with failure modes plus cost. Designs without prices are sketches.
- Deep dive: System Design Course, Lesson 01 starts from outages.
Q1. Design rate limiting for 100M users in 45 minutes. Talk while you draw. Seen at: backend plus API loops.
Q2. Tell me about an API breakage you caused plus fixed. Seen at: gateway plus platform loops.
Bench warmed. Now ship for real. Lesson 15: Capstone →
Ship One API
Start small on purpose. One resource, three clients, one gateway. Write the OpenAPI spec, generate mocks, gate with buckets, sign webhooks, publish pacts, run a breaking-change drill, price the month. An API you have broken on purpose is the only kind you understand.
- Run the fourteen drills from lessons 1 through 13 against one real contract. Gaps show at a glance.
- Kill a version mid-week plus measure migration. Untested sunsets are rumors with headers.
- Close with the invoice. Requests per month times modeled unit price, labeled as estimates.
Q1. Ship the spec plus the gateway config to one reviewer. What do they flag? Seen at: backend plus API loops.
Q2. Your webhook endpoint goes dark Friday night. Walk the runbook. Seen at: gateway plus platform loops.
Course complete. Interview the whole system next: System Design: 15 Lessons From Real Outages
Sources plus method
This course teaches from published Buildopsy postmortems linked under each lesson: Stripe idempotency, Razorpay client contracts, Salesforce queues, Azure admission math, Discord pagination, Figma isolation, Kafka protocol discipline, plus the gateway invoice model. Incident details follow those accounts. Cost framing is modeled from public list prices.