Skip to content

7 chapters · 91 minutes

The workshop

From an API that says yes to everything to limiters you can test, a server that says no properly, and a client that backs off. Work through them in order; each ends with a way to check it worked.

  1. 01Why rate limitsOne busy client can take your API down for everyone. Start a real server with no limit and watch it say yes to everything.10 min
  2. 02Fixed windowThe simplest limiter: count requests in each one-second window. It is easy to build and it has a hole you can drive twice the traffic through.15 min
  3. 03Sliding logRemember when every allowed request happened, and count only the last second. Exact, with a memory bill.12 min
  4. 04Token bucketAllow short bursts on purpose, and hold the average rate steady. The limiter most public APIs use.15 min
  5. 05Say no properly: HTTP 429A limit is only useful if clients can tell they hit it and know when to come back. Status code, headers and a key that cannot be faked.12 min
  6. 06Retry politelyThe other side of the limit: a client that backs off, adds jitter and respects Retry-After, instead of hammering harder.15 min
  7. 07Test it, then ship itLimiters are full of time and threads, which makes them easy to get wrong. Test them with a fake clock, run the tests in CI, and know what changes with more than one server.12 min