Implementation design · 001-rate-limiting

Per-tenant API rate limiting

A pure token-bucket core behind a hexagonal port, wired as request middleware — fail-open, metered, and swappable for a shared store when the distributed slice lands.

draft spec 001-rate-limiting owner Platform API (@platform) created 2026-07-21 read time 6 min

TL;DR

A token-bucket limiter behind a hexagonal port, wired as request middleware: the core is a pure, testable library; in-process policy and state stores are the v1 adapters. It fails open, emits a decision metric per request, and is swappable for a Redis-backed store when the distributed slice lands.

§1 Principles check

Checked against principles v1.0.0 — all twenty are walked. The ones with no surface in this slice are marked n/a with a reason; any violation would need a revised decision or a recorded, accepted risk.

Principle	Gate	How met
P-1 Clarity over cleverness	OK	Pure token-bucket core; no clever indirection.
P-2 Small, reversible changes	OK	Shipped as a slice — the distributed limiter is deferred, not bundled.
P-3 Explicit over implicit	OK	Named ports (policy · state · clock); no action-at-a-distance.
P-4 One set of conventions, enforced	OK	§2 wires Prettier + ESLint, applied automatically.
P-5 Record the why	OK	Every §6 decision carries its Context and Consequences.
P-6 Tests accompany behavior	OK	Tests-first tasks (T-100 before T-110); every FR closed by a task.
P-7 Done means verified	OK	Unit (core), integration (middleware), and a fairness load test (SC-001).
P-8 Ship dark by default	OK	Enabled per tenant via config; can start monitor-only before it enforces.
P-9 Decisions are grounded	OK	Every §6 decision cites a §3 grounding row.
P-10 The enforcement floor holds	OK	§2 wires the 8 categories spectastic enforce requires.
P-11 Contract-first interfaces	OK	The limiter port is a typed interface checked in before the adapters.
P-12 Library-first boundaries	OK	Pure limiter core; the HTTP middleware is a thin adapter.
P-13 Secure by default	OK	Tenant identity from the auth context only (FR-003).
P-14 Least privilege	OK	The limiter reads only the tenant id and its policy — nothing broader.
P-15 Structured observability	OK	One decision metric per request, tagged by tenant + outcome (US3).
P-16 Service level objectives	OK	NFR-001/002 each carry a <spec-slo>.
P-17 Semantic versioning	n/a	An internal feature of the API service; no independently released artifact.
P-18 Supply-chain hygiene	OK	§2 wires npm audit + a committed lockfile.
P-19 Supply-chain provenance	n/a	SBOM + attestation are produced at the service release, not this feature.
P-20 Accessibility conformance	n/a	No user-facing surface — satisfied trivially (per P-20).

§2 Technical context

Language(s): TypeScript 5.x on Node 20 (LTS).
Frameworks: Express 4 (existing API); no new web framework.
Architecture: Hexagonal ports-and-adapters — a pure limiter core behind a port, with in-process store adapters (D-003).
Storage: In-process Map for policy + state (v1); a Redis adapter is the deferred distributed slice.
Testing: Vitest unit tests for the core; a load-test harness for the fairness criterion (SC-001).
Coverage: Diff-aware: changed lines covered; a change never lowers coverage of what it touches.
Perf targets: p99 < 5 ms added latency (NFR-001); O(1) per decision.
Constraints: Single-node, in-process for v1; must fail open (NFR-002).

Toolchain & enforcement — enterprise profile

enterprise hard gate spectastic enforce 1 covered · 8 missing

The enterprise gate needs 8 categories wired. This design maps each to the Node ecosystem's standard tool:

Formatter: Prettier
Linter: ESLint (typescript-eslint)
Type-checker: tsc --noEmit (strict)
Test-runner: Vitest
Security: Semgrep (ci ruleset)
Supply-chain: npm audit + committed lockfile
Coverage: Vitest v8 coverage, diff-aware
Observability: OpenTelemetry metric on every decision

§3 Grounding & evidence

Facts the decisions rest on, each with its source and confidence.

Fact	Source	Status	Finding
429 + Retry-After is the interoperable throttling contract	RFC 6585 §4	verified	Standard status + header clients already back off against.
Token bucket admits bursts within an average rate at O(1)/decision	Tanenbaum, Computer Networks §5.4	verified	Constant-time refill+draw; naturally burst-fair.
An in-process Map decision is sub-millisecond on Node 20	Local micro-benchmark	spike	~0.4 µs/op in a throwaway bench; well inside the 5 ms budget.
Every authenticated request carries a resolvable tenant id	Spec assumption (auth middleware)	verified	Confirmed present in the existing auth context object.

§4 Approach

Put a pure token-bucket limiter behind a port. The HTTP middleware resolves the tenant from the auth context, asks the core for a Decision, and either forwards the request (setting rate-limit headers) or short-circuits with 429.

Policy and state live behind small store interfaces so the in-process v1 adapters swap for a shared store later without touching the core or the middleware.

§5 Alternatives considered

Option	Burst-fair	Memory	Accuracy	Total
Token bucket ✓	5	5	4	14
Sliding-window log	4	2	5	11
Sliding-window counter	3	5	3	11

Token bucket wins: O(1) memory per tenant, natural burst allowance, and accuracy within tolerance for throttling — we don't need per-request audit precision.

§6 Decisions

D-001Architecture decisionverified

Token-bucket limiter

Context: Rests on the §3 rows RFC 6585 §4 and Computer Networks §5.4 — 429 semantics plus a constant-time, burst-fair algorithm.
Decision: Use a token bucket keyed by tenant; refill at limit/window, draw one token per request.
Consequences: + O(1) per decision, burst-tolerant. − Approximate over long windows (accepted; SC-001 measures fairness, not exactness).

D-002Architecture decisionverified

Fail open on limiter faults

Context: NFR-002 and its <spec-slo> (§3: confirmed auth-context path) — the limiter must never be the cause of an outage.
Decision: Any store/backend error in the decision path is caught and the request is allowed, with an error metric emitted.
Consequences: + A limiter fault can't take the API down. − A sustained fault lets traffic through unthrottled (bounded by alerting on the error metric).

D-003Architecture decisionn/a

In-process stores for v1

Context: Scope judgment: the spec defers the distributed limiter; per-node budgets are acceptable for v1 (spec §6 assumption).
Decision: Ship in-process Map policy + state adapters behind the store ports; add a Redis adapter in the deferred slice.
Consequences: + No new infra, fastest path to value. − Per-node drift within a tenant's budget until the shared store lands (tracked as a risk).

§7 Project structure

Only the new or changed paths, grouped by the hexagonal boundaries.

src/ratelimit/
 ├─ core/       token-bucket.ts   (pure)
 ├─ ports/      policy-store.ts · state-store.ts · clock.ts
 ├─ adapters/   memory-policy-store.ts · memory-state-store.ts
 └─ http/       middleware.ts     (thin adapter)
tests/ratelimit/
 └─ token-bucket.test.ts · middleware.test.ts · fairness.loadtest.ts

§8 Risks & mitigations

Risk	L	I	Mitigation
Per-node drift lets a tenant exceed its global budget across N nodes	M	M	Documented v1 limitation; distributed slice deferred with defer-to.
Sustained fail-open masks an abusive tenant during a store outage	L	M	Alert on the limiter error metric; outage is bounded and visible.

§9 Change log

1.0.0 · 2026-07-21

Initial design.

This worked example

principles.html spec.html design.html tasks.html ↳ walkthrough ▶ simulator

One file. Renders anywhere. Degrades to readable static HTML.