DISTRIBUTED SYSTEMS
Distributed Tiered Rate Limiter
A Redis + Lua rate limiting system with three interchangeable algorithms, per-API-key tiers, Prometheus metrics, k6 load tests, and a live React dashboard.
Overview
A production-style distributed rate limiter where Sliding Window Counter, Sliding Window Log, and Token Bucket all run atomically in Redis Lua — each mapped to a different API-key tier (anonymous / free / pro) so the tradeoffs are observable, not just described. Includes Prometheus metrics, health/readiness probes, graceful shutdown, OpenAPI docs, Docker Compose, and real k6 benchmark numbers.
The Problem
Naive in-process rate limiters break across multiple servers. Real SaaS products need shared Redis state, atomic enforcement under concurrency, and different limits per customer tier — without duplicating the enforcement logic.
Architecture
Key Decisions
Three algorithms, one Redis Lua core
Sliding Window Counter (anonymous), Sliding Window Log (free), and Token Bucket (pro) share the same middleware path — tiers swap algorithms, not code paths.
Tiered per-API-key limits
X-Api-Key resolves anonymous/free/pro with deliberately different algorithms and budgets so burst vs exactness tradeoffs are visible in the dashboard and k6 runs.
Observability as first-class
Prometheus histograms/counters, /health + /ready probes, structured pino JSON logs, and OpenAPI at /docs — not bolted on after the demo.
Graceful shutdown
On SIGTERM/SIGINT the server drains in-flight requests, closes Redis cleanly, then exits — safe for rolling deploys and docker compose stop.
Failure Modes
Concurrent hammering of one identifier
Lua runs atomically inside Redis — GET/CHECK/INCR race conditions cannot slip between processes or PM2 workers.
Redis unreachable
/ready returns 503 so load balancers stop routing; /health stays up so the process is not killed for a dependency blip.
Rolling deploy mid-request
Graceful shutdown drains in-flight work with a 10s force-exit safety net instead of dropping connections.
Monitoring blind spots
/logs is intentionally not rate-limited so operators can still inspect behavior under attack or load.
Implementation
- Redis Lua implementations of Sliding Window Counter, Log, and Token Bucket
- Middleware-based enforcement with X-RateLimit-Tier / Limit / Remaining headers
- React + Vite dashboard with live logs and tier simulation buttons
- Prometheus /metrics, /health, /ready, and pino structured logging
- k6 load tests with captured anonymous and pro-tier benchmark numbers
- Docker Compose stack (Redis + API + nginx client) and GitHub Actions CI
- 25 automated tests across Vitest suites (algorithms + middleware + dashboard)
Performance
99.47% 429s
Anon enforcement
~349 / 300+refill
Pro allowed ≈ theory
12.37ms
Anon p95 latency
Lessons
- —Without Lua, distributed rate limiting has a classic GET→CHECK→INCR race under concurrency.
- —Running three algorithms side by side makes tradeoffs measurable instead of theoretical.
- —Health vs readiness probes matter — one checks the process, the other checks dependencies.