# Distributed rate limits need a consistency model too > Choose centralized, local, leased, or approximate quotas by fairness, latency, availability, and acceptable overshoot during failures. Canonical URL: https://www.devobs.io/articles/distributed-rate-limit-semantics/ By: Jonah Reed Published: 2023-04-09T01:34:05.461Z Updated: 2026-09-05 Section: Architecture A global rate limit is a distributed counter with a product contract. Before choosing an algorithm, define the partition key, time model, maximum acceptable overshoot, fairness target, outage behavior, and what a response tells the client. Exactness, low latency, and regional availability cannot all be assumed during a partition. ## Decide what the limit protects A safety limit protects a scarce dependency and may need to reject aggressively. A fairness limit divides capacity among tenants. An abuse control responds to adversarial patterns. A commercial quota tracks an entitlement and may require durable accounting. These purposes tolerate different errors. Brief overshoot can be harmless for a notification endpoint and unacceptable for a paid one-time operation. A centralized counter gives one ordering point and the clearest global view, but adds a network hop and a shared failure domain. During its outage, callers must either deny, allow, or use a local emergency budget. The decision should follow the protected resource: deny when overshoot could cause material harm; permit a small bounded reserve when availability matters more. Local token buckets keep decisions fast and available, but each replica can spend its own burst. Adding replicas increases possible overshoot unless budgets are rebalanced. They suit per-instance protection or limits where approximate fleet behavior is acceptable. Hashing a partition key to one owner reduces duplication but introduces rebalancing and hot-shard concerns. ## Bound overshoot with leased quotas A central allocator can lease chunks of quota to regions or instances. Local decisions spend from the lease without a round trip; global overshoot is bounded by outstanding leases. Smaller leases improve fairness and revocation speed but require more allocator traffic. Larger leases improve autonomy and waste more capacity when a holder disappears. Expiry and epoch numbers prevent an old region from spending a lease after failover. Envoy documents a [global rate limiting architecture](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting) in which the proxy calls an external rate-limit service using configured descriptors. That separation makes policy centralized while enforcement remains at traffic proxies, but teams must still design the service's storage consistency, timeout behavior, and failure policy. For multi-region failover, record whether counters follow the client, tenant, region, or resource. Active-active regions need shared or partitioned quota semantics. If a partition lets both sides spend, quantify the worst case as number of independent spenders times their remaining local budget. “Eventually consistent” is incomplete until that number is acceptable. ## Communicate hints honestly The current IETF [RateLimit header fields draft](https://datatracker.ietf.org/doc/draft-ietf-httpapi-ratelimit-headers/) defines `RateLimit-Policy` and `RateLimit` as work in progress and explicitly avoids mandating a throttling algorithm. Its available quota is not a guarantee that the next request will succeed. If adopting these fields, document the draft version and treat values as client pacing hints, especially when replicas observe lagging state. `Retry-After` should reflect when retry is useful, not merely when a local window resets. Return a stable machine error for rejection and avoid exposing raw partition keys or sensitive tenant data in headers. Clients should use jitter and avoid synchronizing at window boundaries. Server telemetry should track allowed, rejected, allocator failures, local reserve use, lease waste, overshoot estimates, and fairness by bounded cohort. ## Write the partition test first For one limit, fill in purpose, key, window, burst, expected regional topology, maximum overshoot, and behavior when the counter service is slow or split. Simulate two isolated regions spending simultaneously and then reconciling. If the resulting total or user experience is unacceptable, change lease size, ownership, or failure policy before choosing a faster counter implementation. ## Separate enforcement from accounting A low-latency limiter can make an approximate admission decision while a durable ledger records billable consumption, but the two need a reconciliation rule. Never present an approximate counter as an invoice. Test duplicate requests, refunded work, partial completion, delayed ledger events, and a region spending after lease expiry. If quota carries money or entitlement, define which system is authoritative and how customer-visible corrections are made after counters disagree. Reviewed September 2026. ## Source references - -