Design a Rate Limiter Service

By the Syncfusion DevTutorials team · Last updated October 20, 2026

Turn one server's rate limiter into a shared service: local vs shared counters, atomic increments, sliding windows, 429 and Retry-After, and failing open.

At 11 a.m. on Diwali, QuickBite launches a ₹200 coupon. Within minutes, a few hundred accounts begin hitting the coupon API thousands of times every second. They are scripts, not real customers. The Order API starts slowing down, and customers trying to place lunch orders begin seeing errors.

In Lesson 10, you built a rate limiter inside a single server. That protects one server. QuickBite now runs multiple API gateways behind a load balancer, so a limiter stored inside one gateway is no longer enough.

This case study turns the limiter into a shared service that every gateway uses.

The main idea: count every caller's requests in one shared place, check limits in a few milliseconds, and make sure the limiter itself never becomes the reason the site goes down.

What You Will Learn

In this case study, you will:

1. Write the requirements for a rate limiter service.
2. Estimate request volume and memory requirements.
3. Understand why local counters fail behind a load balancer.
4. Store counters in a shared in-memory store.
5. Use atomic operations to avoid race conditions.
6. Compare fail-open and fail-closed designs.
7. Return proper rate-limit responses using 429 and Retry-After.
8. Build and test a distributed rate limiter.

Step 1: Requirements

What It Must Do (Functional)

• Limit each caller: allow at most 100 requests per minute per user.
• Protect sensitive actions: allow only 10 coupon claims per minute.
• Return clear errors: limited requests receive 429 Too Many Requests.
• Include Retry-After: clients should know when they can safely retry.
• Support configuration changes: limits can be updated without redeploying the system.
• Support multiple policies: different APIs can have different limits.

How Well It Must Do It (Non-Functional)

• Fast: every limit check should add only a few milliseconds.
• Accurate: limits should work correctly no matter which gateway handles the request.
• Scalable: the limiter must continue working as traffic grows.
• Reliable: limiter failures should not create a site-wide outage.
• Highly available: the limiter must not become a single point of failure.

Out of Scope

This lesson does not cover:

• Fraud detection
• CAPTCHA systems
• IP reputation systems
• Advanced bot detection

These systems often work alongside rate limiting but are not part of the limiter itself.

Step 2: Estimate

Checks per Second

At the Diwali peak, QuickBite expects:

• 12,000 customer requests per second
• 6,000 bot requests per second

Every request must pass through the limiter, even requests that will eventually be rejected.

What to Notice

• Every request requires a limit check.
• Bots add significant extra traffic.
• Together, customer and bot traffic overload the Order API.
• Blocking bad traffic early is much cheaper than adding more Order API servers.

Memory

A rate-limit counter is very small.

Example key: limit:user:8812:min:2026-10-14T11:00

Assume:

• 2 million active users
• 2 counters per user
• 100 bytes per counter

What to Notice

• Counter data is tiny.
• Memory requirements stay manageable.
• A shared in-memory store can easily hold all active counters.
• Expiring old counters prevents memory growth.

Step 3: Why Local Counters Fail

A common beginner design stores counters inside each API gateway.

This works perfectly when only one gateway exists.

The problem appears when requests are spread across multiple gateways by a load balancer.

Imagine:

• Limit = 100 requests per minute
• 5 gateways behind a load balancer
• A bot sends 1,000 requests

The load balancer spreads traffic across all gateways.

Each gateway sees only part of the traffic. Step through what happens:

What to Notice

• Local counters become weaker as more gateways are added.
• The bot effectively receives a separate limit on each gateway.
• Adding more gateways actually weakens enforcement.
• Limits must be enforced from one shared location.

Step 4: A Shared Limiter Store

The Design

Store counters inside a shared in-memory store such as Redis.

Every gateway checks the same counter.

This guarantees that limits are enforced consistently regardless of which gateway receives the request.

A single limiter node can handle far more than the required 18,000 checks per second.

Keep the Check Atomic

Now consider two gateways processing requests for the same user at the same moment. The count is 99 and the limit is 100.

If both gateways read the counter, add one and save the value, they may both read 99 before either saves 100. Both requests are accepted. This is a race condition.

The solution is an atomic increment. Redis provides the INCR command, which adds one and returns the new value as a single operation. Step through both versions:

Here is the same idea in code:

A fixed-window limiter built on one atomic increment

const store = new Map(); // stands in for the shared in-memory store

// INCR: add 1 and return the new value, as one step.
function incr(key) {
  const next = (store.get(key) ?? 0) + 1;
  store.set(key, next);
  return next;
}

function allow(userId, minute, limit) {
  return incr(`limit:${userId}:${minute}`) <= limit;
}

let allowed = 0;
for (let i = 0; i < 12; i++) if (allow("u8812", "11:00", 10)) allowed++;
console.log(`11:00 -> allowed ${allowed} of 12`);
console.log(`11:01 -> first request allowed? ${allow("u8812", "11:01", 10)}`);

Expected output:

11:00 -> allowed 10 of 12
11:01 -> first request allowed? true

A fixed-window limiter is just one INCR per request on a key such as limit:u8812:11:00: allow the request if the returned value is at or under the limit. When the minute changes, a new key starts from zero.

Fixed Window vs Sliding Window

A fixed window has a weakness. A user can send a burst at 11:00:59 and immediately send another burst at 11:01:00. Both bursts fit inside different windows, so the limiter allows more traffic than intended.

A sliding window counter reduces this problem by counting part of the previous window as well:

Here is the same idea in code:

Sliding window: count part of the previous minute too

const limit = 10;
const previousMinute = 10; // requests in 11:00
const currentMinute = 4;   // requests so far in 11:01
const secondsIntoMinute = 15;

const weight = (60 - secondsIntoMinute) / 60; // how much of last minute still counts
const estimate = currentMinute + previousMinute * weight;
console.log(`Estimated requests in the last 60 s: ${estimate.toFixed(1)}`);
console.log(`Allowed? ${estimate < limit}`);

Expected output:

Estimated requests in the last 60 s: 11.5
Allowed? false

A fixed window would allow the request, because only four requests happened during the current minute. The sliding estimate (4 + 10 × 0.75 = 11.5) notices the burst from the previous minute and blocks it.

Telling Callers They Are Limited

When a caller exceeds the limit:

• Return 429 Too Many Requests
• Include a Retry-After header

This tells well-behaved clients exactly how long to wait before trying again.

Step 5: When the Limiter Breaks

Every request now depends on the limiter.

What should happen if the limiter store becomes unavailable?

Fail Closed

If the limiter cannot check the counter, it rejects the request.

This protects backend services.

However, every customer becomes blocked if the limiter fails.

The limiter becomes a single point of failure.

Fail Open

If the limiter cannot check the counter, it allows the request.

Customers can continue placing orders.

For a short period, some bot traffic may get through.

For customer-facing systems such as QuickBite, this is usually the better choice.

QuickBite uses:

• Fail-open behavior
• Replica limiter nodes
• Short timeouts

If the limiter becomes slow, gateways treat it as unavailable rather than making every request wait.

Step 6: Build It on the Canvas and Test It

Open the design sandbox.

The starting design contains:

• 1 load balancer
• 3 API gateways configured to fail closed
• The Order API
• No shared limiter store

The simulator generates:

• 12,000 customer requests per second
• 6,000 bot requests per second
• Infrastructure failures

What You Will Probably Find

• Three gateways are not enough. They run at about 121% utilization and p99 reaches roughly 839 ms.
• Without a shared limiter store, the Order API drowns. All 18,000 requests reach it and push it beyond capacity.
• One limiter node is a trap. If it fails, fail-open allows all traffic through while fail-closed blocks customers.

• A passing design uses:

  • 2 load balancers
  • 5 API gateways
  • 2 limiter nodes
  • Shared counters
  • Fail-open behavior

This design costs about $740 per month, keeps p99 latency around 23 ms, and blocks most bot traffic before it reaches the Order API.

Common Mistakes

Practice

What to Remember

• A limiter behind a load balancer must count requests in one shared place.
• Local counters become weaker as more gateways are added.
• Use atomic increments to prevent race conditions.
• Sliding window counters reduce burst problems at window boundaries.
• Reject requests early using 429 Too Many Requests and Retry-After.
• Fail open for customer-facing systems.
• Add replicas and timeouts so the limiter does not become the cause of an outage.
• Test every design against peak load, machine failures, and cost constraints.

Further Reading

These are the sources this lesson was checked against:

• Stripe: Rate limits: how a large API limits callers and what it tells them.
• Redis: INCR: the atomic counter command, including the rate limiter pattern.
• RFC 6585, section 4: 429 Too Many Requests: the status code and Retry-After.
• Cloudflare: How we built rate limiting capable of scaling to millions of domains: the sliding window counter at very large scale.

Next: News Feed, where QuickBite builds a personalized feed of offers and new dishes from restaurants that customers follow.