Noisy Neighbours and Fairness
By the Syncfusion DevTutorials team · Last updated
One tenant's huge export spikes everyone's p99. Protect shared capacity with per-tenant quotas, fair queues and a dedicated cell for the whale tenant.
One Tenant Slows Everyone Down
It is Monday morning. Contoso, Northwind's biggest tenant, starts its weekly export: 6,000 batches of tickets. The export goes onto the same worker pool that serves every tenant's small requests.
Within seconds, 49 other companies see their help desk freeze. Nothing is broken. Contoso is simply first in line, 6,000 times.
This is the noisy neighbour problem. Tenant isolation is not only about data (Lesson 2). It is also about capacity: one tenant must not be able to use up what everyone shares.
With one shared first-come-first-served queue, a big job puts every small request behind it. Fair queues, quotas and a cell keep small requests fast.
The main idea: measure latency per tenant, cap how much of the shared pool one tenant can hold, serve tenants in turn instead of in arrival order, and give the biggest tenants capacity of their own.
TL;DR
- Shared capacity (workers, connections, CPU) needs per-tenant limits, or the biggest tenant sets everyone's latency.
- Quotas cap how many workers, requests or rows one tenant may use at once.
- A fair queue keeps one queue per tenant and takes work from them in turn, so a tenant with 6,000 tasks waits behind a tenant with one.
- A cell is a full copy of the stack serving a fixed group of tenants. A whale in its own cell can only slow itself down.
- Watch per-tenant p99. A global average hides one tenant's bad day, and a whale's traffic hides everyone else's.
A Real-World Example
A reporting SaaS lets customers schedule exports. Most exports are tiny. One customer schedules a full-history export for every one of its 300 teams at 9:00 every Monday. For 40 minutes every Monday, every other customer's dashboards time out.
The team tries a bigger worker pool. The export simply finishes faster and still blocks everyone. What fixes it is fairness, not more capacity.
What You Will Learn
- Why one shared first-come-first-served queue is unfair.
- How per-tenant quotas work.
- How a fair queue works.
- When to move a tenant into a dedicated cell.
- How to measure fairness.
First Come, First Served Is Not Fair
A single queue is fair to tasks, not to tenants. If one tenant adds 6,000 tasks, everyone else waits behind all of them.
Fix 1: Per-Tenant Quotas
A quota caps one tenant's share: at most N workers, N requests a second, N open connections. In Lesson 10 of HLD Foundations you met rate limiting per user. The same tools work per tenant.
A quota alone keeps a few workers free, but the shared queue is still first come, first served. Other tenants' tasks still sit behind Contoso's in the line, waiting for one of the free workers.
Fix 2: A Fair Queue
A fair queue keeps one queue per tenant. A worker that becomes free takes the next task from the next tenant in turn (round-robin). A tenant with 6,000 tasks gets one turn per round, like everyone else.
t07 and t12 are served in the first round, even though Contoso queued first. Weighted versions give paying tiers more turns per round.
Fair queueing is old: network routers have used it for decades, so one heavy flow cannot starve others. Cloud services use the same idea, often called "shuffle sharding" or "per-tenant queues", for the same reason.
Fix 3: A Cell for the Whale
A cell is a complete, independent copy of the stack (gateway, workers, databases) that serves a fixed set of tenants. Cells limit the blast radius: a bad deploy, a hot tenant or a failed database affects only that cell's tenants.
The biggest tenants often get a cell of their own. Contoso can then run any export it likes: it only competes with itself, and you can size its cell (and its bill) to its contract.
A cell per tenant brings back the silo's costs from Lesson 1: more infrastructure, more deploys, more monitoring. Give cells to the tenants whose size or contract pays for them, and keep everyone else in shared cells.
Try It: Contoso's Monday Export
The lab runs three minutes of traffic on 8 shared workers. 49 tenants send 300 small requests a second. At 20 s Contoso submits 6,000 export batches. The SLO for everyone else is p99 ≤ 250 ms. Contoso's contract says its export must be done by 100 s.
- Run with nothing on. Watch the bars after 20 s.
- Add only a quota. Then only a fair queue. Which one fixes other tenants' latency? What happens to Contoso's export?
- Give Contoso a cell. Then combine all three.
Pass rule: other tenants' p99 ≤ 250 ms, and Contoso's export finished by 100 s.
Measure Fairness Per Tenant
A global p99 can look fine while one tenant suffers, and the whale's huge volume can drown everyone else's numbers. Tag every metric with the tenant, and alert on the worst tenant's p99.
Watch out for the cardinality of per-tenant metrics. With thousands of tenants, keep per-tenant labels on a few key metrics (latency, errors, throttles) and report the top-N worst tenants instead of a series for every tenant.
Check Yourself
Common mistakes
| Mistake | Wrong | Correct | Why it happens |
|---|---|---|---|
| More capacity as the fix | Double the worker pool | Fair queues and quotas | A bigger pool shortens the incident but keeps it unfair |
| Quota only | Cap the whale, keep one FIFO queue | Quota plus a fair queue | Other tasks still wait behind the whale's queued work |
| Cell for everyone | A dedicated stack per tenant | Cells for whales, shared cells for the rest | Isolation feels safe; the cost shows up later |
| Global metrics | One p99 for the whole system | Per-tenant p99 and worst-tenant alerts | The biggest tenant dominates every average |
| Hidden throttling | Slow responses with no explanation | Clear limits and 429 with Retry-After | Customers cannot plan around limits they cannot see |
Practice
What to Remember
- Shared capacity needs per-tenant limits, or the biggest tenant sets everyone's latency.
- Quotas cap one tenant's share; a fair queue serves tenants in turn.
- A cell gives a whale its own stack and limits the blast radius of any failure.
- More capacity alone does not fix unfairness.
- Measure and alert per tenant, on the worst tenant.
Further Reading
- AWS Builders' Library: Fairness in multi-tenant systems: quotas, fair queueing and load shedding per tenant.
- AWS Well-Architected: Cell-based architecture: cells and blast radius.
- AWS Builders' Library: Workload isolation using shuffle-sharding: limiting how many customers one bad tenant can affect.
Next: Custom Fields and Objects. Every customer wants different fields on a ticket. You store them without a schema change for each tenant, and measure three designs in real SQL.