Rate Limiting
Individual actors can consume a shared resource budget excessively.
- BudgetBudget: Limit for requests or consumption per interval.
- KeyKey: Policy applies to a user, tenant, or resource.
- ResponseResponse: Rejection or delay makes the limit visible.
Context
Why this is important
Control ingress against a measurable budget.
Control ingress against a measurable budget.
Rate Limiting: inspect the decision
The shop example uses the existing shared database and seven modules. Any added service or deployment model is a hypothetical extension. During a campaign launch, search traffic rises sharply. The shop API permits 20 searches per customer account in ten seconds and maintains a separate aggregate budget for the search index. Excess requests receive a Retry-After value; checkout and payment routes do not share the search budget. Load tests cover bursts and distributed clients, while metrics report throttled calls by route and tenant.
This is how you read the picture
- Budget Budget
- Key Key
- Response Response
| Concept | Meaning | Review question |
|---|---|---|
| Budget | Limit for requests or consumption per interval. | Which budget is limited in the example? |
| Key | Policy applies to a user, tenant, or resource. | What happens with a distributed limit configured only per instance? |
| Response | Rejection or delay makes the limit visible. | Which metrics help tune a limit? |
Benefits & limitations
Helps if:
- Control ingress against a measurable budget.
- Set limits by resource, tenant, and load profile.
Boundaries
- A local limit does not protect against aggregate distributed consumption.
- A per-instance limit multiplies the aggregate budget by the instance count and offers limited tenant protection.
In practice
- Set separate budgets per customer account and for the search index overall; model a distributed client burst that would otherwise stay below local limits.
- Test limits with short bursts and sustained traffic, and verify that rejected search requests receive an appropriate Retry-After value.
- Run order and payment requests under the same load profile and use route metrics to prove the search budget does not consume those core paths.
Read the detailed explanation
Rate limiting bounds how many requests or how much consumption an actor may cause within a time window. A budget can apply per user, tenant, API key, or shared dependency.
A limit protects capacity and allocates scarce resources according to a defined policy. Window, burst allowance, counter consistency, and response behavior shape its effect. A limit is not proof of capacity; it must match measured load and usage goals.
The shop example uses the existing shared database and seven modules. Any added service or deployment model is a hypothetical extension. During a campaign launch, search traffic rises sharply. The shop API permits 20 searches per customer account in ten seconds and maintains a separate aggregate budget for the search index. Excess requests receive a Retry-After value; checkout and payment routes do not share the search budget. Load tests cover bursts and distributed clients, while metrics report throttled calls by route and tenant. rate limit load shedding and load shedding help compare this decision with adjacent concerns.
Check your understanding
Test yourself in three stages
ReadingCheck in the picture
Which budget is limited in the example?
View the solution with justification
Search requests per customer account; checkout routes receive a separate budget.
ApplyingTransferred to a new situation
What happens with a distributed limit configured only per instance?
View the solution with justification
The effective aggregate budget grows with the number of instances.
EvaluateWeighing an alternative
Which metrics help tune a limit?
View the solution with justification
Rejection rate, user impact, resource utilization, and retry share.
Explore further
These building blocks continue the thoughts of this page.