How many partitions touched “All orders from today” on the customer ID, and how many on the order date?

For the customer ID all three (fan-out), only one at the order date.

Customer service reads almost only the orders of individual customers, such as K-42. Which key do you choose?

The customer ID: The router finds the partition of K-42 directly, and the query touches only one partition.

Is the order date a good key because the daily report then only touches one partition? How do you weigh up?

The daily report will be cheap, but all write accesses of the day hit a partition (hotspot). Recommendation: Customer ID as a key that serves the daily report via a reading structure.

MechanismChapter II · Data · Building block 6 of 9

Partitioning

Partitioning distributes data sets to separate units using a rule. The key affects load distribution and possible local queries.

By Michael Meierhoff · Meierhoff Systems

Partition keysThe field after which it is distributed.
Fan-outA query must ask all partitions.
HotspotA partition carries almost all the load.

Context

Why this is important

If the order volume grows, a single unit no longer bears the load. Distribution only helps if the key matches the most common query. In English it is called sharding. Identical copies are replication.

This building block explains horizontal data distribution. For identical copies of data, see replication.

At a glance · Data distributed on shards

A router forwards each request to exactly one shard based on the key; each shard only holds a portion of the data.

Shop example · Choose the key

Which key distributes the orders?

Variant: With a strong increase in order volume, the shop distributes orders according to customer ID on three partitions. The remaining modules remain in the modular monolith.

This is how you read the picture

Partition keys The selection of customer ID, order ID or order date.
Fan-out Apricot marked queries touch all three partitions.
Hotspot At the order date, all new orders end up in a partition.

All orders from K-171 of 3 touched partitions · One partition is sufficient
1. Partition A
2. Partition B
3. Partition C
Order B-1042 Reading3 of 3 touched partitions · Fan-out of all partitions
1. Partition A
2. Partition B
3. Partition C
All orders from today3 of 3 touched partitions · Fan-out of all partitions
1. Partition A
2. Partition B
3. Partition C

No key makes all queries cheap. Choose it for the most common query and build a reading structure for rare queries.

Values illustrative

Select a key; for each query, you will see how many partitions it touches (values illustrative).

Benefits & limitations

Helps if:

Partitioning distributes dataset and work as soon as volume or write load exceeds the capacity of a unit.
A stable query key directs requests specifically to the responsible partition.
Range partitioning supports archiving and targeted queries over the chosen range.

Boundaries

An unevenly distributing key creates a hotspot: one partition is overloaded, the others remain unused.
Queries without the partition key must query all partitions. A dedicated read structure can help; see CQRS.
Changing the partition key later requires moving data: this is a Data Migration with its own cost.

In practice

Select a key based on typical accesses and expected distribution. Check which operations target a partition and which require several units.
Simulate load and key distribution with realistic orders of magnitude. Search hotspots and document how new partitions or key changes are migrated.
Measure cross-partition query costs and verify migrations using control values. Replication distributes copies; Performance explains the metrics.

Read the detailed explanation

Partitioning assigns different records to separate data areas or nodes using a key. Hash partitioning distributes keys; range partitioning groups contiguous values. The key therefore affects load distribution and the scope of typical queries.

Requests with a known key can target one partition; other queries need fan-out or additional views. Uneven key distributions create hotspots. Data movement, growth and key changes require controlled migration.

Replication copies data for availability or read capacity; partitioning divides the dataset. Performance helps measure the effect of the key and identify bottlenecks.

Horizontal partitioning is often called Sharding.

Check your understanding

Test yourself in three stages

ReadingCheck in the picture
How many partitions touched “All orders from today” on the customer ID, and how many on the order date?
View the solution with justification
For the customer ID all three (fan-out), only one at the order date.
ApplyingTransferred to a new situation
Customer service reads almost only the orders of individual customers, such as K-42. Which key do you choose?
View the solution with justification
The customer ID: The router finds the partition of K-42 directly, and the query touches only one partition.
EvaluateWeighing an alternative
Is the order date a good key because the daily report then only touches one partition? How do you weigh up?
View the solution with justification
The daily report will be cheap, but all write accesses of the day hit a partition (hotspot). Recommendation: Customer ID as a key that serves the daily report via a reading structure.

Explore further

These building blocks continue the thoughts of this page.

References

Continue on the learning path

Next in Data & state: Caching.

9 Building blocks

Continue withCaching