Software ArchitectureAtlas

Software Architecture Atlas

Gain an overview. Understand the connections.

PatternChapter II · Data · Building block 7 of 9

Caching

A cache holds already calculated or read results for faster follow-ups. Its timeliness rule must fit the business significance of the data.

By Michael Meierhoff · Meierhoff Systems

  1. RuntimeHow long a value can be old.
  2. Consequence of obsolescenceNot every old value hurts the same.
  3. Rule per type of dataEach type of data gets its own duration and reaction.

Context

Why this is important

A cache speeds up accesses, but may show outdated things. The price or stock has different consequences than a description. Permanent storage and their models are their own building block.

This building block explains stored access results and disability. For permanent sources and storage models, see SQL & NoSQL.

At a glance · Cache-Aside

The application asks the cache first; only if it fails, it reads the database and caches the result.
Shop example · Playing through stages

Cache-runtime and obsolescence on the product page

The catalogue shows product texts, prices and available quantities. These values change different times and have different consequences when obsolete.

This is how you read the picture

  1. Runtime You extend the runtime; the values become so old.
  2. Consequence of obsolescence An old text remains green, price and stick color yellow and red earlier.
  3. Rule per type of data With a switch, all three values are green: text 30 min, price disabled, stock short.
  • up to 1 Min oldDescription
  • up to 1 Min oldSales price
  • up to 1 Min oldInventory

Price and stock are up to 1 Min old. This is already risky for the stock.

Values illustrative

Extend the runtime and see which type of data becomes a risk first. Then turn on the rules for each data type.

Benefits & limitations

Helps if:

  • Repeated read accesses to the same values benefit from shorter response times and lower source load through a cache.
  • A defined validity window makes the freshness of derived data controllable.
  • Caching results of expensive calculations free up computing capacity for other work.

Boundaries

  • A cache may return stale state; authoritative decisions about price or stock must consult the source. See Consistency Models.
  • During a cold start, the cache is empty: many simultaneous misses hit the source (cache stampede).
  • The hard part is invalidation: identifying when a cached value must no longer be used.

In practice

  1. Define cache keys and freshness. Specify the lifetime and invalidation path of each derived value. Use a hit only within the access path's freshness guarantee; stricter requirements need suitable validation against the source.
  2. Test expiry, cache misses and complete cache failure under concurrent load. Limit requests to the source so a cold start does not overload it.
  3. Monitor hit rate and data age together, alongside failures and source load. Consistency Models explains visibility guarantees; Performance explains how to measure the speed improvement.
Read the detailed explanation

A cache keeps derived values close to where they are read, for a limited time or until explicit invalidation. Cache-aside loads missing entries from a source. The cache supplies a copy; only the source holds authoritative state, and the copy must meet the freshness guarantee of the access path.

Expiry and invalidation determine freshness and load. A cache outage may direct many simultaneous requests to the source. Hit rate, value age and fallback behaviour reveal whether caching delivers the intended benefit.

Consistency Models defines permitted visibility; a cache adds expiry and invalidation rules. Performance checks whether caching actually reduces the bottleneck.

TTL (Time to Live) defines how long an entry remains valid.

A Cache Stampede occurs when many simultaneous misses overload the source. One remedy is to allow only one refresh per key at a time. The observation that cache invalidation is one of computer science's two hard problems is attributed to Phil Karlton.

Check your understanding

Test yourself in three stages

  1. ReadingCheck in the picture

    At which runtime does the price turn red without a rule, and what does it look like with the rule on?

    View the solution with justification

    Absence 30 The price becomes red (until the price is red). 30 min alt. As a rule, he is disabled in the event of change and remains green.

  2. ApplyingTransferred to a new situation

    The catalogue changes the price of the cup. What needs to be done with the product page cache?

    View the solution with justification

    The cached price will be invalidated. The price comes from the catalogue, and before the checkout it will be re-read anyway.

  3. EvaluateWeighing an alternative

    Should the stock be caged as long as the description? How do you weigh up?

    View the solution with justification

    Longer duration saves accesses. However, an outdated stock can lead to false promises, unlike an older text. Therefore, only briefly display and check binding when booking.

Explore further

These building blocks continue the thoughts of this page.

DataSQL & NoSQLYou choose SQL or NoSQL persistence based on access patterns, consistency and operational requirements.Opening DataConsistency modelsConsistency models define when changes become visible and the order in which they appear.Opening RequirementsPerformancePerformance makes response time, throughput and resource consumption comparable under load.Opening

References

Continue on the learning path

Next in Data & state: Polyglot Persistence.

9 Building blocks

Continue withPolyglot Persistence