Software ArchitectureAtlas

Software Architecture Atlas

Gain an overview. Understand the connections.

ActivityChapter III · Operations · Building block 10 of 10

Recovery

Recovery returns data and functions to a working state after a failure. A backup is only practically usable when restore and result checking are practiced.

By Michael Meierhoff · Meierhoff Systems

  1. RPOAllowed data loss in minutes.
  2. RTOAllowed downtime to operation.
  3. Backup clockThe denser, the less loss.

Context

Why this is important

A backup is only useful if a workable stand can be produced from it. Time of backup and way of recovery determine how much data is missing and how long the shop is standing. Deepening deployment and risk log.

This building block deals with restart and restoration proof. Deployment controls versions; data migration ensures schema transitions.

At a glance · From failure to verified recovery

A backup only helps once restore and result checks have been practised; RPO and RTO set the limits.
Shop example · Failure and recovery

Data loss and downtime against targets

An error damages the order index. The team restores a secure stand in an isolated environment and compares orders with the payment system.

This is how you read the picture

  1. RPO The “Data Loss (RPO)” line shows the time since the last backup against the target.
  2. RTO The “Downtime (RTO)” line shows the duration of the selected recovery path.
  3. Backup clock ‘Daily backup’ in case of failure: 14:30 more loss than “hourly backup”.
Data Loss (RPO)
14 h 30 min min Order data
Target 1 h · Missing target
Downtime (RTO)
4 h
Target 4 h · Objective met

At least one goal is missed: Safer more often or choose a faster recovery route.

Times and goals illustrative

Move the time of failure and select Backup and Restore; check if RPO and RTO are reached.

Benefits & limitations

Helps if:

  • Critical persistent data requires a measured recovery within agreed time and loss limits.
  • Corrupted data or operator mistakes can be reversed using a validated recovery point.

Boundaries

  • Replication also propagates accidental changes; combine it with retained recovery points and test targeted restoration.
  • A restore rehearsal proves only the process tested; document the recovered data state, duration and deviations from agreed targets.
  • Technical restoration alone is insufficient; business reconciliation of orders and payments belongs in the Risk log and the recovery process.

In practice

  1. Derive targets from impact. Agree tolerable data loss and recovery duration with the service owners. Map different stores and functions to their respective consequences. This produces RPO and RTO values justified by business needs.
  2. Safeguard the recovery chain. Check backups, change logs, keys, configuration, versions and required dependencies. Make sure the runbook clearly documents the sequence and permissions. This ensures the material for a complete recovery is available.
  3. Perform and measure a restore rehearsal. Restore data in isolation, validate business consistency and measure duration and data loss. Compare both measurements with agreed targets and assign ownership of deviations. This provides an evidenced status and concrete improvement needs.
Read the detailed explanation

Recovery restores data, dependencies and operational capability. The Recovery Point Objective (RPO) describes tolerable data loss measured in time; the Recovery Time Objective (RTO) sets the target recovery duration. Point-in-time recovery requires a base backup and an uninterrupted sequence of archived changes. Recovery complements the failure rules in Availability and must fit regular Deployment & Rollout.

Starting a data store does not by itself make a service usable for business. Configuration, keys, access rights, dependent systems and consistency checks all belong in recovery. An isolated rehearsal measures actual data loss and duration without changing production data. Runbooks state the sequence and responsibilities. Repeated rehearsals support realistic targets and targeted removal of remaining bottlenecks.

Check your understanding

Test yourself in three stages

  1. ReadingCheck in the picture

    CET capital 14:30, hourly backup, restore from backup. How big is data loss and downtime, and are the goals met?

    View the solution with justification

    Data loss 30 Minutes (backup around) 14:00, downtime 240 Minutes. Both goals (RPO) 60, RTO 240) are fulfilled.

  2. ApplyingTransferred to a new situation

    CET capital 09:00, daily backup, switching to standby. Which goal is missed, and what do you change?

    View the solution with justification

    The RPO: 540 Minutes since the backup 00:00 at destination 60. The downtime of 15 Minutes fit. Change to hourly backup or log backup.

  3. EvaluateWeighing an alternative

    Standby instead of restore from backup: What speaks for it, what against it, what do you recommend?

    View the solution with justification

    For this purpose: 15 instead of 240 Minutes of downtime, significantly more reserve to the target. On the other hand: additional operating expenses, and a standby can take on accidental changes. Recommendation: Restore is just enough for RTO 240; for reserve supplement standby and keep restore points.

Explore further

These building blocks continue the thoughts of this page.

OperationsDeployment & rolloutDeployment and rollout distribute software changes to the runtime environment under control.Opening DocumentationCommunicate risksA risk log records uncertainties, possible consequences, owners and next validation steps.Opening OperationsObservabilityObservability combines signals that let you inspect a system's internal state.Opening

References

Continue on the learning path

With that, you have Runtime & operations run through. It continues with Security & risks.

10 Building blocks

Next areaSecurity & risks