ActivityChapter III · Evolution · Building block 7 of 7

Data migration

Data migration changes the schema or data stock, while software versions and accesses can exist in parallel. Extension, filling and later conversion arrange this transition.

  1. Step by stepSmall compatible changes instead of a big jump.
  2. Parallel versionsOld and new run at the same time.
  3. Mandatory too earlyA mandatory column before filling breaks versions.

Context

Why this is important

During a rollout, old and new versions run simultaneously. If the new version assumes a field that does not write the old one, gaps or failures occur. Small compatible steps avoid this.

This building block deals with data and schema transitions. Deployment coordinates versions; Recovery checks recovery of a data status.

At a glance · Migration in four phases

Compatible steps expand first, switch later and clean up last, while old and new versions run side by side.
Shop example · Compatibility per phase

Invoice ID: Which app version runs in which phase

The shop adds an invoice identifier for orders. First, the column is optionally created, then filled by the application and only later mandatory read.

This is how you read the picture

  1. Step by step You go through four phases; in the phases 1 up to 3 Both versions run without errors.
  2. Parallel versions The lines old and new app version show who is running in which phase.
  3. Mandatory too early In the “duty too early” phase, all three lines are red.

The invoice identifier column shall be created as an option.

  • Old app versionrunning – ignores the new column
  • New App Versionrunning – can write the identifier
  • Rollback to old versionrunning – possible, scheme remains compatible
Go through the phases. Only “duty too early” shows what breaks if you make the column binding before filling.

Benefits & limitations

Helps if:

  • Large datasets benefit from controllable partial transfers with visible progress.
  • Compatible schema changes keep old and new software versions operational during continuous delivery.

Boundaries

  • Dual-write paths diverge; reliable propagation and reconciliation are needed before switchover.
  • Destructive steps such as dropping a column close off rollback; verify all consumers and recovery options beforehand.
  • Large tables make backfills take longer; with a shared table (Shared Database), the complete set of readers may also be unknown.

In practice

  1. Design a compatible extension. Add new structures so active versions can keep working. Define how values are initially written and read. This creates a schema that supports the planned parallel operation.
  2. Control the backfill. Bound batch size and load, store progress and define resumption after interruption. Handle concurrent changes transparently. This keeps progress and production impact visible.
  3. Verify completeness and plan the contract phase. Reconcile data against business rules, check readers and monitor access to the old form. Then decide when to remove the old path. This provides a verifiable transition with a clear cleanup boundary.
Read the detailed explanation

A data migration changes stored values, their structure or their location. During continuous operation, old and new software versions may run in parallel. Expand-Migrate-Contract first adds the new structure, transfers existing data in a controlled way, switches consumers, then removes the old form. It relates to SQL & NoSQL and compatible Deployment & Rollout.

Backfills need bounded batches, progress markers, restartability and rules for concurrent changes. Counts, samples and business constraints verify the transfer. Dual writes can diverge and require atomic coupling or reliable propagation. The contract phase follows evidence that active readers no longer use the old form.

Check your understanding

Test yourself in three stages

  1. ReadingCheck in the picture

    What is red in the “duty too early” phase?

    View the solution with justification

    All three lines: The old version cannot save, the new one reads empty identifiers, a rollback is impossible.

  2. ApplyingTransferred to a new situation

    An old worker still generates orders without an invoice identifier. Can you create the column as a duty?

    View the solution with justification

    No. First, create it as an option, update all the writing paths and fill the stick. It will only become binding in step 4.

  3. EvaluateWeighing an alternative

    The team is considering doing column, filling, and duty in a single deployment. How do you rate that?

    View the solution with justification

    A deployment is faster and has fewer intermediates. In parallel versions, however, it risks errors and leaves no safe way back, while the individual steps remain compatible.

Continue on the learning path

With that, you have Evolution & assessment run through. It continues with Documentation & communication.