Software ArchitectureAtlas

Software Architecture Atlas

Gain an overview. Understand the connections.

Quality attributeChapter III · Operations · Building block 7 of 10

Observability

Observability uses signals to study the internal state of a running system. Metrics show distributions, logs events, and traces of related execution paths.

By Michael Meierhoff · Meierhoff Systems

  1. MetricsOver time and version.
  2. LogIndividual event with identifier.
  3. Tracetime portions of an expiration.

Context

Why this is important

After a release, orders take longer, although no alarm reports an error. Without metrics, logs and traces with a common context, the cause remains guesswork. SLO and deployment deepen their own building blocks.

This building block explains signal types and investigation questions. SLO defines target values on selected metrics; deployment provides version context.

At a glance · Telemetry from service to evaluation

The service sends logs, metrics and traces; the evaluation turns them into dashboards and alarms.
Shop example · Signal for the question

Which signal answers the question?

Orders take longer after a release. A trace links shopping cart, payment, and shipping query; metrics show the latency increase by version.

This is how you read the picture

  1. Metrics Signal 'metric' and the questions 'Since when are response times rising?' and error frequency.
  2. Log Signal 'log', matching the question 'Which order was affected by the error?'
  3. Trace Signal 'trace', matching the question about the dependency that delays the order flow.
  • MetricsCourse of a number over time and versionp95 latency per version: rise directly after the release
  • LogSingle event with order identifier
  • TraceTime shares of a flow across all calls

A trend over time and version shows the start and the release time.

For each operations question, choose the signal that answers it most directly.

Benefits & limitations

Helps if:

  • Distributed execution paths benefit from signals that assign delays and errors to a responsible dependency.
  • Comparable measurements and inspectable individual cases make operational targets and version effects traceable.

Boundaries

  • Sampling can hide rare events; add robust error counters and targeted retention for relevant cases.
  • Unlimited metric dimensions and sensitive content increase costs and privacy risks; limit labels, fields and access rights.
  • Signals without a goal remain data; an SLO defines which measurement triggers action.

In practice

  1. Define investigation questions. State which operational decisions the signals should support and which areas must be visible. Link each question to a metric, event or execution path. This produces targeted instrumentation requirements.
  2. Define signal types and context. Use metrics for rates and distributions, logs for individual events and traces for execution flows. Define shared identifiers and restrict highly variable values to suitable signal types. This connects observations without unnecessarily inflating metrics.
  3. Review use, cost and protection. Define sampling, retention, access and filtering of sensitive content. Rehearse an investigation with representative operational data and check whether the question can be answered. This reveals both information value and ongoing effort.
Read the detailed explanation

Observability uses metrics, structured logs and traces to answer different questions about a running system. Metrics show frequencies and distributions, logs describe individual events, and traces connect execution paths. Shared context identifiers link signals across process and service boundaries. They provide measurement data for SLIs, SLOs & Error Budgets and help assess the effects of Deployment & Rollout.

Instrumentation starts with investigation questions and selects suitable signals. Unbounded label values increase costs, while sampling can hide rare operations. Structured fields, defined retention and privacy boundaries therefore belong in the design. Observability comes from signals that let a team narrow down effects and identify the next checks. Large signal volumes without this investigative value mainly add cost.

RED (Rate, Errors, Duration) and USE (Utilization, Saturation, Errors) are common metric sets; OpenTelemetry is an open standard for traces, metrics and logs.

Check your understanding

Test yourself in three stages

  1. ReadingCheck in the picture

    Which signal in the visual answers 'Which order was affected by the error?'

    View the solution with justification

    The log: a single event with an order identifier, such as B-1042, names the order.

  2. ApplyingTransferred to a new situation

    After a release, orders take longer. Which signals do you choose first, and then which?

    View the solution with justification

    First the metric, to compare the start of the rise with the release. Then the trace, to find the delaying dependency.

  3. EvaluateWeighing an alternative

    The team wants to use only logs for every question. What speaks for it, what speaks against it, and what do you recommend?

    View the solution with justification

    Advantage: one tool, and single errors are easy to find. Disadvantage: the trend per version and the time shares per call are tedious to reconstruct. Recommendation: connect metric, log and trace and link them via identifiers.

Explore further

These building blocks continue the thoughts of this page.

OperationsSLIs, SLOs & error budgetsSLIs, SLOs and error budgets connect measurable operational quality with targets and agreed deviations.Opening OperationsDeployment & rolloutDeployment and rollout distribute software changes to the runtime environment under control.Opening OperationsRetry StormA retry storm occurs when many retries continue to overload an already strained dependency.Opening

References

Continue on the learning path

Next in Runtime & operations: SLIs, SLOs & error budgets.

10 Building blocks

Continue withSLIs, SLOs & error budgets