Decision spaceChapter II · Communication · Building block 4 of 9

Queues, Pub/Sub & Streams

Queues distribute work, pub/sub distributes a message to multiple subscribers, and streams keep an orderly sequence readable. Delivery and storage rules characterize processing.

  1. QueueDistributes work to a worker.
  2. Pub/SubEvery subscriber receives the message.
  3. StreamOrdered sequence rereadable.

Context

Why this is important

Not every message needs the same distribution. Those who choose the form incorrectly lose work, reach too few recipients or cannot read again. Technical events deal with event-driven architecture.

This building block compares message transports. For business events and reactions, see Event-driven Architecture.

At a glance · One topic, many subscribers

The publisher publishes a message in the topic; each subscriber receives it and processes it independently.
Shop example · Playing through scenarios

Who gets the message?

The shop processes invoice orders, distributes orderPaid for shipping and analysis and keeps order status changes for later evaluation.

This is how you read the picture

  1. Queue In Queue, only invoice is green: a worker gets the task “create invoice”.
  2. Pub/Sub At Pub/Sub, shipping and analysis are green and receive orderPaid once each.
  3. Stream At Stream, shipping and analysis read the status changes from their own location.
Message format
  • SendOrdersets up the task “Create invoice”
  • ReceiptInvoiceA worker gets the job
  • UninvolvedShippingreceives nothing
  • UninvolvedAnalysisreceives nothing

One worker per orderConsignee

The queue distributes work. Exactly one worker handles every invoice order, re-reading is not the purpose.

Choose the message form. You can see which recipients receive the message and whether it remains readable.

Benefits & limitations

Helps if:

  • Queue, pub/sub and stream arrange messages according to the recipient group and desired storage.
  • The appropriate messaging model lets each recipient resume from its own progress or replay the history it needs.
  • A durably retained stream makes earlier events available to new consumers.

Boundaries

  • The distribution model determines recipients and replay capabilities; each consumer must verify its guarantees.
  • Transport acknowledgement and business processing are distinct states; Delivery Guarantees covers the transport.
  • Delay and backlog remain consequences of temporal decoupling; see Asynchronous communication.

In practice

  1. Choose a Queue, Pub/Sub or Stream according to recipients, retention and replay needs. Record who owns the message and processing progress.
  2. A joint transport gives different business processing operations their own status. Save progress per consumer and link it to the delivery identifier.
  3. Test retention and catch-up behaviour under backlog. Asynchronous Communication explains temporal decoupling; Delivery Guarantees specifies redelivery.
Read the detailed explanation

Queues, Pub/Sub and Streams distribute messages with different rules for recipients, retention and replay. A queue distributes pending work; Pub/Sub gives independent subscriptions their own deliveries; a stream retains ordered entries for later reading.

The choice determines who stores progress, how long data remains available and whether new consumers can read the history. Delivery guarantees and business effects remain separate commitments.

Asynchronous Communication explains temporal decoupling; Delivery Guarantees specifies loss, redelivery and acknowledgement for the chosen transport.

Kafka is a widely used streaming platform. A Consumer Group shares a topic's messages among its members.

Check your understanding

Test yourself in three stages

  1. ReadingCheck in the picture

    Who receives orderPaid at Pub/Sub?

    View the solution with justification

    Dispatch and analysis, once each.

  2. ApplyingTransferred to a new situation

    The shop wants to evaluate order status changes later again. What form do you choose?

    View the solution with justification

    The stream. He considers the episode to be a fixed duration, and readers have positions of their own.

  3. EvaluateWeighing an alternative

    Should orderPaid run over a queue instead of pub/sub? How do you weigh up?

    View the solution with justification

    A queue distributes work to workers, each message goes to one. However, shipping and analysis should receive the event independently once, pub/sub provides.

Explore further

These optional deepenings complement the learning goal of this page.

Priority Queue and Competing Consumers

A Priority Queue can place urgent work ahead of normal work; Competing Consumers distribute pending messages across workers. Prioritisation serves urgent work sooner but may starve normal messages when urgent traffic remains high. Fairness can be provided through separate queues, reserved capacity or limits on consecutive priority messages. Workers may receive messages again, so processing must handle duplicates.

For testingTest priority and fairness. Divide shop tasks into urgent and normal work, simulate competing workers and a sustained stream of urgent messages. Define a fair scheduling rule and verify that redelivery of the same message does not duplicate its business effect.