Microservices · Editable reference architecture

Kafka Event-Driven Microservices Architecture

Map how an order service publishes events and independent consumers handle fulfillment and notifications. Use the Kafka diagram to discuss delivery guarantees, consumer ownership and recovery when processing fails.

Open & Edit in DrawArc (Free, No Login Required)

Your changes stay in a separate draft on this browser. Download an editable project to keep a portable copy.

Read the step-by-step architecture tutorial →

LinkedIn X (Twitter)WhatsApp

6 components · 6 connections · Open the editor to zoom, move nodes and edit source.

Architectural overview

Event-driven microservices communicate through durable records rather than requiring every downstream service to respond during a user request. This starter shows an order API writing its database and publishing to Kafka, where fulfillment and notification consumers process events independently. A dead-letter topic makes the failed-processing path visible instead of hiding it inside implementation details.

The database write and event publication are separate arrows because their consistency must be designed explicitly. An outbox with change-data capture is one possible extension; two independent writes do not become atomic just because they appear next to each other. Choose event keys and partitions around the ordering guarantees your domain requires, and make consumer handlers idempotent before enabling retries or replay.

During design review, record event schemas, retention periods, consumer lag targets and the team that owns each topic. Explain how incompatible schema changes are rolled out and how failed messages are inspected without leaking sensitive payloads. Extend the diagram with observability, access controls and a documented replay process so operational concerns remain part of the same conversation as the happy path.

The real-world problem

An order API must accept orders without waiting for fulfilment and notification systems. Independent consumers need durable events and an explicit path for records that repeatedly fail.

Data flow

Connections from this diagram. Branches and control signals are not a single chronological sequence.

Order API → Order database: Persist order.
Order API → Kafka order events: Publish via consistency strategy.
Kafka order events → Fulfillment consumer: Consumer group A.
Kafka order events → Notification consumer: Consumer group B.
Fulfillment consumer → Dead-letter topic: After bounded retries.
Notification consumer → Dead-letter topic: After bounded retries.

Security boundaries

Limit topic produce/consume permissions by service identity and encrypt broker connections.
Minimize personal data in events and restrict dead-letter inspection and replay access.
Validate event schemas and tenant ownership in consumers; broker access alone does not authorize a business action.

Failure handling

Order commits but event publication fails

Response: Implement an outbox/relay or another verified consistency strategy; the two API arrows are not an atomic transaction.

Verify: Crash between commit and publish, then verify eventual publication without losing the order.

Consumer repeats a side effect

Response: Make handlers idempotent using event IDs or business constraints; commit progress at the chosen transaction boundary.

Verify: Replay an event after a simulated crash and verify a single business outcome.

A poison event blocks processing

Response: Bound retries and route to the dead-letter topic with an owner and controlled replay procedure. Decide how this affects ordering.

Verify: Inject an invalid event and verify alerting, isolation and replay after correction.

Trade-offs and alternatives

Asynchronous consumers

Benefit: Independent processing and recovery schedules.
Cost: Eventual consistency, lag and more operational components.
Alternative: Keep a synchronous call when the caller needs an immediate authoritative outcome.

Partition by order ID

Benefit: Related events can retain partition ordering.
Cost: Hot keys limit parallelism; topic-wide ordering is not supplied.
Alternative: Choose a different key only after defining which ordering guarantees can change.

When to use this architecture

Decouple order processing from notifications or other integrations with different availability requirements.
Discuss partition ordering, replay and idempotency with teams that own independent consumers.
Document asynchronous dependencies and failure handling before migrating a synchronous workflow to events.

Key components

Order API: Validates commands and owns the order lifecycle.
Order database: Stores accepted orders; coordinate event publication with a consistent outbox strategy.
Kafka order events: Retains partitioned events for independent consumer groups.
Fulfillment consumer: Processes orders idempotently and tracks its own progress.
Notification consumer: Sends customer updates without blocking the order API.
Dead-letter topic: Retains repeatedly failing records for controlled inspection and replay.

Questions about this template

Does Kafka guarantee exactly-once business effects?

Not by itself. External database writes and notifications still need explicit idempotency and transaction boundaries.

Where should I add an outbox?

Add an outbox beside the order database and connect a relay or CDC service to Kafka to show your chosen consistency strategy.

Related architecture templates

AWS