Event-driven architecture continues in trial with the Cayenne team joining. Our event catalog now documents 40+ domain events with schema validation enforced at publish time. The pattern has enabled several new cross-service features that would have been impractical with synchronous REST calls, such as real-time inventory updates across dealer networks.

Event Infrastructure

Our event backbone runs on Amazon MSK (managed Kafka) with the following setup:

• Schema Registry — Confluent Schema Registry enforces Avro schemas at produce time; incompatible schema changes are rejected before any event is published
• Event Catalog — an AsyncAPI-based catalog auto-generated from our schema definitions, documenting all events with producer/consumer ownership
• Dead Letter Queue — failed events route to a DLQ with automatic retry policies (3 retries with exponential backoff, then alert)

Key Domain Events

Lessons Learned

• Schema evolution is critical — we enforce backward-compatible changes only. Adding optional fields is fine; removing or renaming fields requires a new event version.
• Idempotency by default — every consumer must handle duplicate events gracefully. We use event IDs for deduplication.
• Observability matters more — debugging async flows is harder than REST. We trace events end-to-end using OpenTelemetry correlation IDs, visible in Dash0.
• Start with choreography — we avoided a central orchestrator. Services react to events independently. This keeps coupling low but requires clear event ownership documentation.