Change Data Capture diagram template
Stream database changes from the transaction log to search, cache and analytics.
About this design
Change data capture turns a database's own commit log into a stream of events, which is more reliable than asking applications to also write to a queue. A connector reads inserts, updates and deletes from the log in order and publishes them to a Kafka topic, without adding load to the tables or touching application code. Downstream consumers subscribe independently: one keeps a search index current, another invalidates cache entries, a third lands the changes in a warehouse for analytics. Because the source of truth stays the database, there is no dual-write inconsistency. The template is a good basis for discussing initial snapshots, schema changes that break consumers, exactly-once expectations versus at-least-once reality, deleting personal data downstream, and the lag a consumer can tolerate before users notice stale results.
Diagram as text
This is the source of the diagram, in the ArchBoard diagram DSL. Paste it into Tools, Diagram from text to rebuild or change it.
title "Change data capture"
direction LR
service app "Application" -> db postgres "Source database"
source-database -[WAL]-> worker cdc "CDC connector" -> topic kafka "Change stream"
change-stream -> worker idx "Index updater" -> search elasticsearch "Search index"
change-stream -> worker inv "Cache invalidator" -> cache redis "Cache"
change-stream -> warehouse snowflake "Warehouse"
