How to Build Customer-Facing Analytics Pipelines with CDC
A practical architecture guide to customer-facing analytics with CDC, covering data freshness, serving layers, tenant isolation, and real-world SaaS examples.
13 of 800 posts
A practical architecture guide to customer-facing analytics with CDC, covering data freshness, serving layers, tenant isolation, and real-world SaaS examples.
Move NetSuite data to ClickHouse with SuiteAnalytics Connect, an ELT tool or a script. Compare 5 methods and plan for deletes, custom fields and limits.
Stream Kafka to ClickHouse with the Kafka table engine, ClickPipes or Kafka Connect. Compare 4 methods, delivery guarantees and duplicate handling.
Move Salesforce to S3 as Parquet with Amazon AppFlow, AWS Glue, Python or a managed pipeline. Compare how each handles types, formula fields and deletes.
Salesforce to Iceberg: compare 4 ways to load CRM objects into Iceberg tables, see where Data 360 zero copy fits, and handle deletes and formulas.
Move data from Oracle to ClickHouse: map types, handle updates and deletes with ReplacingMergeTree, and compare 4 CDC and batch methods.
Move S3 data into Kafka with an S3 source connector, AWS Lambda, Spark or a managed pipeline. Compare file ordering, formats, duplicates and setup.
Send webhooks to Kafka with a webhook source connector, an HTTP gateway, AWS Lambda or a managed pipeline. Compare signatures, retries and duplicates.
Move Oracle to S3 as Parquet with AWS DMS, AWS Glue, GoldenGate or a managed CDC pipeline. Compare how each handles NUMBER, DATE, LOBs and deletes.
A hands-on look at how CDC pipelines handle primary key updates, deletes, and orphan rows, with tests using MySQL, Debezium, and Estuary.
Move MongoDB to S3 as Parquet with Atlas Data Federation, AWS DMS, Spark or a managed pipeline. Compare how each handles nested documents and deletes.
Sync HubSpot to ClickHouse with a managed tool, open-source ELT or the HubSpot API. Compare 4 methods, plan for rate limits and dedupe with FINAL.
A practical guide to multi-tenant CDC with PostgreSQL, tenant-aware keys, and shared analytics tables using Estuary.