Estuary Blog the company blog of Estuary: latest posts and feed

Nov 11

How to Build Customer-Facing Analytics Pipelines with CDC

A practical architecture guide to customer-facing analytics with CDC, covering data freshness, serving layers, tenant isolation, and real-world SaaS examples.

Nov 6

NetSuite to ClickHouse: SuiteAnalytics Connect, Sync Methods & Ledger Modeling Guide

Move NetSuite data to ClickHouse with SuiteAnalytics Connect, an ELT tool or a script. Compare 5 methods and plan for deletes, custom fields and limits.

Nov 6

Kafka to ClickHouse: Streaming Ingestion, Duplicates & Tombstones Guide

Stream Kafka to ClickHouse with the Kafka table engine, ClickPipes or Kafka Connect. Compare 4 methods, delivery guarantees and duplicate handling.

Nov 6

Salesforce to S3: Export Objects as Parquet and Capture Changes

Move Salesforce to S3 as Parquet with Amazon AppFlow, AWS Glue, Python or a managed pipeline. Compare how each handles types, formula fields and deletes.

Nov 6

Salesforce to Iceberg: How to Load CRM Objects and Keep Tables Current

Salesforce to Iceberg: compare 4 ways to load CRM objects into Iceberg tables, see where Data 360 zero copy fits, and handle deletes and formulas.

Nov 6

Oracle to ClickHouse: CDC, Batch & Type Mapping Guide

Move data from Oracle to ClickHouse: map types, handle updates and deletes with ReplacingMergeTree, and compare 4 CDC and batch methods.

Nov 6

S3 to Kafka: How to Turn New S3 Files into Kafka Messages

Move S3 data into Kafka with an S3 source connector, AWS Lambda, Spark or a managed pipeline. Compare file ordering, formats, duplicates and setup.

Nov 6

Webhooks to Kafka: How to Receive Events and Publish Them to Topics

Send webhooks to Kafka with a webhook source connector, an HTTP gateway, AWS Lambda or a managed pipeline. Compare signatures, retries and duplicates.

Nov 6

Oracle to S3 as Parquet: Export Tables and Replicate Changes with CDC

Move Oracle to S3 as Parquet with AWS DMS, AWS Glue, GoldenGate or a managed CDC pipeline. Compare how each handles NUMBER, DATE, LOBs and deletes.

Nov 6

What your CDC pipeline does when a primary key changes

A hands-on look at how CDC pipelines handle primary key updates, deletes, and orphan rows, with tests using MySQL, Debezium, and Estuary.

Nov 4

MongoDB to S3: Parquet Files, Change Streams & Nested Documents Guide

Move MongoDB to S3 as Parquet with Atlas Data Federation, AWS DMS, Spark or a managed pipeline. Compare how each handles nested documents and deletes.

Nov 4

HubSpot to ClickHouse: Sync Methods, Setup & API Limits Guide

Sync HubSpot to ClickHouse with a managed tool, open-source ELT or the HubSpot API. Compare 4 methods, plan for rate limits and dedupe with FINAL.

Nov 3

Multi-Tenant CDC: Database-per-Tenant SaaS Architecture Guide

A practical guide to multi-tenant CDC with PostgreSQL, tenant-aware keys, and shared analytics tables using Estuary.

Oct 27

Bigger Warehouse, Slower Queries: Benchmarking The Top 7 Cloud Data Warehouses

Estuary and CorrDyn benchmarked 7 cloud data warehouses across 244 queries, measuring performance, scalability, and cost from 1 GB to 1 TB.

Oct 22

How Much Does Real-Time Data Ingestion Cost? Pricing Models Compared

Compare real-time data ingestion pricing by GB, active rows, compute, and connectors. Learn what drives total cost and how to estimate your pipeline budget.

Oct 12

How to Migrate a Production CDC Pipeline With Minimal Downtime and No Data Gaps

Migrating a production CDC pipeline requires careful planning to avoid missing changes, duplicate records, or disrupting downstream systems. This guide covers the key steps, risks, and strategies for a safe migration with minimal downtime.

Oct 2

Monitoring a Pipeline with Estuary’s Observability Tools

Learn how to monitor a real-time data pipeline with Estuary’s native dashboard, alerts, flowctl, and OpenMetrics API, then extend monitoring with Prometheus to track freshness, throughput, backlog, errors, and overall pipeline health.

Sep 21

Does CDC Slow Down a Production Database? The Real Cost of Log-Based CDC

See where PostgreSQL CDC adds overhead, from WAL and logical decoding to replication slots, vacuum pressure, and production query performance.

Sep 21

Why Transactional Consistency Matters

Transactional consistency is an important concept for applications and their underlying databases. Learn why this concept can no longer be neglected for analytical applications with near real-time data movement and autonomous agents.

Sep 17

A decade of Gazette: a streaming broker that doesn’t store data

Sep 10

PostgreSQL CDC Backfills at Scale: Running a Multi-Day Backfill While Production Keeps Writing

Learn how PostgreSQL CDC backfills affect WAL retention, source load, chunk sizing, and recovery while production continues writing.

Sep 1

What Is Estuary? A Platform for CDC, Streaming & Batch Data

Estuary captures data once and lets teams reuse it across warehouses, operational systems, and AI workflows without maintaining separate batch and streaming stacks.

Aug 26

Why we rebuilt the engine behind Estuary

Estuary rebuilt its runtime for faster, more scalable data movement with transactional correctness, autonomous scaling, and simpler operations.

Aug 20

A Hands-On Guide to Building Streaming Transformation with Estuary Derivation Agent Skills

Build streaming data transformations with Estuary Agent Skills using natural-language prompts, from filtering and joins to stateful logic, windowing, and Python.

Aug 19

PostgreSQL Replication Slots: How to Monitor, Recover, and Prevent WAL Bloat

Replication slots protect CDC continuity, but stalled consumers can retain excessive WAL or make slots unusable.