streaming

1. August 19, 202631 min read

   Mastering Apache Iceberg v3 Deletion Vectors for High-Throughput Streaming Ingest

   Here is a bill that surprises teams every quarter. A Flink pipeline streams change data capture events into an Apache Iceberg table, a few thousand…

2. August 4, 202621 min read

   How Iceberg V3 Deletion Vectors Fixed Merge-on-Read for Streaming Tables

   A change data capture pipeline lands updates from an operational database every thirty seconds. Under Iceberg V2 with merge-on-read, each of those…

3. August 4, 202621 min read

   Reading the Apache Iceberg V4 Proposals Before They Land

   A Flink job commits every five seconds. Each commit writes one small Parquet file. It also writes a manifest, rewrites a manifest list, and writes a…

4. July 28, 202631 min read

   Why Iceberg V4 Wants to Retire Equality Deletes, and What Streaming Teams Should Do About It

   A Flink job writes upserts into an Apache Iceberg table at a few thousand rows per second. The write side looks healthy. Checkpoints land on time,…

5. July 28, 202631 min read

   Apache Fluss and Kafka Solve Different Problems in an Iceberg Pipeline

   A streaming team is asked to cut dashboard latency from six minutes to under thirty seconds. The pipeline is Kafka into Flink into Apache Iceberg,…

6. July 28, 202630 min read

   Serving Sub-Second Queries Over an Iceberg Lakehouse With a Hot Tier

   A fraud detection agent evaluates a transaction. It needs the customer's activity over the last four hours, their ninety-day baseline, and the…

7. July 25, 202630 min read

   Freshness Is a Contract, Not a Note on a Dashboard

   An inventory agent rerouted a shipment last quarter for a company I spoke with, based on stock levels that were six hours old. The warehouse had…

8. July 6, 202630 min read

   The State of Streaming to Apache Iceberg in July 2026: Every Path, Its Latency, and What to Do When Seconds Are Not Fast Enough

   The most common architecture question I get in 2026 is no longer "should we use Iceberg." That one is settled. The question now is "how fresh can our…

9. July 29, 20253 min read

   Optimizing Compaction for Streaming Workloads in Apache Iceberg

   Learn how to design fast, incremental compaction strategies in Apache Iceberg to support high-throughput streaming pipelines without disrupting freshness or performance.

10. October 16, 20245 min read

    Data Lakehouse Roundup 1 - News and Insights on the Lakehouse

    What's Going on in the Data Lakehouse Space

11. October 4, 202414 min read

    Change Data Capture (CDC) when there is no CDC

    Handling Synching Changing Data Across Systems

Browse all posts