Inside the Query Optimizer: How Engines Pick a Plan
This is Part 5 of a 10-part series on query engine design. Part 4 covered indexing strategies. This article covers what happens after the…
The archive
Writing, page 24
593 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
B-Trees, LSM Trees, and the Indexing Tradeoff Spectrum
This is Part 4 of a 10-part series on query engine design. Part 3 covered how data is structured within files. This article covers the…
How Databases Organize Data on Disk: Pages, Blocks, and File Formats
This is Part 3 of a 10-part series on query engine design. Part 2 covered row vs. column storage layouts. This article goes one level…
Row vs. Column: How Storage Layout Shapes Everything
This is Part 2 of a 10-part series on query engine design. Part 1 (Overview) introduced the nine decisions every engine must make. This…
How Query Engines Think: The Tradeoffs Behind Every Data System
Every database you have ever used is a collection of deliberate engineering tradeoffs. PostgreSQL is fast at looking up a single customer…
Migrating to Apache Iceberg: Strategies for Every Source System
This is Part 15, the final article of a 15-part Apache Iceberg Masterclass. Part 14 covered hands-on Dremio Cloud. This article covers the…
Hands-On with Apache Iceberg Using Dremio Cloud
This is Part 14 of a 15-part Apache Iceberg Masterclass. Part 13 covered streaming approaches. This article is a practical walkthrough of…
Approaches to Streaming Data into Apache Iceberg Tables
This is Part 13 of a 15-part Apache Iceberg Masterclass. Part 12 covered Python and MPP engines. This article covers the three primary…
Using Apache Iceberg with Python and MPP Query Engines
This is Part 12 of a 15-part Apache Iceberg Masterclass. Part 11 covered metadata tables. This article covers the two main ways to access…
Apache Iceberg Metadata Tables: Querying the Internals
This is Part 11 of a 15-part Apache Iceberg Masterclass. Part 10 covered maintenance operations. This article covers the metadata tables…