Maintaining Apache Iceberg Tables: Compaction, Expiry, and Cleanup
This is Part 10 of a 15-part Apache Iceberg Masterclass. Part 9 covered how tables degrade. This article covers the four maintenance…
The archive
Writing, page 25
593 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
How Data Lake Table Storage Degrades Over Time
This is Part 9 of a 15-part Apache Iceberg Masterclass. Part 8 covered embedded catalogs. This article explains the five ways Iceberg table…
When Catalogs Are Embedded in Storage
This is Part 8 of a 15-part Apache Iceberg Masterclass. Part 7 covered the traditional catalog landscape. This article examines a newer…
What Are Lakehouse Catalogs? The Role of Catalogs in Apache Iceberg
This is Part 7 of a 15-part Apache Iceberg Masterclass. Part 6 covered the write process and explained how the catalog enables atomic…
Writing to an Apache Iceberg Table: How Commits and ACID Actually Work
This is Part 6 of a 15-part Apache Iceberg Masterclass. Part 5 covered hidden partitioning. This article walks through the exact steps an…
Hidden Partitioning: How Iceberg Eliminates Accidental Full Table Scans
This is Part 5 of a 15-part Apache Iceberg Masterclass. Part 4 covered partition evolution. This article covers hidden partitioning, the…
Partition Evolution: Change Your Partitioning Without Rewriting Data
This is Part 4 of a 15-part Apache Iceberg Masterclass. Part 3 covered metadata-driven performance. This article explains how Iceberg…
Performance and Apache Iceberg's Metadata
This is Part 3 of a 15-part Apache Iceberg Masterclass. Part 2 covered the metadata structures of all five table formats. This article…
The Metadata Structure of Modern Table Formats
This is Part 2 of a 15-part Apache Iceberg Masterclass. Part 1 covered why table formats exist. This article breaks down exactly how each…
What Are Table Formats and Why Were They Needed?
This is Part 1 of a 15-part Apache Iceberg Masterclass. This article covers the fundamental question: what problem do table formats solve,…