How Data Lake Table Storage Degrades Over Time
Iceberg tables degrade through small files, orphan files, metadata bloat, sort order decay, and partition skew. Here is how to diagnose each problem.
The archive
Writing, page 31
652 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
When Catalogs Are Embedded in Storage
S3 Tables and MinIO AI Stor embed the Iceberg catalog directly in the storage layer. Here is when embedded catalogs make sense and when they do not.
What Are Lakehouse Catalogs? The Role of Catalogs in Apache Iceberg
Lakehouse catalogs store metadata pointers, manage namespaces, and enforce access control. Here is the complete catalog landscape from Polaris to Glue.
Writing to an Apache Iceberg Table: How Commits and ACID Actually Work
Here is exactly how an engine writes to an Iceberg table, step by step, from data files through the atomic commit that makes ACID guarantees possible.
Hidden Partitioning: How Iceberg Eliminates Accidental Full Table Scans
Iceberg's hidden partitioning separates physical layout from user queries using transform functions.
Partition Evolution: Change Your Partitioning Without Rewriting Data
Iceberg lets you change partition schemes without rewriting data. Here is how partition evolution works internally and why Hive-style partitioning could.
Performance and Apache Iceberg's Metadata
Iceberg's three-layer metadata tree eliminates directory listing and enables multi-level data skipping. Here is how scan planning actually works.
The Metadata Structure of Modern Table Formats
Iceberg uses a metadata tree, Delta Lake uses a transaction log, Hudi uses a timeline. Here is exactly how each format organizes metadata and why it matters.
What Are Table Formats and Why Were They Needed?
Table formats like Apache Iceberg solved the ACID, schema, and performance problems that turned data lakes into data swamps. Here is how each one works.
2025 Year in Review Apache Iceberg, Polaris, Parquet, and Arrow
A look back at key developments in Apache Iceberg, Polaris, Parquet, and Arrow in 2025.