Building a Universal Lakehouse Catalog - Beyond Iceberg Tables
Exploring paths to a universal lakehouse catalog that supports multiple data formats and engines, building on Apache Iceberg's success.
The archive
Writing, page 33
657 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
Intro to Apache Iceberg with Apache Polaris and Apache Spark
Learn how to leverage Apache Iceberg with Apache Polaris and Apache Spark to build scalable and efficient data lakehouses.
The State of Apache Iceberg v4 - October 2025 Edition
What's Coming in Apache Iceberg v4: A Deep Dive into the Future of Open Table Formats
The Ultimate Guide to Open Table Formats - Iceberg, Delta Lake, Hudi, Paimon, and DuckLake
Understanding Iceberg, Delta Lake, Hudi, Paimon, and DuckLake
The 2025 & 2026 Ultimate Guide to the Data Lakehouse and the Data Lakehouse Ecosystem
What is the Data Lakehouse and the Data Lakehouse Ecosystem? This comprehensive guide covers everything you need to know about the Data Lakehouse architecture, open table formats like Apache Iceberg, Delta Lake, Apache Hudi, and Apache Paimon, and the modern data ecosystem that supports them.
The Endgame – Building an Autonomous Optimization Pipeline for Apache Iceberg
Learn how to automate compaction, snapshot expiration, and layout optimization in Apache Iceberg using metadata-driven triggers and orchestration tools for a self-healing lakehouse.
Managing Large-Scale Optimizations – Parallelism, Checkpointing, and Fail Recovery
Learn how to scale Apache Iceberg table optimizations across large datasets using parallelism, checkpointing, and fail recovery to ensure reliability and performance.
Unlocking the Power of Agentic AI with Apache Iceberg and Dremio
Unlocking the Power of Agentic AI with Apache Iceberg and Dremio
Hidden Pitfalls – Compaction and Partition Evolution in Apache Iceberg
Partition evolution in Apache Iceberg is a powerful feature, but if not managed carefully, it can introduce fragmentation and impact compaction performance. Learn how to handle it effectively.
Using Iceberg Metadata Tables to Determine When Compaction Is Needed
Discover how to use Apache Iceberg's metadata tables to proactively detect small files, bloated manifests, and table fragmentation - so you can trigger compaction only when it's needed.