B-Trees, LSM Trees, and the Indexing Tradeoff Spectrum
B-trees balance reads and writes for OLTP. LSM trees maximize write throughput. Bitmap indexes accelerate OLAP filtering. Here is when to use each.
The archive
Writing, page 30
652 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
How Databases Organize Data on Disk: Pages, Blocks, and File Formats
Databases structure data on disk as heap files, sorted files, or LSM trees, then wrap it in formats like Parquet with metadata that lets engines skip.
Row vs. Column: How Storage Layout Shapes Everything
Row stores keep records together for fast transactions. Column stores keep field values together for fast analytics.
How Query Engines Think: The Tradeoffs Behind Every Data System
Every database is a collection of engineering tradeoffs. Learn the 9 design decisions that shape how query engines store, index, and process your data.
Migrating to Apache Iceberg: Strategies for Every Source System
Migrate to Iceberg from Hive, data warehouses, or raw files using in-place migration, full rewrite, or the zero-downtime view swap pattern.
Hands-On with Apache Iceberg Using Dremio Cloud
A practical walkthrough of creating, querying, and optimizing Iceberg tables on Dremio Cloud, from account setup to AI-powered analytics.
Approaches to Streaming Data into Apache Iceberg Tables
Stream data into Iceberg with Spark Structured Streaming, Flink, or Kafka Connect. Here is how each works and the trade-offs between latency and maintenance.
Using Apache Iceberg with Python and MPP Query Engines
Access Iceberg tables from Python with PyIceberg, DuckDB, and Polars, or through MPP engines like Dremio, Spark, and Trino. Here is how each approach works.
Apache Iceberg Metadata Tables: Querying the Internals
Iceberg metadata tables let you query snapshots, files, manifests, and partitions using SQL. Here is every metadata table and how to use them.
Maintaining Apache Iceberg Tables: Compaction, Expiry, and Cleanup
Keep Iceberg tables fast with compaction, snapshot expiry, orphan cleanup, and manifest rewriting. Here is when and how to run each operation.