Hash, Sort-Merge, Broadcast: How Distributed Joins Work
Distributed joins move data across the network using shuffle, broadcast, or co-location strategies. Here is how each works and when engines choose which.
The archive
Writing, page 30
657 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
Partitioning, Sharding, and Data Distribution Strategies
Hash partitioning distributes data evenly. Range partitioning enables fast range scans. Both create tradeoffs.
Buffer Pools, Caches, and the Memory Hierarchy
Databases use buffer pools, column caches, and result caches to keep hot data in RAM. Here is how each caching strategy works and what happens when data.
Volcano, Vectorized, Compiled: How Engines Execute Your Query
The Volcano model processes one row at a time. Vectorized execution processes batches with SIMD. Code generation fuses operators into compiled code.
Inside the Query Optimizer: How Engines Pick a Plan
Query optimizers transform SQL into execution plans using rule-based rewrites, cost-based search, and adaptive runtime adjustments.
B-Trees, LSM Trees, and the Indexing Tradeoff Spectrum
B-trees balance reads and writes for OLTP. LSM trees maximize write throughput. Bitmap indexes accelerate OLAP filtering. Here is when to use each.
How Databases Organize Data on Disk: Pages, Blocks, and File Formats
Databases structure data on disk as heap files, sorted files, or LSM trees, then wrap it in formats like Parquet with metadata that lets engines skip.
Row vs. Column: How Storage Layout Shapes Everything
Row stores keep records together for fast transactions. Column stores keep field values together for fast analytics.
How Query Engines Think: The Tradeoffs Behind Every Data System
Every database is a collection of engineering tradeoffs. Learn the 9 design decisions that shape how query engines store, index, and process your data.
Migrating to Apache Iceberg: Strategies for Every Source System
Migrate to Iceberg from Hive, data warehouses, or raw files using in-place migration, full rewrite, or the zero-downtime view swap pattern.