Autonomous Table Optimization When Your Query Workload Stops Being Predictable
Autonomous table optimization when query workloads stop being predictable: observing file layout and query patterns, scoring compaction work, adaptive.
The archive
Writing, page 12
652 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
Building Apache Iceberg Lakehouses That Run Without an Internet Connection
How to build an Apache Iceberg lakehouse that runs fully offline: storage, catalog, compute, cross-zone transfer, compliance, and the failure modes that bite.
Wiring Analytical Queries to Transactional APIs in Closed-Loop Decision Agents
Wiring analytical queries to transactional APIs in closed-loop decision agents: conditional writes, sagas with compensations, decision records, and blast.
Surviving Optimistic Commit Collisions When Hundreds of Agents Write to Iceberg
Surviving optimistic commit collisions when hundreds of agents write to Iceberg: which conflicts are real, commit buffers, partitioning, and the patterns.
How Iceberg V3 Deletion Vectors Fixed Merge-on-Read for Streaming Tables
How Iceberg V3 deletion vectors replaced accumulating positional delete files and made merge-on-read viable for streaming and CDC tables.
How Iceberg V3 Variant Shredding Changed Semi-Structured Data on S3 Tables
How Iceberg V3's Variant type and Parquet shredding turn JSON columns into prunable typed columns, with real benchmark tradeoffs and a migration path.
Reading the Apache Iceberg V4 Proposals Before They Land
A field guide to the Apache Iceberg V4 proposals: adaptive metadata trees, single-file commits, typed statistics, column families, and what is safe.
Building an Honest TCO Model for Open Lakehouses and Proprietary Warehouses
An honest TCO framework for open lakehouses versus proprietary warehouses: five cost categories, measured numbers, sensitivity analysis, and where each.
Why Agentic AI Needs a Governed Semantic Layer Behind the Model Context Protocol
Why agentic AI needs a governed semantic layer behind the Model Context Protocol: metric consistency, access control, Apache Ossie for portable.
Moving From Supply Chain Dashboards to Decision Loops With the Model Context Protocol
Moving from supply chain dashboards to decision loops with MCP: sense, decide, act, and verify, with typed action tools, idempotency keys, and graduated.