Building Iceberg Pipelines in Python Without Standing Up Spark
A data scientist has a transformation that takes forty lines of pandas. It reads two Iceberg tables, joins them, applies a scoring function…
The archive
Writing, page 5
546 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
Governing What Agents Cost You
A platform team I spoke with watched their query volume rise 40 times in six weeks. No new dashboards, no new users, no new data sources.…
Every AI Model Family That Matters in Mid-2026
A team I talked with recently had hardcoded one model name into forty places in their codebase. The model was deprecated with sixty days…
Freshness Is a Contract, Not a Note on a Dashboard
An inventory agent rerouted a shipment last quarter for a company I spoke with, based on stock levels that were six hours old. The…
The Apache Iceberg Market in the Middle of 2026
Two years ago the interesting question was whether your data platform supported Apache Iceberg. Today every platform claims it does, and…
The Whole Lakehouse Fits on Your Laptop Now
A colleague spent forty minutes last month provisioning a cluster to profile a 90 GB Parquet dataset. Startup, dependency resolution, a…
Iceberg's Next Version Depends on Decisions Being Made in Parquet
Follow an Apache Iceberg design discussion about column-level updates long enough and it stops being an Iceberg discussion. The question of…
Building a Lakehouse That Stays Inside the Border
A manufacturer in southern Germany asked me a question during an architecture review that I have thought about since. Their data sat in a…
Table Maintenance Stopped Being a Product
A team I worked with had a dashboard that loaded in three seconds in January and forty seconds in June. Data volume grew 20 percent over…
The Model Class Enterprise Data Teams Are Not Using Yet
A finance team asks which of their 40,000 open invoices will pay late. The data sits in a table with 22 columns: customer, terms, amount,…