Serving Sub-Second Queries Over an Iceberg Lakehouse With a Hot Tier
A fraud detection agent evaluates a transaction. It needs the customer's activity over the last four hours, their ninety-day baseline, and…
The archive
Writing, page 9
593 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
Surviving Commit Conflicts When Dozens of Writers Hit the Same Iceberg Table
A compaction job runs for three hours, rewrites 4,000 files, and dies at the last step with CommitFailedException: Cannot commit changes…
The Jackson 3 Problem in Apache Iceberg, and What It Means for Your Code
A team upgrades to Spring Boot 4. The build breaks in a place nobody expected: a service that reads Iceberg table metadata through…
Wiring an AI Agent to Apache Polaris with the Model Context Protocol
An engineer opens Cursor, types "what tables do we have in the sales namespace, and which ones have a customerid column," and gets an…
Governing Iceberg Tables Across Regions Without Three Sets of Permissions
An engineer needs to join sales data in eu-west-1 with product data in us-east-1. The sales tables live in a Polaris instance the European…
Federating Oracle With an Open Lakehouse Instead of Migrating It
A team is eighteen months into moving reporting off Oracle. Forty percent of the tables are in Iceberg. The remaining sixty percent are the…
The Parquet Versioning Problem, and Why Iceberg Cares About It
A Spark job writes a table. A Trino query against the same table fails with a decoding error on one column. Nothing in the Iceberg metadata…
Building Iceberg Pipelines in Python Without Standing Up Spark
A data scientist has a transformation that takes forty lines of pandas. It reads two Iceberg tables, joins them, applies a scoring function…
Governing What Agents Cost You
A platform team I spoke with watched their query volume rise 40 times in six weeks. No new dashboards, no new users, no new data sources.…
Every AI Model Family That Matters in Mid-2026
A team I talked with recently had hardcoded one model name into forty places in their codebase. The model was deprecated with sixty days…