Implementing Positional Deletes in Iceberg v3: Streamlining Merge-on-Read for Fast-Inbound Event Lakes
Event data has a way of humbling neat architecture diagrams. It arrives late. It arrives twice. It arrives with incorrect attributes. It…
The archive
Writing, page 15
593 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
Mapping the Variant Type in Iceberg v3: Standardizing Semi-Structured AI JSON Payloads
AI applications are messy data producers. They create prompts, completions, tool calls, retrieval traces, ranking signals, evaluation…
Designing Idempotent Pipelines in the Agentic Lakehouse: Eliminating Double-Write Anomalies
Agents retry. Networks fail. Jobs time out after doing some work. APIs return ambiguous responses. Schedulers run the same workflow twice.…
File Encryption for the Lakehouse: The Terminology, the Machinery, and the Hard Problem of Interoperable Encrypted Tables
For years, the open lakehouse had an honest gap that practitioners whispered about and slide decks skipped: encryption. Not the checkbox…
The 2026-07-28 Model Context Protocol Release Candidate: What the Stateless Spec Means for Data Platforms
The date in this topic matters. Today is July 6, 2026. A release candidate dated July 28, 2026 is still in the future. That means this…
The Metric Contract Mandate: Standardizing Semantic Layers Before AI Agent Access
AI agents are very good at moving quickly. That is the opportunity and the risk. If an agent can inspect metadata, generate queries,…
Multi-Engine Catalog Federation with Apache Polaris: Syncing Google Cloud, AWS, and Azure Metadata
Open table formats changed the data lakehouse conversation, but they did not finish it. A table can be stored in an open format and still…
Open Source Foundations, Explained: What Apache, Linux, Eclipse, and Their Peers Actually Do, and Why Governance Differences Matter
Writing about open data and AI means repeating the same phrases over and over: donated to the Apache Software Foundation, incubating at the…
Operational vs. Analytical Systems: Why the Oldest Divide in Data Exists, What Physics Enforces It, and the Honest Truth About Hybrid Systems
Every data architecture ever drawn contains the same fault line, so old and so universal that most engineers stop seeing it: on one side,…
Personal Context vs. Shared Context: A Deep Dive Into How Humans and Organizations Should Feed Their AI Agents
The most important discovery of the agent era fits in one sentence: most AI failures are context failures, not model failures. When your…