Mapping the Variant Type in Iceberg v3: Standardizing Semi-Structured AI JSON Payloads
AI applications are messy data producers. They create prompts, completions, tool calls, retrieval traces, ranking signals, evaluation scores, safet...
The archive
Writing, page 21
652 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
Designing Idempotent Pipelines in the Agentic Lakehouse: Eliminating Double-Write Anomalies
Agents retry. Networks fail. Jobs time out after doing some work. APIs return ambiguous responses. Schedulers run the same workflow twice. A human...
File Encryption for the Lakehouse: The Terminology, the Machinery, and the Hard Problem of Interoperable Encrypted Tables
For years, the open lakehouse had an honest gap that practitioners whispered about and slide decks skipped: encryption. Not the checkbox kind, ever...
The 2026-07-28 Model Context Protocol Release Candidate: What the Stateless Spec Means for Data Platforms
The date in this topic matters. Today is July 6, 2026. A release candidate dated July 28, 2026 is still in the future. That means this article shou...
The Metric Contract Mandate: Standardizing Semantic Layers Before AI Agent Access
AI agents are very good at moving quickly. That is the opportunity and the risk. If an agent can inspect metadata, generate queries, compare result...
Multi-Engine Catalog Federation with Apache Polaris: Syncing Google Cloud, AWS, and Azure Metadata
Open table formats changed the data lakehouse conversation, but they did not finish it. A table can be stored in an open format and still be hard t...
Open Source Foundations, Explained: What Apache, Linux, Eclipse, and Their Peers Actually Do, and Why Governance Differences Matter
Writing about open data and AI means repeating the same phrases over and over: donated to the Apache Software Foundation, incubating at the Linux F...
Operational vs. Analytical Systems: Why the Oldest Divide in Data Exists, What Physics Enforces It, and the Honest Truth About Hybrid Systems
Every data architecture ever drawn contains the same fault line, so old and so universal that most engineers stop seeing it: on one side, the syste...
Personal Context vs. Shared Context: A Deep Dive Into How Humans and Organizations Should Feed Their AI Agents
The most important discovery of the agent era fits in one sentence: most AI failures are context failures, not model failures. When your assistant...
Designing Private, Air-Gapped Data Lakehouses: Scaling Iceberg in Highly Secure, On-Premises Clouds
Some of the most important lakehouse work happens in environments that will never look like a simple public-cloud reference architecture. Defense...