Building Closed-Loop Decision Agents: Moving from Passive BI Dashboards to Active Goal-Directed Workflows
Dashboards are excellent at showing people what happened. They are less good at deciding what should happen next. That gap is where…
The archive
Writing, page 14
593 posts on Apache Iceberg, lakehouse architecture, data engineering and applied AI.
Conversational AI on Managed Iceberg: Exposing Amazon S3 Tables through MCP
The most interesting part of conversational analytics is not the chat box. The chat box is just the surface area. The harder question is…
Decoupled Catalogs vs. Managed Tables: Architectural Freedom in the Age of Table Format Convergence
Open table formats have changed buyer expectations. A few years ago, the question was whether an organization should put more analytical…
Designing Your Own AI Harness: A Deep Dive Into the Architecture of Agent Loops, Tools, Context, and Control
The most underappreciated finding in applied AI this year fits in one statistic: a major framework team took the same model, changed…
Deterministic Data Engineering With AI Harnesses: Using Claude Code, Codex, Antigravity, and OpenCode for Data Work You Can Actually Trust
There is an apparent contradiction at the heart of using AI agents for data work, and resolving it properly is worth an entire article,…
Preparing Your Data Lakehouse for the EU AI Act: Auditable Lineage and Data Provenance
The EU AI Act changes the conversation around AI architecture because it makes trust operational. It is not enough to say that an AI system…
Federation and the Lakehouse: Two Roads to Unified Data Access, and How to Know Which One to Take
Every data strategy document written this decade contains some version of the same sentence: we need a single place to access all our data.…
A Deep Dive Into File Compression: How Data Gets Smaller, Why Codecs Differ, and What to Actually Use in the Lakehouse
Somewhere in your data platform right now, a single configuration property is quietly deciding a meaningful percentage of your storage…
The File Format Renaissance: Parquet, Lance, Vortex, Nimble, BtrBlocks, and the New Physics of Columnar Storage
For a decade, the file format layer was the most settled real estate in data. Apache Parquet held the analytical world, ORC held the Hive…
Enforcing Fine-Grained Security at Machine Speed: Dynamic Access Control for High-Frequency AI Agents
AI agents change the security model for analytics. A human user may run a handful of queries, pause, interpret the answer, and ask a…