Topic
metadata
10 posts tagged “metadata”.
-
Context Engineering for Data Agents
Why text-to-SQL accuracy collapses on enterprise schemas, the five kinds of context an agent needs, where each one hides, and how to make the semantics legible.
-
Migrating Into Iceberg Without Moving Data
add_files, snapshot, and migrate compared: the three in-place paths into Iceberg, the reconciliation each requires, the layout traps, and the rollback story.
-
Partition Statistics Files in Apache Iceberg
The underused Iceberg metadata for planning: what the partition statistics file holds, what the spec guarantees, how to write one, and when it earns its slot.
-
Metadata Platforms in 2026: DataHub, OpenMetadata, Atlan, and Catalog Convergence
How the technical catalog and the metadata platform are converging in 2026, and how to arrange the two layers for a lakehouse.
-
Parquet-Only Manifests in Iceberg v4: Why the Metadata Layer Is Going Columnar
Iceberg v4 is moving manifests from Avro to Parquet so planners can read only the stats they need. Why the metadata layer is going columnar.
-
The Hidden Cost of Tiny Iceberg Commits
Trace what one tiny Iceberg commit writes, then model hourly, per-minute, and per-second cadences so streaming costs become arithmetic, not adjectives.
-
Iceberg v4's Adaptive Metadata Tree, Explained From First Principles
Iceberg v4's adaptive metadata tree, explained from first principles: why commits rewrite too much today and how the tree makes change cheaper.
-
Why Iceberg v4 Is Really About Making the Cost of Change Proportional to the Change
Iceberg v4 is really about making the cost of a change proportional to the change. The principle, the current tax, and what the redesign pays down.
-
Reading the Apache Iceberg V4 Proposals Before They Land
A field guide to the Apache Iceberg V4 proposals: adaptive metadata trees, single-file commits, typed statistics, column families, and what is safe.
-
The Endgame – Building an Autonomous Optimization Pipeline for Apache Iceberg
Learn how to automate compaction, snapshot expiration, and layout optimization in Apache Iceberg using metadata-driven triggers and orchestration tools for a self-healing lakehouse.