Alex Merced's Data, Dev and AI Blog

Topic

lakehouse

36 posts tagged “lakehouse”.

  1. 30 min read

    Your Data Agents Are CPU-Bound, and the Hardware Market Just Noticed

    Analytics agents spend most of their time on CPU-bound tool calls, not the model. Size three planes separately and measure your ratio from traces.

  2. 31 min read

    How Apache Polaris Plans to Share Iceberg Tables Across Organizations

    Apache Polaris's Open Sharing proposal adds first-class shares, external consumers, and listings so any Iceberg REST engine can read shared tables.

  3. 30 min read

    What a Query Costs

    Four meters, their real proportions, and how to attribute compute to a query, a table, and a team so a platform can answer what a dashboard costs to run.

  4. 29 min read

    Postgres Meets the Lakehouse: pg_lake, pg_duckdb, and When Postgres Is Enough

    What pg_lake, pg_duckdb, and pg_mooncake do at the Iceberg level, and honest thresholds for when Postgres is enough.

  5. 28 min read

    Synthetic Data in the Lakehouse: Generation, Governance, and Testing

    What synthetic data in a lakehouse is for, the generation methods, how to preserve fidelity, and where synthetic tables belong.

  6. 30 min read

    Securing the Agentic Lakehouse Gateway: Preventing Prompt Injection and Data Exfiltration

    Agentic lakehouse gateways face prompt injection and exfiltration through query results. A threat model and defenses for the layer in front of data.

  7. 30 min read

    Query Routing at Machine Scale: Dynamic Workload Distribution Across Lakehouse Engines

    Route each lakehouse query by shape, not by sender. Signals, rules, and how to keep dashboards, batch jobs, and agents from sharing one engine.

  8. 20 min read

    Defending the Lakehouse Gateway Against Prompt Injection and Data Exfiltration

    Defending the lakehouse gateway against prompt injection and data exfiltration: per-user identity, no-SQL tool surfaces, volume bounds, and detection.

  9. 31 min read

    Designing Batch Pipelines That Write Well Into Apache Iceberg

    How to design batch pipelines that write well into Apache Iceberg: commit strategy, partitioning, sort order, write-audit-publish, and maintenance done right.

  10. 31 min read

    Federating Oracle With an Open Lakehouse Instead of Migrating It

    Federate first so analytics work now, migrate what benefits from migrating, and leave the rest where it is indefinitely.

  11. 31 min read

    The Apache Iceberg Market in the Middle of 2026

    A survey of the Apache Iceberg market in July 2026: the state of the specification, platform support, the acquisition wave, the catalog contest, and how.

  12. 30 min read

    The Whole Lakehouse Fits on Your Laptop Now

    Consumer hardware, columnar formats, single-node engines, and the Iceberg REST catalog crossed a threshold: a large share of cluster work now runs locally.

  13. 30 min read

    Iceberg's Next Version Depends on Decisions Being Made in Parquet

    The Iceberg and Parquet specifications are co-evolving. Several of the most consequential Iceberg v4 proposals are waiting on Parquet work.

  14. 30 min read

    Building a Lakehouse That Stays Inside the Border

    Residency is a storage location. Sovereignty is who can compel access, who operates the systems, and whether you can leave.

  15. 31 min read

    Table Maintenance Stopped Being a Product

    Iceberg table maintenance commoditized when every platform started shipping it. What the six operations are, what they cost, and the observability you.

  16. 31 min read

    Three Vendors Are Rebuilding the Path From Transaction to Agent

    Databricks, Snowflake, and SAP are closing the gap between operational databases and analytical platforms through acquisition, betting on different layers.

  17. 18 min read

    Data Lakehouse Open Standards for AI Agents

    An in-depth exploration of data lakehouse open standards for ai agents

  18. 18 min read

    Dremio Lakehouse AI Report: Agentic Lessons

    An in-depth exploration of dremio lakehouse ai report: agentic lessons

  19. 17 min read

    Zero-Copy Mirroring to Open Iceberg Tables

    An in-depth exploration of zero-copy mirroring to open iceberg tables

  20. 15 min read

    Migrating Proprietary Warehouses to Open Lakehouses: The 2026 Playbook for Zero-Copy Metadata Translation

    Every warehouse migration sounds simpler before the first inventory. Then the team discovers old dashboards, hidden dependencies, undocumented stor...

  21. 14 min read

    Lakehouse as the Operating Layer for Agentic AI

    Agentic AI announcements are useful when they validate the need for governed data, semantic context, and cost-aware execution.

  22. 13 min read

    What Is LTAP in the Lakehouse?

    Lakehouse transactional analytical processing is useful only when teams define freshness, isolation, and workload boundaries clearly.

  23. 14 min read

    Agentic Lakehouse Concurrency and Isolation

    How Iceberg optimistic concurrency control, partition-level isolation, and idempotency keys enable safe concurrent writes from multiple AI agents.

  24. 14 min read

    Anatomy of an Agentic Lakehouse

    The four-layer architecture of the agentic lakehouse: object storage, Apache Iceberg table format, Apache Polaris catalog, and the semantic/agent layer.

  25. 14 min read

    Lakehouse Context Layers with Atlan and Iceberg v3

    Lakehouse context layers bridge the gap between raw Iceberg tables and AI agents that need business meaning.

  26. 14 min read

    Goal-Directed Analytics Agents on Apache Iceberg

    How goal-directed analytics agents decompose business questions into sub-tasks, execute action loops over Apache Iceberg tables, and use the lakehouse.

  27. 14 min read

    Iceberg Remote Signing for Regulated Datasets

    Iceberg REST catalog remote signing provides per-file pre-signed URL access for regulated datasets.

  28. 14 min read

    Apache Iceberg v3 Deletion Vectors on Snowflake

    Apache Iceberg v3 deletion vectors replace positional delete files with binary bitmaps in Puffin files, delivering up to 10x faster DML on Snowflake.

  29. 15 min read

    CDC Without Complexity Using Iceberg v3 Row Lineage

    Iceberg v3 row lineage adds _row_id and _last_updated_sequence_number to every table, enabling native change data capture without Debezium or Kafka.

  30. 14 min read

    The 2026 Guide to Iceberg View Federation

    Iceberg views standardize SQL view definitions across engines, enabling view federation across Polaris, Nessie, and Gravitino catalogs.

  31. 14 min read

    Implementing MCP in the Lakehouse

    How to build a Model Context Protocol (MCP) server that exposes lakehouse tables and semantic views as AI-accessible tools, with Python implementation.

  32. 14 min read

    Modern Python Tooling for Apache Iceberg

    PyIceberg, IceFrame, and the Iceberg CLI form a complete Python toolchain for Iceberg table management.

  33. 14 min read

    REST Catalog Credential Vending for Lakehouse Security

    Iceberg REST catalog credential vending issues short-lived, table-scoped storage tokens for S3, ADLS, and GCS.

  34. 14 min read

    Securing Agent Identities in the Lakehouse

    How OAuth 2. 0 token exchange, OAuth 2. 1 device authorization grant, credential vending, and fine-grained access control secure AI agent identities.

  35. 14 min read

    Bidirectional Iceberg Writes with Horizon Catalog

    Snowflake Horizon Catalog enables bidirectional Iceberg writes from external engines like Spark and Trino, powered by Apache Polaris.

  36. 14 min read

    Zero-Copy Mirroring for Modern Lakehouse Migration

    Zero-copy mirroring gives teams a safer migration path because they can expose a lakehouse surface before they duplicate every byte or rewrite every workload.

Browse all posts

Work with Alex

Menu

Search

Type at least two characters.