Topic
PyIceberg
6 posts tagged “PyIceberg”.
-
Local Iceberg Development Environments: Docker, MinIO, and In-Memory Catalogs for CI
A data engineer changes the merge logic in a pipeline that writes to an Apache Iceberg table. To test it, they run the job against the development…
-
Moving Iceberg Tables Between Catalogs Without Rewriting Data
A platform team has 3,000 Iceberg tables in a Hive Metastore and 900 terabytes of Parquet behind them. They are moving to a REST catalog. Someone on…
-
Serverless Iceberg Ingestion with PyIceberg and DuckDB: Micro-Batches Without a Spark Cluster
A team has 40 event feeds landing in an object store. Most of them produce a few hundred megabytes an hour. A handful spike to a few gigabytes during…
-
Building Lightweight Serverless Ingestion to Apache Iceberg with PyIceberg and DuckDB
The most common ingestion job in most companies is small. A vendor drops a CSV in a bucket every hour. A webhook delivers a few thousand JSON events…
-
Building Iceberg Pipelines in Python Without Standing Up Spark
A data scientist has a transformation that takes forty lines of pandas. It reads two Iceberg tables, joins them, applies a scoring function from a…
-
PyIceberg at Scale Without Apache Spark
Python-first Iceberg work is useful when it stays honest about what Python should and should not do. For Python data engineers and platform teams,…