table format

Apache Iceberg

Apache Iceberg is an open table format for huge analytic datasets. This profile covers what it does, how it is adopted, and where it stops being the right choice.

What the product actually does

Iceberg is not a database or a query engine. It is a specification for how to lay out a table as files in object storage, plus a metadata layer that records which files belong to the table at any given snapshot. Any engine that implements the spec can read the same table.

That portability is the point. A table written by Spark can be read by Trino, queried by Snowflake and updated by Flink, all against the same bytes in S3. Before Iceberg, moving between engines meant copying data.

How it is adopted and what it costs

There is no licence. The cost is the catalogue — the service that tracks table metadata and hands out the current snapshot — and the operational discipline of compaction. Small files accumulate and a table that is never compacted becomes slow to read.

Managed catalogues exist from every major cloud and from the warehouse vendors, and they are where the money is. The format is free; the coordination around it is a product.

Where it breaks down

Write latency. A commit creates a new metadata file and a snapshot, which is fine for batch and awkward for a workload that wants sub-second visibility. Streaming into Iceberg works, but the tuning is real.

Ecosystem maturity varies by engine. Reads are universally good; writes, row-level deletes and incremental reads are supported unevenly, and a feature that works in one engine may be missing in the one you actually run.

What switching costs

Low. The format is open and the data is files. The catalogue and the compaction jobs are the parts that have to be replaced.

Sources

Apache Iceberg public specification and documentation, read October 2026.