Hi everyone,
The materialized view working group has a draft of the materialized view
spec that is ready for broader review, and we'd like to invite feedback
from the community.
The draft is in this PR: https://github.com/apache/iceberg/pull/11041
In short, the spec defines:
- Materialized view metadata: a materialized view is an Iceberg view
whose version references a storage table. Readers unaware of
materialized views ignore the reference and evaluate the view query
directly.
- Refresh state: stored in the storage table's snapshot summary. It
records the view version and refresh start time used for the refresh,
plus the state of the source tables and views it chose to track. Readers
can use it to decide whether the precomputed data is still fresh enough
to use.
To validate the design, we have proof-of-concept implementations in
Spark and Trino:
Spark: https://github.com/apache/iceberg/pull/9830
Trino: https://github.com/trinodb/trino/pull/31027
We'd especially appreciate feedback from engine and catalog implementers
on whether the spec covers your use cases. Please leave comments on the
PR or reply to this thread.
Thanks to everyone in the working group who contributed to the design
and reviews.
Best,
Jan
