Tim Armstrong created IMPALA-10453:
--------------------------------------
Summary: Support file/partition pruning via runtime filters on
Iceberg
Key: IMPALA-10453
URL: https://issues.apache.org/jira/browse/IMPALA-10453
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Tim Armstrong
This is a placeholder to figure out what we'd need to do to support dynamic
file-level pruning in Iceberg using runtime filters, i.e. have parity for
partition pruning.
* If there is a single partition value per file, then applying bloom filters to
the row group stats would be effective at pruning files.
* If there are partition transforms, e.g. hash-based, then I think we probably
need to track the partition that the file is associated with and then have some
custom logic in the parquet scanner to do partition pruning.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
