Ádám Szita created HIVE-26137:
---------------------------------
Summary: Optimized transfer of Iceberg residual expressions from
AM to execution
Key: HIVE-26137
URL: https://issues.apache.org/jira/browse/HIVE-26137
Project: Hive
Issue Type: Improvement
Reporter: Ádám Szita
HIVE-25967 introduced a hack to prevent Iceberg filter expressions to be
serialized into splits. This temporary fix was to avoid OOM problems on Tez AM
side, but at the same time prevented predicate pushdowns to work on the
execution side too.
This ticket intends to incorporate the long term solution. It turns out that
the file scan tasks created by Iceberg actually don't contain a "residual"
expressions, but rather a complete/original one. It becomes residual only when
it is evaluated against the tasks' partition value, which only happens on the
execution site. This means that the original filter is the same expression for
all splits in Tez AM, so we can transfer it via job conf instead.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
