Joe McDonnell created IMPALA-14271:
--------------------------------------
Summary: Tuple caching needs to handle runtime filters destined
for other fragments
Key: IMPALA-14271
URL: https://issues.apache.org/jira/browse/IMPALA-14271
Project: IMPALA
Issue Type: Task
Components: Backend, Frontend
Affects Versions: Impala 5.0.0
Reporter: Joe McDonnell
In benchmarking with the cost based placement, for a couple queries, some
fragments get stuck waiting for runtime filters. In particular, Q5 for TPC-H
and Q61 for TPC-DS get stuck. They time out of the wait after 5 seconds, which
this is a significant performance regression.
Consider a plan like this where we are caching above a string of hash joins.
With mt_dop, each hash join has a separate fragment on the build side:
Fragment 1:
Cache location
Hash join 3 <--- broadcast: build side fragment 2
Hash join 2 <--- broadcast: build side fragment 3
Hash join 1 <--- broadcast: build side fragment 4
Probe scan node
An example problematic runtime filter goes from hash join 3 to hash join 2's
build side fragment 3. With a cache hit above everything, the runtime filter
never gets generated, so build side fragment 3 is waiting for a filter that
never comes.
We need some approach to handle this to avoid the performance issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
