Sahil Takiar created HIVE-17958:
-----------------------------------
Summary: spark_dynamic_partition_pruning.q fails when
hive.tez.dynamic.semijoin.reduction is false
Key: HIVE-17958
URL: https://issues.apache.org/jira/browse/HIVE-17958
Project: Hive
Issue Type: Sub-task
Components: Spark
Reporter: Sahil Takiar
Assignee: Sahil Takiar
Looks like {{RedundantDynamicPruningConditionsRemoval}} causes DPP to be
disabled in a few cases (not sure why). When
{{hive.tez.dynamic.semijoin.reduction}} is {{true}} (the default), then this
rule is disabled so the normal tests don't hit this issue.
But when I disable {{hive.tez.dynamic.semijoin.reduction}} then the following
query no longer fully triggers DPP:
{code}
EXPLAIN select count(*) from srcpart join srcpart_date on (srcpart.ds =
srcpart_date.ds) join srcpart_hour on (srcpart.hr = srcpart_hour.hr)
5777 where srcpart_date.`date` = '2008-04-08' and srcpart_hour.hour = 11 and
srcpart.hr = 11
{code}
There should be two DPP sinks, but when the config is set to false, there is
only one.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
