liyunzhang_intel created HIVE-16862: ---------------------------------------
Summary: Implement a similar feature like "hive.tez.dynamic.semijoin.reduction" in hive on spark Key: HIVE-16862 URL: https://issues.apache.org/jira/browse/HIVE-16862 Project: Hive Issue Type: Bug Reporter: liyunzhang_intel Currently if we enable "hive.tez.dynamic.semijoin.reduction" (the default value is true) in hive on spark, following script fail {code} set hive.optimize.ppd=true; set hive.ppd.remove.duplicatefilters=true; set hive.spark.dynamic.partition.pruning=true; set hive.optimize.metadataonly=false; set hive.optimize.index.filter=true; set hive.strict.checks.cartesian.product=false; set hive.spark.dynamic.partition.pruning=true; -- multiple sources, single key select count(*) from srcpart join srcpart_date on (srcpart.ds = srcpart_date.ds) join srcpart_hour on (srcpart.hr = srcpart_hour.hr) {code} {code} the reason why this fail see HIVE-16780, currently we only disable "hive.tez.dynamic.semijoin.reduction" when running hive on spark to pass the test. Later we can implement a similar feature like what hive on tez does. -- This message was sent by Atlassian JIRA (v6.3.15#6346)