[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15349987#comment-15349987 ] Lefty Leverenz commented on HIVE-13982: --- Doc note: This adds *hive.order.columnalignment* to HiveConf.java, so it will need to be documented in the wiki for 2.2.0 and 2.1.1. * [Configuration Properties -- Query and DDL Execution | https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-QueryandDDLExecution] Added TODOC2.2 and TODOC2.1.1 labels. > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Labels: TODOC2.1.1, TODOC2.2 > Fix For: 2.2.0, 2.1.1 > > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, > HIVE-13982.4.patch, HIVE-13982.5.patch, HIVE-13982.6.patch, > HIVE-13982.7.patch, HIVE-13982.8.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15349742#comment-15349742 ] Hive QA commented on HIVE-13982: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12813381/HIVE-13982.8.patch {color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10270 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/263/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/263/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-263/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 5 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12813381 - PreCommit-HIVE-MASTER-Build > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, > HIVE-13982.4.patch, HIVE-13982.5.patch, HIVE-13982.6.patch, > HIVE-13982.7.patch, HIVE-13982.8.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15349512#comment-15349512 ] Hive QA commented on HIVE-13982: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12813180/HIVE-13982.7.patch {color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 10267 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_bucket_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_limit_pushdown3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lineage3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_parquet_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_selectDistinctStar org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union2 org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_selectDistinctStar org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_unionDistinct_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_13 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query66 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query80 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query82 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query89 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_parquet_join org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_union25 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vectorization_13 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/261/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/261/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-261/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 24 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12813180 - PreCommit-HIVE-MASTER-Build > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, > HIVE-13982.4.patch, HIVE-13982.5.patch, HIVE-13982.6.patch, > HIVE-13982.7.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15349023#comment-15349023 ] Ashutosh Chauhan commented on HIVE-13982: - +1 pending tests. Lets create follow-up jira for 2 identified issues. > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, > HIVE-13982.4.patch, HIVE-13982.5.patch, HIVE-13982.6.patch, > HIVE-13982.7.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15346156#comment-15346156 ] Hive QA commented on HIVE-13982: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12812609/HIVE-13982.6.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10260 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables_compact org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_lineage3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_all org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_vector_complex_join org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query89 org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskSchedulerService.testDelayedLocalityNodeCommErrorImmediateAllocation {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/229/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/229/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-229/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 11 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12812609 - PreCommit-HIVE-MASTER-Build > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, > HIVE-13982.4.patch, HIVE-13982.5.patch, HIVE-13982.6.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15344462#comment-15344462 ] Jesus Camacho Rodriguez commented on HIVE-13982: Updated three remaining q files. > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, > HIVE-13982.4.patch, HIVE-13982.5.patch, HIVE-13982.6.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15343180#comment-15343180 ] Hive QA commented on HIVE-13982: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12812247/HIVE-13982.5.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10253 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13 org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query17 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query85 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query89 org.apache.hive.jdbc.TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/213/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/213/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-213/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12812247 - PreCommit-HIVE-MASTER-Build > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, > HIVE-13982.4.patch, HIVE-13982.5.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15342249#comment-15342249 ] Jesus Camacho Rodriguez commented on HIVE-13982: Updated patch and RB with q file changes. > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, > HIVE-13982.4.patch, HIVE-13982.5.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341736#comment-15341736 ] Hive QA commented on HIVE-13982: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12812082/HIVE-13982.4.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 89 failed/errored test(s), 10252 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_cbo_gby_empty org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_constantPropagateForSubQuery org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynamic_rdd_cache org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_cond_pushdown org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_merge_join_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ptfgroupbyjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_regex_col org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_semijoin4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_semijoin5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_exists org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_in org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_in_having org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notexists org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notexists_having org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_notin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_unqualcolumnrefs org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_views org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union11 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union14 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union25 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_unionDistinct_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_interval_mapjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_outer_join3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_15 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver_tez_union2 org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_cbo_gby_empty org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_subquery_exists org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_subquery_in org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union5 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union7 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_unionDistinct_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_groupby_reduce org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_interval_mapjoin org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_outer_join3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_13 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_15 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query17 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query19 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query20 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query25 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query29 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query3 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query39 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query40 org.apache.hadoop.hive.cli.TestPerfCliDriver.testPerfCliDriver_query46
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15340825#comment-15340825 ] Jesus Camacho Rodriguez commented on HIVE-13982: New patch uploaded; I want to get a QA run. I still need to check whether PTF would cause troubles with the new dedup extension. I will update the JIRA case shortly. > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, > HIVE-13982.4.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15329237#comment-15329237 ] Jesus Camacho Rodriguez commented on HIVE-13982: [~ashutoshc], fails are unrelated. Could you review the patch? Thanks > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13982) Extensions to RS dedup: execute with different column order and sorting direction if possible
[ https://issues.apache.org/jira/browse/HIVE-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327706#comment-15327706 ] Hive QA commented on HIVE-13982: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12809823/HIVE-13982.3.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10225 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_globallimit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/107/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/107/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-107/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12809823 - PreCommit-HIVE-MASTER-Build > Extensions to RS dedup: execute with different column order and sorting > direction if possible > - > > Key: HIVE-13982 > URL: https://issues.apache.org/jira/browse/HIVE-13982 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 2.2.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-13982.2.patch, HIVE-13982.3.patch, HIVE-13982.patch > > > Pointed out by [~gopalv]. > RS dedup should kick in for these cases, avoiding an additional shuffle stage. > {code} > select state, city, sum(sales) from table > group by state, city > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state, city > limit 10; > {code} > {code} > select state, city, sum(sales) from table > group by city, state > order by state desc, city > limit 10; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)