[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684177#comment-16684177 ] Jesus Camacho Rodriguez commented on HIVE-20850: +1 > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch, > HIVE-20850.04.patch, HIVE-20850.05.patch, HIVE-20850.06.patch, > HIVE-20850.07.patch, HIVE-20850.07.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when (d_day_name='Monday') then ss_sales_price else null > end) mon_sales, > sum(case when (d_day_name='Tuesday') then ss_sales_price else null > end) tue_sales, > sum(case when (d_day_name='Wednesday') then ss_sales_price else null > end) wed_sales, > sum(case when (d_day_name='Thursday') then ss_sales_price else null > end) thu_sales, > sum(case when (d_day_name='Friday') then ss_sales_price else null > end) fri_sales, > sum(case when (d_day_name='Saturday') then ss_sales_price else null > end) sat_sales > from date_dim, store_sales, store > where d_date_sk = ss_sold_date_sk and >s_store_sk = ss_store_sk and >s_gmt_offset = -6 and >d_year = 1998 > group by s_store_name, s_store_id > order by s_store_name, > s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales > limit 100; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683117#comment-16683117 ] Hive QA commented on HIVE-20850: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947758/HIVE-20850.07.patch {color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 15539 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14878/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14878/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14878/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12947758 - PreCommit-HIVE-Build > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch, > HIVE-20850.04.patch, HIVE-20850.05.patch, HIVE-20850.06.patch, > HIVE-20850.07.patch, HIVE-20850.07.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when (d_day_name='Monday') then ss_sales_price else null > end) mon_sales, > sum(case when (d_day_name='Tuesday') then ss_sales_price else null > end) tue_sales, > sum(case when (d_day_name='Wednesday') then ss_sales_price else null > end) wed_sales, > sum(case when (d_day_name='Thursday') then ss_sales_price else null > end) thu_sales, > sum(case when (d_day_name='Friday') then ss_sales_price else null > end) fri_sales, > sum(case when (d_day_name='Saturday') then ss_sales_price else null > end) sat_sales > from date_dim, store_sales, store > where d_date_sk = ss_sold_date_sk and >s_store_sk = ss_store_sk and >s_gmt_offset = -6 and >d_year = 1998 > group by s_store_name, s_store_id > order by s_store_name, > s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales > limit 100; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683101#comment-16683101 ] Hive QA commented on HIVE-20850: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 49s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 31s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 14s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 17s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 29s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 27s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 40s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 39s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 14s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 17s{color} | {color:red} itests/hive-unit: The patch generated 2 new + 51 unchanged - 0 fixed = 53 total (was 51) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 39s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 34m 36s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-14878/dev-support/hive-personality.sh | | git revision | master / d22fc5b | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-14878/yetus/diff-checkstyle-itests_hive-unit.txt | | modules | C: accumulo-handler hbase-handler itests/hive-unit ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-14878/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch, > HIVE-20850.04.patch, HIVE-20850.05.patch, HIVE-20850.06.patch, > HIVE-20850.07.patch, HIVE-20850.07.patch > > > noticed by
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683080#comment-16683080 ] Hive QA commented on HIVE-20850: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947750/HIVE-20850.07.patch {color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15539 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[constprog_partitioner] (batchId=191) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14877/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14877/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14877/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12947750 - PreCommit-HIVE-Build > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch, > HIVE-20850.04.patch, HIVE-20850.05.patch, HIVE-20850.06.patch, > HIVE-20850.07.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when (d_day_name='Monday') then ss_sales_price else null > end) mon_sales, > sum(case when (d_day_name='Tuesday') then ss_sales_price else null > end) tue_sales, > sum(case when (d_day_name='Wednesday') then ss_sales_price else null > end) wed_sales, > sum(case when (d_day_name='Thursday') then ss_sales_price else null > end) thu_sales, > sum(case when (d_day_name='Friday') then ss_sales_price else null > end) fri_sales, > sum(case when (d_day_name='Saturday') then ss_sales_price else null > end) sat_sales > from date_dim, store_sales, store > where d_date_sk = ss_sold_date_sk and >s_store_sk = ss_store_sk and >s_gmt_offset = -6 and >d_year = 1998 > group by s_store_name, s_store_id > order by s_store_name, > s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales > limit 100; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683056#comment-16683056 ] Hive QA commented on HIVE-20850: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 27s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 21s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 11s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 18s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 27s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 29s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 36s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 39s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 15s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 15s{color} | {color:red} itests/hive-unit: The patch generated 2 new + 51 unchanged - 0 fixed = 53 total (was 51) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 39s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 34m 37s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-14877/dev-support/hive-personality.sh | | git revision | master / d22fc5b | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-14877/yetus/diff-checkstyle-itests_hive-unit.txt | | modules | C: accumulo-handler hbase-handler itests/hive-unit ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-14877/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch, > HIVE-20850.04.patch, HIVE-20850.05.patch, HIVE-20850.06.patch, > HIVE-20850.07.patch > > > noticed by [~gopalv]: If there is a
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682240#comment-16682240 ] Hive QA commented on HIVE-20850: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947547/HIVE-20850.06.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14851/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14851/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14851/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Tests exited with: Exception: Patch URL https://issues.apache.org/jira/secure/attachment/12947547/HIVE-20850.06.patch was found in seen patch url's cache and a test was probably run already on it. Aborting... {noformat} This message is automatically generated. ATTACHMENT ID: 12947547 - PreCommit-HIVE-Build > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch, > HIVE-20850.04.patch, HIVE-20850.05.patch, HIVE-20850.06.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when (d_day_name='Monday') then ss_sales_price else null > end) mon_sales, > sum(case when (d_day_name='Tuesday') then ss_sales_price else null > end) tue_sales, > sum(case when (d_day_name='Wednesday') then ss_sales_price else null > end) wed_sales, > sum(case when (d_day_name='Thursday') then ss_sales_price else null > end) thu_sales, > sum(case when (d_day_name='Friday') then ss_sales_price else null > end) fri_sales, > sum(case when (d_day_name='Saturday') then ss_sales_price else null > end) sat_sales > from date_dim, store_sales, store > where d_date_sk = ss_sold_date_sk and >s_store_sk = ss_store_sk and >s_gmt_offset = -6 and >d_year = 1998 > group by s_store_name, s_store_id > order by s_store_name, > s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales > limit 100; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682041#comment-16682041 ] Hive QA commented on HIVE-20850: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947547/HIVE-20850.06.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14841/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14841/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14841/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Tests exited with: Exception: Patch URL https://issues.apache.org/jira/secure/attachment/12947547/HIVE-20850.06.patch was found in seen patch url's cache and a test was probably run already on it. Aborting... {noformat} This message is automatically generated. ATTACHMENT ID: 12947547 - PreCommit-HIVE-Build > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch, > HIVE-20850.04.patch, HIVE-20850.05.patch, HIVE-20850.06.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when (d_day_name='Monday') then ss_sales_price else null > end) mon_sales, > sum(case when (d_day_name='Tuesday') then ss_sales_price else null > end) tue_sales, > sum(case when (d_day_name='Wednesday') then ss_sales_price else null > end) wed_sales, > sum(case when (d_day_name='Thursday') then ss_sales_price else null > end) thu_sales, > sum(case when (d_day_name='Friday') then ss_sales_price else null > end) fri_sales, > sum(case when (d_day_name='Saturday') then ss_sales_price else null > end) sat_sales > from date_dim, store_sales, store > where d_date_sk = ss_sold_date_sk and >s_store_sk = ss_store_sk and >s_gmt_offset = -6 and >d_year = 1998 > group by s_store_name, s_store_id > order by s_store_name, > s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales > limit 100; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681298#comment-16681298 ] Hive QA commented on HIVE-20850: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947547/HIVE-20850.06.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 15527 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] (batchId=154) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_smb_ptf] (batchId=164) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[constprog_semijoin] (batchId=191) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[dynamic_rdd_cache] (batchId=190) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1] (batchId=190) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_use_ts_stats_for_mapjoin] (batchId=190) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill (batchId=258) org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveBackKill (batchId=258) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomReadOps (batchId=258) org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerDagTotalTasks (batchId=258) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14830/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14830/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14830/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12947547 - PreCommit-HIVE-Build > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch, > HIVE-20850.04.patch, HIVE-20850.05.patch, HIVE-20850.06.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when (d_day_name='Monday') then ss_sales_price else null > end) mon_sales, > sum(case when (d_day_name='Tuesday') then ss_sales_price else null > end) tue_sales, > sum(case when (d_day_name='Wednesday') then ss_sales_price else null > end) wed_sales, > sum(case when (d_day_name='Thursday') then ss_sales_price else null > end) thu_sales, > sum(case when (d_day_name='Friday') then ss_sales_price else null > end) fri_sales, > sum(case when (d_day_name='Saturday') then ss_sales_price else null > end) sat_sales > from date_dim, store_sales, store > where d_date_sk = ss_sold_date_sk and >s_store_sk = ss_store_sk and >s_gmt_offset = -6 and >d_year = 1998 > group by s_store_name, s_store_id > order by s_store_name, > s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales > limit 100; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681244#comment-16681244 ] Hive QA commented on HIVE-20850: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 38s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 35s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 35s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 1s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 26s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 27s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs warnings. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 38s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 15s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 29m 28s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-14830/dev-support/hive-personality.sh | | git revision | master / 961224c | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | modules | C: accumulo-handler hbase-handler ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-14830/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch, > HIVE-20850.04.patch, HIVE-20850.05.patch, HIVE-20850.06.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677711#comment-16677711 ] Hive QA commented on HIVE-20850: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947102/HIVE-20850.03.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14783/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14783/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14783/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Tests exited with: Exception: Patch URL https://issues.apache.org/jira/secure/attachment/12947102/HIVE-20850.03.patch was found in seen patch url's cache and a test was probably run already on it. Aborting... {noformat} This message is automatically generated. ATTACHMENT ID: 12947102 - PreCommit-HIVE-Build > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when (d_day_name='Monday') then ss_sales_price else null > end) mon_sales, > sum(case when (d_day_name='Tuesday') then ss_sales_price else null > end) tue_sales, > sum(case when (d_day_name='Wednesday') then ss_sales_price else null > end) wed_sales, > sum(case when (d_day_name='Thursday') then ss_sales_price else null > end) thu_sales, > sum(case when (d_day_name='Friday') then ss_sales_price else null > end) fri_sales, > sum(case when (d_day_name='Saturday') then ss_sales_price else null > end) sat_sales > from date_dim, store_sales, store > where d_date_sk = ss_sold_date_sk and >s_store_sk = ss_store_sk and >s_gmt_offset = -6 and >d_year = 1998 > group by s_store_name, s_store_id > order by s_store_name, > s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales > limit 100; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677653#comment-16677653 ] Hive QA commented on HIVE-20850: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947102/HIVE-20850.03.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 214 failed/errored test(s), 15526 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries] (batchId=264) org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite] (batchId=272) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[allcolref_in_udf] (batchId=56) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_join_pkfk] (batchId=15) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join13] (batchId=87) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join19] (batchId=70) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join19_inclause] (batchId=19) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join2] (batchId=69) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join32] (batchId=92) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join9] (batchId=82) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats2] (batchId=94) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats] (batchId=52) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark1] (batchId=73) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark2] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark3] (batchId=48) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark4] (batchId=1) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_4] (batchId=26) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_5] (batchId=62) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_8] (batchId=5) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_const] (batchId=19) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_join1] (batchId=77) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_outer_join_ppr] (batchId=7) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constantPropagateForSubQuery] (batchId=68) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constprog2] (batchId=14) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constprog_partitioner] (batchId=78) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer8] (batchId=14) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_5] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[deleteAnalyze] (batchId=33) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_cond_pushdown] (batchId=65) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_join_breaktask] (batchId=80) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_map_operators] (batchId=71) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_join_preds] (batchId=26) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[innerjoin1] (batchId=26) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input23] (batchId=50) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_3] (batchId=70) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join45] (batchId=21) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join47] (batchId=34) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[louter_join_ppr] (batchId=46) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin1] (batchId=6) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin46] (batchId=63) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin47] (batchId=64) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_12] (batchId=1) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_1] (batchId=91) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_3] (batchId=59) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_4] (batchId=28) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mergejoin] (batchId=64) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mergejoins] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_with_join2] (batchId=85) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[outer_join_ppr] (batchId=21) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcr] (batchId=64) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pointlookup2] (batchId=88)
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677625#comment-16677625 ] Hive QA commented on HIVE-20850: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 30s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 44s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 23m 17s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-14781/dev-support/hive-personality.sh | | git revision | master / 6d713b6 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-14781/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when (d_day_name='Monday') then ss_sales_price else null > end) mon_sales, > sum(case when (d_day_name='Tuesday') then ss_sales_price else null > end) tue_sales, > sum(case when (d_day_name='Wednesday') then ss_sales_price else null > end) wed_sales, > sum(case when (d_day_name='Thursday') then ss_sales_price else null > end) thu_sales, > sum(case when (d_day_name='Friday') then ss_sales_price else null > end) fri_sales, > sum(case when (d_day_name='Saturday') then ss_sales_price else null > end) sat_sales > from date_dim, store_sales, store > where d_date_sk = ss_sold_date_sk and >s_store_sk = ss_store_sk and >s_gmt_offset = -6 and >d_year = 1998 > group by
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677022#comment-16677022 ] Zoltan Haindrich commented on HIVE-20850: - thank you [~jcamachorodriguez]; I've adjusted the patch! > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch, HIVE-20850.03.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when (d_day_name='Monday') then ss_sales_price else null > end) mon_sales, > sum(case when (d_day_name='Tuesday') then ss_sales_price else null > end) tue_sales, > sum(case when (d_day_name='Wednesday') then ss_sales_price else null > end) wed_sales, > sum(case when (d_day_name='Thursday') then ss_sales_price else null > end) thu_sales, > sum(case when (d_day_name='Friday') then ss_sales_price else null > end) fri_sales, > sum(case when (d_day_name='Saturday') then ss_sales_price else null > end) sat_sales > from date_dim, store_sales, store > where d_date_sk = ss_sold_date_sk and >s_store_sk = ss_store_sk and >s_gmt_offset = -6 and >d_year = 1998 > group by s_store_name, s_store_id > order by s_store_name, > s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales > limit 100; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676902#comment-16676902 ] Jesus Camacho Rodriguez commented on HIVE-20850: [~kgyrtkirk], HIVE-20835 has gone in, probably you need to rebase this patch. > Push case conditional from projections to dimension tables if possible > -- > > Key: HIVE-20850 > URL: https://issues.apache.org/jira/browse/HIVE-20850 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20850.01.patch, HIVE-20850.01.patch, > HIVE-20850.01.patch, HIVE-20850.02.patch > > > noticed by [~gopalv]: If there is a project which could be only evaluated > after the join; but the condition references only a single column from a > small dimension table; hive will end up evaluating the same thing over and > over again... > {code} > explain > select s_store_name, s_store_id, > sum(case when (d_day_name='Sunday') then ss_sales_price else null > end) sun_sales, > sum(case when (d_day_name='Monday') then ss_sales_price else null > end) mon_sales, > sum(case when (d_day_name='Tuesday') then ss_sales_price else null > end) tue_sales, > sum(case when (d_day_name='Wednesday') then ss_sales_price else null > end) wed_sales, > sum(case when (d_day_name='Thursday') then ss_sales_price else null > end) thu_sales, > sum(case when (d_day_name='Friday') then ss_sales_price else null > end) fri_sales, > sum(case when (d_day_name='Saturday') then ss_sales_price else null > end) sat_sales > from date_dim, store_sales, store > where d_date_sk = ss_sold_date_sk and >s_store_sk = ss_store_sk and >s_gmt_offset = -6 and >d_year = 1998 > group by s_store_name, s_store_id > order by s_store_name, > s_store_id,sun_sales,mon_sales,tue_sales,wed_sales,thu_sales,fri_sales,sat_sales > limit 100; > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20850) Push case conditional from projections to dimension tables if possible
[ https://issues.apache.org/jira/browse/HIVE-20850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674206#comment-16674206 ] Hive QA commented on HIVE-20850: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12946748/HIVE-20850.01.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14731/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14731/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14731/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-11-03 21:00:26.489 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-14731/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-11-03 21:00:26.492 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at ae1eb15 HIVE-20778: Join reordering may not be triggered if all joins in plan are created by decorrelation logic (Vineet Garg via Jesus Camacho Rodriguez) + git clean -f -d Removing standalone-metastore/metastore-server/src/gen/ + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at ae1eb15 HIVE-20778: Join reordering may not be triggered if all joins in plan are created by decorrelation logic (Vineet Garg via Jesus Camacho Rodriguez) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-11-03 21:00:27.167 + rm -rf ../yetus_PreCommit-HIVE-Build-14731 + mkdir ../yetus_PreCommit-HIVE-Build-14731 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-14731 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14731/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java:2065 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java' cleanly. error: patch failed: ql/src/test/results/clientpositive/perf/tez/cbo_query15.q.out:49 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/perf/tez/cbo_query15.q.out' with conflicts. Going to apply patch with: git apply -p0 /data/hiveptest/working/scratch/build.patch:219: trailing whitespace. group by ss_store_sk, /data/hiveptest/working/scratch/build.patch:226: trailing whitespace. from (select ss_item_sk item_sk,avg(ss_net_profit) rank_col /data/hiveptest/working/scratch/build.patch:250: trailing whitespace. where asceding.rnk = descending.rnk /data/hiveptest/working/scratch/build.patch:5119: trailing whitespace. group by ss_store_sk, /data/hiveptest/working/scratch/build.patch:5130: trailing whitespace. group by ss_store_sk, error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java:2065 Falling back to three-way merge... Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java' cleanly. error: patch failed: ql/src/test/results/clientpositive/perf/tez/cbo_query15.q.out:49 Falling back to three-way merge... Applied patch to 'ql/src/test/results/clientpositive/perf/tez/cbo_query15.q.out' with conflicts. U ql/src/test/results/clientpositive/perf/tez/cbo_query15.q.out warning: squelched 5 whitespace errors warning: 10 lines add whitespace errors. + result=1 + '[' 1 -ne 0 ']' + rm -rf yetus_PreCommit-HIVE-Build-14731 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12946748 - PreCommit-HIVE-Build > Push case conditional from projections to dimension tables if