[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[ https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270133#comment-14270133 ] Laljo John Pullokkaran commented on HIVE-8988: -- Committed to trunk, thanks [~jcamachorodriguez] Support advanced aggregation in Hive to Calcite path - Key: HIVE-8988 URL: https://issues.apache.org/jira/browse/HIVE-8988 Project: Hive Issue Type: Improvement Affects Versions: 0.15.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: grouping, logical, optiq Fix For: 0.15.0 Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch, HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.05.patch, HIVE-8988.05.patch, HIVE-8988.06.patch, HIVE-8988.07.patch, HIVE-8988.patch CLEAR LIBRARY CACHE To close the gap between Hive and Calcite, we need to support the translation of GroupingSets into Calcite; currently this is not implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[ https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14267979#comment-14267979 ] Laljo John Pullokkaran commented on HIVE-8988: -- +1 Support advanced aggregation in Hive to Calcite path - Key: HIVE-8988 URL: https://issues.apache.org/jira/browse/HIVE-8988 Project: Hive Issue Type: Improvement Affects Versions: 0.15.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: grouping, logical, optiq Fix For: 0.15.0 Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch, HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.05.patch, HIVE-8988.05.patch, HIVE-8988.06.patch, HIVE-8988.07.patch, HIVE-8988.patch CLEAR LIBRARY CACHE To close the gap between Hive and Calcite, we need to support the translation of GroupingSets into Calcite; currently this is not implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[ https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264845#comment-14264845 ] Jesus Camacho Rodriguez commented on HIVE-8988: --- [~jpullokkaran], test fail is not related to the patch. I think it is good to go, can you check it? Thanks! Support advanced aggregation in Hive to Calcite path - Key: HIVE-8988 URL: https://issues.apache.org/jira/browse/HIVE-8988 Project: Hive Issue Type: Improvement Affects Versions: 0.15.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: grouping, logical, optiq Fix For: 0.15.0 Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch, HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.05.patch, HIVE-8988.05.patch, HIVE-8988.06.patch, HIVE-8988.07.patch, HIVE-8988.patch CLEAR LIBRARY CACHE To close the gap between Hive and Calcite, we need to support the translation of GroupingSets into Calcite; currently this is not implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264833#comment-14264833
]
Hive QA commented on HIVE-8988:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12690095/HIVE-8988.07.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6724 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2254/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2254/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2254/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12690095 - PreCommit-HIVE-TRUNK-Build
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch,
HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.05.patch,
HIVE-8988.05.patch, HIVE-8988.06.patch, HIVE-8988.07.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14264671#comment-14264671
]
Hive QA commented on HIVE-8988:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12690076/HIVE-8988.06.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6724 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_grouping_operators
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2252/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2252/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2252/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12690076 - PreCommit-HIVE-TRUNK-Build
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch,
HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.05.patch,
HIVE-8988.05.patch, HIVE-8988.06.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[ https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14253269#comment-14253269 ] Jesus Camacho Rodriguez commented on HIVE-8988: --- If the Calcite library is refreshed in Hive, that test passes; the problem has been fixed in CALCITE-542. Support advanced aggregation in Hive to Calcite path - Key: HIVE-8988 URL: https://issues.apache.org/jira/browse/HIVE-8988 Project: Hive Issue Type: Improvement Affects Versions: 0.15.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: grouping, logical, optiq Fix For: 0.15.0 Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch, HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.05.patch, HIVE-8988.patch CLEAR LIBRARY CACHE To close the gap between Hive and Calcite, we need to support the translation of GroupingSets into Calcite; currently this is not implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[ https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14253730#comment-14253730 ] Julian Hyde commented on HIVE-8988: --- I've pushed a snapshot of Calcite that contains the fix to CALCITE-542. Support advanced aggregation in Hive to Calcite path - Key: HIVE-8988 URL: https://issues.apache.org/jira/browse/HIVE-8988 Project: Hive Issue Type: Improvement Affects Versions: 0.15.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: grouping, logical, optiq Fix For: 0.15.0 Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch, HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.05.patch, HIVE-8988.05.patch, HIVE-8988.patch CLEAR LIBRARY CACHE To close the gap between Hive and Calcite, we need to support the translation of GroupingSets into Calcite; currently this is not implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254222#comment-14254222
]
Hive QA commented on HIVE-8988:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12688371/HIVE-8988.05.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6719 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nonmr_fetch
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_virtual_column
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2145/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2145/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2145/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12688371 - PreCommit-HIVE-TRUNK-Build
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch,
HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.05.patch,
HIVE-8988.05.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[ https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14254547#comment-14254547 ] Jesus Camacho Rodriguez commented on HIVE-8988: --- Thanks [~julianhyde]. Tests fails are not related to the patch, I think it is ready to go. [~jpullokkaran] , what do you think? Support advanced aggregation in Hive to Calcite path - Key: HIVE-8988 URL: https://issues.apache.org/jira/browse/HIVE-8988 Project: Hive Issue Type: Improvement Affects Versions: 0.15.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: grouping, logical, optiq Fix For: 0.15.0 Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch, HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.05.patch, HIVE-8988.05.patch, HIVE-8988.patch CLEAR LIBRARY CACHE To close the gap between Hive and Calcite, we need to support the translation of GroupingSets into Calcite; currently this is not implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14252895#comment-14252895
]
Hive QA commented on HIVE-8988:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12688175/HIVE-8988.05.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6719 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_grouping_id2
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2136/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2136/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2136/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12688175 - PreCommit-HIVE-TRUNK-Build
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch,
HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.05.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250226#comment-14250226
]
Hive QA commented on HIVE-8988:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12687709/HIVE-8988.04.patch
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 6713 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_grouping_id2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_grouping_sets1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_grouping_sets2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_grouping_sets3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_grouping_sets5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_rollup1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_grouping_operators
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2111/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2111/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2111/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12687709 - PreCommit-HIVE-TRUNK-Build
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch,
HIVE-8988.03.patch, HIVE-8988.04.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[ https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14248652#comment-14248652 ] Laljo John Pullokkaran commented on HIVE-8988: -- [~jcamachorodriguez] could you replied the patch. Now after Sergeys patch CBO is on by default; we want to make sure all Cube/Rollup/GroupingSet tests succeed. Support advanced aggregation in Hive to Calcite path - Key: HIVE-8988 URL: https://issues.apache.org/jira/browse/HIVE-8988 Project: Hive Issue Type: Improvement Affects Versions: 0.15.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: grouping, logical, optiq Fix For: 0.15.0 Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch, HIVE-8988.02.patch, HIVE-8988.patch CLEAR LIBRARY CACHE To close the gap between Hive and Calcite, we need to support the translation of GroupingSets into Calcite; currently this is not implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14249396#comment-14249396
]
Hive QA commented on HIVE-8988:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12687542/HIVE-8988.03.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2099/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2099/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2099/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-2099/source-prep.txt
+ [[ true == \t\r\u\e ]]
+ rm -rf ivy maven
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'shims/0.20S/pom.xml'
Reverted 'shims/0.23/pom.xml'
Reverted 'shims/scheduler/pom.xml'
Reverted 'jdbc/pom.xml'
Reverted 'ql/pom.xml'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20S/target
shims/0.23/target shims/aggregator/target shims/common/target
shims/scheduler/target packaging/target hbase-handler/target testutils/target
jdbc/target metastore/target itests/target itests/hcatalog-unit/target
itests/test-serde/target itests/qtest/target itests/hive-unit-hadoop2/target
itests/hive-minikdc/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target hcatalog/core/target
hcatalog/streaming/target hcatalog/server-extensions/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target
hcatalog/hcatalog-pig-adapter/target accumulo-handler/target hwi/target
common/target common/src/gen contrib/target service/target serde/target
beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target
+ svn update
U
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java
U
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java
Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1646143.
Updated to revision 1646143.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12687542 - PreCommit-HIVE-TRUNK-Build
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.02.patch,
HIVE-8988.03.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14246548#comment-14246548
]
Hive QA commented on HIVE-8988:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12687218/HIVE-8988.02.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2079/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2079/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2079/
Messages:
{noformat}
This message was trimmed, see log for full details
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as LPAREN LPAREN KW_FALSE using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as LPAREN KW_NOT SmallintLiteral using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:115:5:
Decision can match input such as KW_CLUSTER KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:127:5:
Decision can match input such as KW_PARTITION KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:138:5:
Decision can match input such as KW_DISTRIBUTE KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:149:5:
Decision can match input such as KW_SORT KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:166:7:
Decision can match input such as STAR using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_ARRAY using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_STRUCT using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_UNIONTYPE using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_TRUE using multiple alternatives: 2, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_FALSE using multiple alternatives: 2, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_NULL using multiple alternatives: 1, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_CLUSTER
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_OVERWRITE using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_UNION
KW_ALL using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_INTO using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245408#comment-14245408
]
Hive QA commented on HIVE-8988:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12687044/HIVE-8988.02.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2074/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2074/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2074/
Messages:
{noformat}
This message was trimmed, see log for full details
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as LPAREN LPAREN KW_FALSE using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:68:4:
Decision can match input such as LPAREN KW_NOT SmallintLiteral using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:115:5:
Decision can match input such as KW_CLUSTER KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:127:5:
Decision can match input such as KW_PARTITION KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:138:5:
Decision can match input such as KW_DISTRIBUTE KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:149:5:
Decision can match input such as KW_SORT KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:166:7:
Decision can match input such as STAR using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_ARRAY using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_STRUCT using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_UNIONTYPE using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_TRUE using multiple alternatives: 2, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_FALSE using multiple alternatives: 2, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_NULL using multiple alternatives: 1, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_CLUSTER
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_OVERWRITE using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_UNION
KW_ALL using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_INTO using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243936#comment-14243936
]
Jesus Camacho Rodriguez commented on HIVE-8988:
---
[~julianhyde], thanks for the comments.
{quote}
I am a bit surprised that translating GROUPING__ID to Calcite's GROUP_ID
function works. The logic to expand it is in SqlToRelConverter, which Hive does
not call (IIRC). How does it work?
{quote}
It works because we never call the function. In fact, when we go back from
Calcite to Hive, the GROUP_ID call in the Calcite project operator is replaced
by a pointer to the column containing the GROUPING__ID that comes out of the
Hive GroupBy operator.
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[ https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244310#comment-14244310 ] Julian Hyde commented on HIVE-8988: --- You'd better make sure that that function isn't pushed down. Support advanced aggregation in Hive to Calcite path - Key: HIVE-8988 URL: https://issues.apache.org/jira/browse/HIVE-8988 Project: Hive Issue Type: Improvement Affects Versions: 0.15.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: grouping, logical, optiq Fix For: 0.15.0 Attachments: HIVE-8988.01.patch, HIVE-8988.patch CLEAR LIBRARY CACHE To close the gap between Hive and Calcite, we need to support the translation of GroupingSets into Calcite; currently this is not implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244390#comment-14244390
]
Jesus Camacho Rodriguez commented on HIVE-8988:
---
{quote}
You'd better make sure that that function isn't pushed down.
{quote}
I thought it would not be pushed through the GroupBy clause. Then, do you think
it is better to formulate it in the translation to Calcite as
grouping_id(e1,...) instead of group_id()? Then, as it is dependent on the
other columns, it will not go down for sure.
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244400#comment-14244400
]
Julian Hyde commented on HIVE-8988:
---
It looks just like a function with no arguments, so Calcite will treat
accordingly.
You can't translate to grouping_id because that's a pseudo function too. Safest
might be to translate it to {{$Foo(i0, i1, i2, i3, ...)}} where iin/i are
the indicator columns, and look for it afterwards. The column references will
prevent it from being pushed down. Putatively $Foo would expand to {{i0 + 2 *
i1 + 4 * i2 + ...}} (in fact the same logic that SqlToRelConverter translates
GROUP_ID to) but it will be substituted before that ever happens.
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244528#comment-14244528
]
Jesus Camacho Rodriguez commented on HIVE-8988:
---
{quote}
You can't translate to grouping_id because that's a pseudo function too.
{quote}
I am trying to understand why {{grouping_id}} would not be the right choice.
Since {{grouping_id}} has as arguments the column references {{i0, i1, i2, i3,
...}}, we are certain that it won't get pushed down, right? That should be
enough.
Sorry about insisting, I'm just trying to avoid creating another pseudo
function {{$Foo}} on my side if it is not really necessary.
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244535#comment-14244535
]
Julian Hyde commented on HIVE-8988:
---
If I wrote {{select f(deptno, gender), count(*) from emp group by deptno,
gender having f(deptno, gender) 100}} and f was a regular function, then
Calcite could push it down. And you'd probably want it to.
So we'd have to flag that grouping_id is not a regular function.
In my approach you apply the $Foo function to the indicator columns, which do
not exist until the Aggregate creates them, hence the function cannot be pushed
down. (The call to $Foo could still be pulled up.)
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244550#comment-14244550
]
Jesus Camacho Rodriguez commented on HIVE-8988:
---
OK, I was confused with the {{grouping_id}} function, as I thought the input
were the _indicator_ columns instead of the _grouping_ columns themselves. Now
I get your point.
Would that {{$foo}} function be something interesting to have in Calcite
itself? Meaning, a function that given _n_ booleans, computes the integer value
as {{i0 + 2 * i1 + 4 * i2 + ...}}? Otherwise, I'll implement it and keep it in
Hive.
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.01.patch, HIVE-8988.patch
CLEAR LIBRARY CACHE
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[ https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244585#comment-14244585 ] Julian Hyde commented on HIVE-8988: --- $Foo is probably not interesting to end-users, so let's keep it in Hive. You should sub-class SqlInternalOperator. In SqlStdOperatorTable, you'll see $SLICE, $ELEMENT_SLICE, $SCALAR_QUERY and EXTEND are all examples of it. Support advanced aggregation in Hive to Calcite path - Key: HIVE-8988 URL: https://issues.apache.org/jira/browse/HIVE-8988 Project: Hive Issue Type: Improvement Affects Versions: 0.15.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: grouping, logical, optiq Fix For: 0.15.0 Attachments: HIVE-8988.01.patch, HIVE-8988.patch CLEAR LIBRARY CACHE To close the gap between Hive and Calcite, we need to support the translation of GroupingSets into Calcite; currently this is not implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245157#comment-14245157
]
Hive QA commented on HIVE-8988:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12686821/HIVE-8988.01.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2065/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2065/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2065/
Messages:
{noformat}
This message was trimmed, see log for full details
warning(200): IdentifiersParser.g:127:5:
Decision can match input such as KW_PARTITION KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:138:5:
Decision can match input such as KW_DISTRIBUTE KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:149:5:
Decision can match input such as KW_SORT KW_BY LPAREN using multiple
alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:166:7:
Decision can match input such as STAR using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_ARRAY using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_STRUCT using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_UNIONTYPE using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_TRUE using multiple alternatives: 2, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_FALSE using multiple alternatives: 2, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_NULL using multiple alternatives: 1, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_CLUSTER
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_OVERWRITE using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_UNION
KW_ALL using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_INTO using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as KW_BETWEEN KW_MAP LPAREN using multiple
alternatives: 8, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_LATERAL
KW_VIEW using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER
KW_BY using multiple alternatives: 2, 9
As a
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[ https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14242596#comment-14242596 ] Jesus Camacho Rodriguez commented on HIVE-8988: --- [~jpullokkaran], [~julianhyde], can you take a look? Support advanced aggregation in Hive to Calcite path - Key: HIVE-8988 URL: https://issues.apache.org/jira/browse/HIVE-8988 Project: Hive Issue Type: Improvement Affects Versions: 0.15.0 Reporter: Jesus Camacho Rodriguez Assignee: Jesus Camacho Rodriguez Labels: grouping, logical, optiq Fix For: 0.15.0 Attachments: HIVE-8988.patch To close the gap between Hive and Calcite, we need to support the translation of GroupingSets into Calcite; currently this is not implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243272#comment-14243272
]
Hive QA commented on HIVE-8988:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12686661/HIVE-8988.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2045/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2045/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2045/
Messages:
{noformat}
This message was trimmed, see log for full details
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:166:7:
Decision can match input such as STAR using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_ARRAY using multiple alternatives: 2, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_STRUCT using multiple alternatives: 4, 6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:179:5:
Decision can match input such as KW_UNIONTYPE using multiple alternatives: 5,
6
As a result, alternative(s) 6 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_TRUE using multiple alternatives: 2, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_FALSE using multiple alternatives: 2, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:270:5:
Decision can match input such as KW_NULL using multiple alternatives: 1, 7
As a result, alternative(s) 7 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_SORT KW_BY
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_MAP LPAREN
using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_GROUP
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_CLUSTER
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_DISTRIBUTE
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_OVERWRITE using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_UNION
KW_ALL using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_INSERT
KW_INTO using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as KW_BETWEEN KW_MAP LPAREN using multiple
alternatives: 8, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_LATERAL
KW_VIEW using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:401:5:
Decision can match input such as {KW_LIKE, KW_REGEXP, KW_RLIKE} KW_ORDER
KW_BY using multiple alternatives: 2, 9
As a result, alternative(s) 9 were disabled for that input
warning(200): IdentifiersParser.g:526:5:
Decision can match input such as {AMPERSAND..BITWISEXOR, DIV..DIVIDE,
EQUAL..EQUAL_NS, GREATERTHAN..GREATERTHANOREQUALTO, KW_AND, KW_ARRAY,
KW_BETWEEN..KW_BOOLEAN, KW_CASE, KW_DOUBLE, KW_FLOAT, KW_IF, KW_IN, KW_INT,
KW_LIKE, KW_MAP, KW_NOT, KW_OR, KW_REGEXP, KW_RLIKE, KW_SMALLINT,
KW_STRING..KW_STRUCT, KW_TINYINT, KW_UNIONTYPE, KW_WHEN,
LESSTHAN..LESSTHANOREQUALTO, MINUS..NOTEQUAL, PLUS, STAR, TILDE} using
multiple
[jira] [Commented] (HIVE-8988) Support advanced aggregation in Hive to Calcite path
[
https://issues.apache.org/jira/browse/HIVE-8988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243533#comment-14243533
]
Julian Hyde commented on HIVE-8988:
---
I am a bit surprised that translating GROUPING__ID to Calcite's GROUP_ID
function works. The logic to expand it is in SqlToRelConverter, which Hive does
not call (IIRC). How does it work?
You don't need to call BitSets.toIter anymore. ImmutableBitSet implements
Iterable (BitSet, which we used to use for Aggregate.groupKeys, does not).
For {{ImmutableBitSet convert(int value)}}, I'll add
ImmutableBitSet.valueOf(long...). valueOf(x) would be equivalent to convert(x).
You can migrate to it in a future version of Calcite, if you like.
Support advanced aggregation in Hive to Calcite path
-
Key: HIVE-8988
URL: https://issues.apache.org/jira/browse/HIVE-8988
Project: Hive
Issue Type: Improvement
Affects Versions: 0.15.0
Reporter: Jesus Camacho Rodriguez
Assignee: Jesus Camacho Rodriguez
Labels: grouping, logical, optiq
Fix For: 0.15.0
Attachments: HIVE-8988.patch
To close the gap between Hive and Calcite, we need to support the translation
of GroupingSets into Calcite; currently this is not implemented.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
