[jira] [Commented] (HIVE-16164) Provide mechanism for passing HMS notification ID between transactional and non-transactional listeners.
[
https://issues.apache.org/jira/browse/HIVE-16164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990163#comment-15990163
]
Lefty Leverenz commented on HIVE-16164:
---
Okay, thanks Sergio.
> Provide mechanism for passing HMS notification ID between transactional and
> non-transactional listeners.
>
>
> Key: HIVE-16164
> URL: https://issues.apache.org/jira/browse/HIVE-16164
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Sergio Peña
>Assignee: Sergio Peña
> Fix For: 2.3.0, 3.0.0, 2.4.0
>
> Attachments: HIVE-16164.1.patch, HIVE-16164.2.patch,
> HIVE-16164.3.patch, HIVE-16164.6.patch, HIVE-16164.7.patch, HIVE-16164.8.patch
>
>
> The HMS DB notification listener currently stores an event ID on the HMS
> backend DB so that external applications (such as backup apps) can request
> incremental notifications based on the last event ID requested.
> The HMS DB notification and backup applications are asynchronous. However,
> there are sometimes that applications may be required to be in sync with the
> latest HMS event in order to process an action. These applications will
> provide a listener implementation that is called by the HMS after an HMS
> transaction happened.
> The problem is that the listener running after the transaction (or during the
> non-transactional context) may need the DB event ID in order to sync all
> events happened previous to that event ID, but this ID is never passed to the
> non-transactional listeners.
> We can pass this event information through the EnvironmentContext found on
> each ListenerEvent implementations (such as CreateTableEvent), and send the
> EnvironmentContext to the non-transactional listeners to get the event ID.
> The DbNotificactionListener already knows the event ID after calling the
> ObjectStore.addNotificationEvent(). We just need to set this event ID to the
> EnvironmentContext from each of the event notifications and make sure that
> this EnvironmentContext is sent to the non-transactional listeners.
> Here's the code example when creating a table on {{create_table_core}}:
> {noformat}
> ms.createTable(tbl);
> if (transactionalListeners.size() > 0) {
> CreateTableEvent createTableEvent = new CreateTableEvent(tbl, true, this);
> createTableEvent.setEnvironmentContext(envContext);
> for (MetaStoreEventListener transactionalListener :
> transactionalListeners) {
> transactionalListener.onCreateTable(createTableEvent); // <-
> Here the notification ID is generated
> }
> }
> success = ms.commitTransaction();
> } finally {
> if (!success) {
> ms.rollbackTransaction();
> if (madeDir) {
> wh.deleteDir(tblPath, true);
> }
> }
> for (MetaStoreEventListener listener : listeners) {
> CreateTableEvent createTableEvent =
> new CreateTableEvent(tbl, success, this);
> createTableEvent.setEnvironmentContext(envContext);
> listener.onCreateTable(createTableEvent);// <-
> Here we would like to consume notification ID
> }
> {noformat}
> We could use a specific key name that will be used on the EnvironmentContext,
> such as DB_NOTIFICATION_EVENT_ID.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15321) Change to read as long for HiveConf.ConfVars.METASTORESERVERMAXMESSAGESIZE
[
https://issues.apache.org/jira/browse/HIVE-15321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990165#comment-15990165
]
Lefty Leverenz commented on HIVE-15321:
---
Thanks, Aihua Xu.
> Change to read as long for HiveConf.ConfVars.METASTORESERVERMAXMESSAGESIZE
> --
>
> Key: HIVE-15321
> URL: https://issues.apache.org/jira/browse/HIVE-15321
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 1.2.0, 1.1.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Fix For: 2.2.0
>
> Attachments: HIVE-15321.1.patch, HIVE-15321.2.patch
>
>
> Follow up on HIVE-11240 which tries to change the type from int to long,
> while we are still read with {{conf.getIntVar()}}.
> Seems we should use {{conf.getLongVar()}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-14431) Recognize COALESCE as CASE
[
https://issues.apache.org/jira/browse/HIVE-14431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990261#comment-15990261
]
Remus Rusanu commented on HIVE-14431:
-
Since the original path from Jesus the simplify rules have moved into Calcite
(RexSimplify). We have to follow up as a Calcite issue, and see if they
desire/accept this rule.
An alternative is to do something similar to the
{{HivePointLookupOptimizerRule}}, ie. have a RelOptRule for Filter/Join
operators and inspect the condition for COALESCE => CASE simplification. This
would be entirely in Hive space.
> Recognize COALESCE as CASE
> --
>
> Key: HIVE-14431
> URL: https://issues.apache.org/jira/browse/HIVE-14431
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Remus Rusanu
> Attachments: HIVE-14431.01.patch, HIVE-14431.2.patch, HIVE-14431.patch
>
>
> Transform:
> {code}
> (COALESCE(a, '') = '') OR
>(a = 'A' AND b = c) OR
>(a = 'B' AND b = d) OR
>(a = 'C' AND b = e) OR
>(a = 'D' AND b = f) OR
>(a = 'E' AND b = g) OR
>(a = 'F' AND b = h)
> {code}
> into:
> {code}
> (a='') OR
>(a is null) OR
>(a = 'A' AND b = c) OR
>(a = 'B' AND b = d) OR
>(a = 'C' AND b = e) OR
>(a = 'D' AND b = f) OR
>(a = 'E' AND b = g) OR
>(a = 'F' AND b = h)
> {code}
> With complex queries, this will lead us to factor more predicates that could
> be pushed to the TS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-14431) Recognize COALESCE as CASE
[
https://issues.apache.org/jira/browse/HIVE-14431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990267#comment-15990267
]
Hive QA commented on HIVE-14431:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12823995/HIVE-14431.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4947/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4947/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4947/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-04-30 15:41:45.514
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-4947/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-04-30 15:41:45.517
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 66ff590 HIVE-16553: Change default value for
hive.tez.bigtable.minsize.semijoin.reduction (Jason Dere, reviewed by gopalv)
+ git clean -f -d
Removing
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSortProjectRealignRule.java
Removing ql/src/test/queries/clientpositive/order_by_expr_1.q
Removing ql/src/test/queries/clientpositive/order_by_expr_2.q
Removing ql/src/test/results/clientpositive/order_by_expr_1.q.out
Removing ql/src/test/results/clientpositive/order_by_expr_2.q.out
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 66ff590 HIVE-16553: Change default value for
hive.tez.bigtable.minsize.semijoin.reduction (Jason Dere, reviewed by gopalv)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-04-30 15:41:47.254
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRexUtil.java: No
such file or directory
error:
a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java:
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12823995 - PreCommit-HIVE-Build
> Recognize COALESCE as CASE
> --
>
> Key: HIVE-14431
> URL: https://issues.apache.org/jira/browse/HIVE-14431
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Remus Rusanu
> Attachments: HIVE-14431.01.patch, HIVE-14431.2.patch, HIVE-14431.patch
>
>
> Transform:
> {code}
> (COALESCE(a, '') = '') OR
>(a = 'A' AND b = c) OR
>(a = 'B' AND b = d) OR
>(a = 'C' AND b = e) OR
>(a = 'D' AND b = f) OR
>(a = 'E' AND b = g) OR
>(a = 'F' AND b = h)
> {code}
> into:
> {code}
> (a='') OR
>(a is null) OR
>(a = 'A' AND b = c) OR
>(a = 'B' AND b = d) OR
>(a = 'C' AND b = e) OR
>(a = 'D' AND b = f) OR
>(a = 'E' AND b = g) OR
>(a = 'F' AND b = h)
> {code}
> With complex queries, this will lead us to factor more predicates that could
> be pushed to the TS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-14431) Recognize COALESCE as CASE
[
https://issues.apache.org/jira/browse/HIVE-14431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Remus Rusanu updated HIVE-14431:
Status: Open (was: Patch Available)
> Recognize COALESCE as CASE
> --
>
> Key: HIVE-14431
> URL: https://issues.apache.org/jira/browse/HIVE-14431
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Remus Rusanu
> Attachments: HIVE-14431.01.patch, HIVE-14431.2.patch, HIVE-14431.patch
>
>
> Transform:
> {code}
> (COALESCE(a, '') = '') OR
>(a = 'A' AND b = c) OR
>(a = 'B' AND b = d) OR
>(a = 'C' AND b = e) OR
>(a = 'D' AND b = f) OR
>(a = 'E' AND b = g) OR
>(a = 'F' AND b = h)
> {code}
> into:
> {code}
> (a='') OR
>(a is null) OR
>(a = 'A' AND b = c) OR
>(a = 'B' AND b = d) OR
>(a = 'C' AND b = e) OR
>(a = 'D' AND b = f) OR
>(a = 'E' AND b = g) OR
>(a = 'F' AND b = h)
> {code}
> With complex queries, this will lead us to factor more predicates that could
> be pushed to the TS.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16527) Support outer and mixed reference aggregates in windowed functions
[
https://issues.apache.org/jira/browse/HIVE-16527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990271#comment-15990271
]
Ashutosh Chauhan commented on HIVE-16527:
-
+1
> Support outer and mixed reference aggregates in windowed functions
> --
>
> Key: HIVE-16527
> URL: https://issues.apache.org/jira/browse/HIVE-16527
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Reporter: Remus Rusanu
>Assignee: Remus Rusanu
> Attachments: HIVE-16527.00.patch, HIVE-16527.02.patch,
> HIVE-16527.03.patch
>
>
> {noformat}
> select sum(sum(c1)) over() from e011_01;
> select sum(sum(c1)) over(partition by c2 order by c1) from e011_01 group by
> e011_01.c1, e011_01.c2;
> select sum(sum(e011_01.c1)) over(partition by e011_01.c2 order by e011_01.c1)
> from e011_01 join e011_03 on e011_01.c1 = e011_03.c1 group by e011_01.c1,
> e011_01.c2;
> select sum(sum(e011_01.c1)) over(partition by e011_03.c2 order by e011_03.c1)
> from e011_01 join e011_03 on e011_01.c1 = e011_03.c1 group by e011_03.c1,
> e011_03.c2;
> select sum(corr(e011_01.c1, e011_03.c1)) over(partition by e011_01.c2 order
> by e011_03.c2) from e011_01 join e011_03 on e011_01.c1 = e011_03.c1 group by
> e011_03.c2, e011_01.c2;
> {noformat}
> We fail to generate a plan for any of the above. The issue is that in
> {{SemanticAnalyzer.doPhase1GetAllAggregations}}, for {{TOK_WINDOWSPEC}} we
> ignore all children except the last (the window spec child). Additionally the
> typecheck processor is not prepared to encounter UDAF expressions
> ({{TypeCheckProcFactory.DefaultExpreProcessor.validateUDF}},
> {{getXpathOrFuncExprNodeDesc}}).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16527) Support outer and mixed reference aggregates in windowed functions
[
https://issues.apache.org/jira/browse/HIVE-16527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Remus Rusanu updated HIVE-16527:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
https://git1-us-west.apache.org/repos/asf?p=hive.git;a=commit;h=dac3786d86462e4d08d62d23115e6b7a3e534f5d
> Support outer and mixed reference aggregates in windowed functions
> --
>
> Key: HIVE-16527
> URL: https://issues.apache.org/jira/browse/HIVE-16527
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Reporter: Remus Rusanu
>Assignee: Remus Rusanu
> Fix For: 3.0.0
>
> Attachments: HIVE-16527.00.patch, HIVE-16527.02.patch,
> HIVE-16527.03.patch
>
>
> {noformat}
> select sum(sum(c1)) over() from e011_01;
> select sum(sum(c1)) over(partition by c2 order by c1) from e011_01 group by
> e011_01.c1, e011_01.c2;
> select sum(sum(e011_01.c1)) over(partition by e011_01.c2 order by e011_01.c1)
> from e011_01 join e011_03 on e011_01.c1 = e011_03.c1 group by e011_01.c1,
> e011_01.c2;
> select sum(sum(e011_01.c1)) over(partition by e011_03.c2 order by e011_03.c1)
> from e011_01 join e011_03 on e011_01.c1 = e011_03.c1 group by e011_03.c1,
> e011_03.c2;
> select sum(corr(e011_01.c1, e011_03.c1)) over(partition by e011_01.c2 order
> by e011_03.c2) from e011_01 join e011_03 on e011_01.c1 = e011_03.c1 group by
> e011_03.c2, e011_01.c2;
> {noformat}
> We fail to generate a plan for any of the above. The issue is that in
> {{SemanticAnalyzer.doPhase1GetAllAggregations}}, for {{TOK_WINDOWSPEC}} we
> ignore all children except the last (the window spec child). Additionally the
> typecheck processor is not prepared to encounter UDAF expressions
> ({{TypeCheckProcFactory.DefaultExpreProcessor.validateUDF}},
> {{getXpathOrFuncExprNodeDesc}}).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Attachment: HIVE-16557.01.patch > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.01.patch.tar.gz > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Status: Patch Available (was: Open) > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.01.patch.tar.gz > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[
https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990314#comment-15990314
]
Hive QA commented on HIVE-16557:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12865709/HIVE-16557.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4948/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4948/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4948/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-04-30 17:41:54.784
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-4948/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-04-30 17:41:54.787
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
66ff590..dac3786 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 66ff590 HIVE-16553: Change default value for
hive.tez.bigtable.minsize.semijoin.reduction (Jason Dere, reviewed by gopalv)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at dac3786 HIVE-16527 Support outer and mixed reference aggregates
in windowed functions (Remus Rusanu, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-04-30 17:41:55.760
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: patch -p0
patching file itests/src/test/resources/testconfiguration.properties
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkCommonOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkEmptyKeyOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkLongOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkMultiKeyOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkObjectHashOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkStringOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkUniformHashOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
patching file ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
Hunk #1 succeeded at 13195 (offset 4 lines).
Hunk #2 succeeded at 13217 (offset 4 lines).
Hunk #3 succeeded at 13362 (offset 4 lines).
Hunk #4 succeeded at 13376 (offset 4 lines).
patching file ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/plan/VectorReduceSinkDesc.java
patching file ql/src/test/queries/clientpositive/vector_windowing_navfn.q
patching file ql/src/test/queries/clientpositive/vectorization_limit.q
patching file ql/src/test/queries/clientpositive/windowing_navfn.q
patching file
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_id3.q.out
patching file ql/src/test/results/clientpositive/llap/vector_inner_join.q.out
patching file
ql/src/test/results/clientpositive/llap/vector_join_part_col_char.q.out
patching file
ql/src/test/results/clientpositive/llap/vector_leftsemi_mapjoin.q.out
patching file
ql/src/test/results/
[jira] [Updated] (HIVE-15160) Can't order by an unselected column
[
https://issues.apache.org/jira/browse/HIVE-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-15160:
---
Attachment: HIVE-15160.13.patch
> Can't order by an unselected column
> ---
>
> Key: HIVE-15160
> URL: https://issues.apache.org/jira/browse/HIVE-15160
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15160.01.patch, HIVE-15160.02.patch,
> HIVE-15160.04.patch, HIVE-15160.05.patch, HIVE-15160.06.patch,
> HIVE-15160.07.patch, HIVE-15160.08.patch, HIVE-15160.09.patch,
> HIVE-15160.09.patch, HIVE-15160.10.patch, HIVE-15160.11.patch,
> HIVE-15160.12.patch, HIVE-15160.13.patch
>
>
> If a grouping key hasn't been selected, Hive complains. For comparison,
> Postgres does not.
> Example. Notice i_item_id is not selected:
> {code}
> select i_item_desc
>,i_category
>,i_class
>,i_current_price
>,sum(cs_ext_sales_price) as itemrevenue
>,sum(cs_ext_sales_price)*100/sum(sum(cs_ext_sales_price)) over
>(partition by i_class) as revenueratio
> from catalog_sales
> ,item
> ,date_dim
> where cs_item_sk = i_item_sk
>and i_category in ('Jewelry', 'Sports', 'Books')
>and cs_sold_date_sk = d_date_sk
> and d_date between cast('2001-01-12' as date)
> and (cast('2001-01-12' as date) + 30 days)
> group by i_item_id
> ,i_item_desc
> ,i_category
> ,i_class
> ,i_current_price
> order by i_category
> ,i_class
> ,i_item_id
> ,i_item_desc
> ,revenueratio
> limit 100;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15160) Can't order by an unselected column
[
https://issues.apache.org/jira/browse/HIVE-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-15160:
---
Status: Open (was: Patch Available)
> Can't order by an unselected column
> ---
>
> Key: HIVE-15160
> URL: https://issues.apache.org/jira/browse/HIVE-15160
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15160.01.patch, HIVE-15160.02.patch,
> HIVE-15160.04.patch, HIVE-15160.05.patch, HIVE-15160.06.patch,
> HIVE-15160.07.patch, HIVE-15160.08.patch, HIVE-15160.09.patch,
> HIVE-15160.09.patch, HIVE-15160.10.patch, HIVE-15160.11.patch,
> HIVE-15160.12.patch, HIVE-15160.13.patch
>
>
> If a grouping key hasn't been selected, Hive complains. For comparison,
> Postgres does not.
> Example. Notice i_item_id is not selected:
> {code}
> select i_item_desc
>,i_category
>,i_class
>,i_current_price
>,sum(cs_ext_sales_price) as itemrevenue
>,sum(cs_ext_sales_price)*100/sum(sum(cs_ext_sales_price)) over
>(partition by i_class) as revenueratio
> from catalog_sales
> ,item
> ,date_dim
> where cs_item_sk = i_item_sk
>and i_category in ('Jewelry', 'Sports', 'Books')
>and cs_sold_date_sk = d_date_sk
> and d_date between cast('2001-01-12' as date)
> and (cast('2001-01-12' as date) + 30 days)
> group by i_item_id
> ,i_item_desc
> ,i_category
> ,i_class
> ,i_current_price
> order by i_category
> ,i_class
> ,i_item_id
> ,i_item_desc
> ,revenueratio
> limit 100;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15160) Can't order by an unselected column
[
https://issues.apache.org/jira/browse/HIVE-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-15160:
---
Status: Patch Available (was: Open)
> Can't order by an unselected column
> ---
>
> Key: HIVE-15160
> URL: https://issues.apache.org/jira/browse/HIVE-15160
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15160.01.patch, HIVE-15160.02.patch,
> HIVE-15160.04.patch, HIVE-15160.05.patch, HIVE-15160.06.patch,
> HIVE-15160.07.patch, HIVE-15160.08.patch, HIVE-15160.09.patch,
> HIVE-15160.09.patch, HIVE-15160.10.patch, HIVE-15160.11.patch,
> HIVE-15160.12.patch, HIVE-15160.13.patch
>
>
> If a grouping key hasn't been selected, Hive complains. For comparison,
> Postgres does not.
> Example. Notice i_item_id is not selected:
> {code}
> select i_item_desc
>,i_category
>,i_class
>,i_current_price
>,sum(cs_ext_sales_price) as itemrevenue
>,sum(cs_ext_sales_price)*100/sum(sum(cs_ext_sales_price)) over
>(partition by i_class) as revenueratio
> from catalog_sales
> ,item
> ,date_dim
> where cs_item_sk = i_item_sk
>and i_category in ('Jewelry', 'Sports', 'Books')
>and cs_sold_date_sk = d_date_sk
> and d_date between cast('2001-01-12' as date)
> and (cast('2001-01-12' as date) + 30 days)
> group by i_item_id
> ,i_item_desc
> ,i_category
> ,i_class
> ,i_current_price
> order by i_category
> ,i_class
> ,i_item_id
> ,i_item_desc
> ,revenueratio
> limit 100;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16147) Rename a partitioned table should not drop its partition columns stats
[
https://issues.apache.org/jira/browse/HIVE-16147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990334#comment-15990334
]
Pengcheng Xiong commented on HIVE-16147:
+1
> Rename a partitioned table should not drop its partition columns stats
> --
>
> Key: HIVE-16147
> URL: https://issues.apache.org/jira/browse/HIVE-16147
> Project: Hive
> Issue Type: Bug
>Reporter: Chaoyu Tang
>Assignee: Chaoyu Tang
> Attachments: HIVE-16147.1.patch, HIVE-16147.patch, HIVE-16147.patch
>
>
> When a partitioned table (e.g. sample_pt) is renamed (e.g to
> sample_pt_rename), describing its partition shows that the partition column
> stats are still accurate, but actually they all have been dropped.
> It could be reproduce as following:
> 1. analyze table sample_pt compute statistics for columns;
> 2. describe formatted default.sample_pt partition (dummy = 3): COLUMN_STATS
> for all columns are true
> {code}
> ...
> # Detailed Partition Information
> Partition Value: [3]
> Database: default
> Table:sample_pt
> CreateTime: Fri Jan 20 15:42:30 EST 2017
> LastAccessTime: UNKNOWN
> Location: file:/user/hive/warehouse/apache/sample_pt/dummy=3
> Partition Parameters:
> COLUMN_STATS_ACCURATE
> {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"code\":\"true\",\"description\":\"true\",\"salary\":\"true\",\"total_emp\":\"true\"}}
> last_modified_byctang
> last_modified_time 1485217063
> numFiles1
> numRows 100
> rawDataSize 5143
> totalSize 5243
> transient_lastDdlTime 1488842358
> ...
> {code}
> 3: describe formatted default.sample_pt partition (dummy = 3) salary: column
> stats exists
> {code}
> # col_namedata_type min
> max num_nulls distinct_count
> avg_col_len max_col_len num_trues
> num_falses comment
>
>
> salaryint 1 151370
> 0 94
>
> from deserializer
> {code}
> 4. alter table sample_pt rename to sample_pt_rename;
> 5. describe formatted default.sample_pt_rename partition (dummy = 3):
> describe the rename table partition (dummy =3) shows that COLUMN_STATS for
> columns are still true.
> {code}
> # Detailed Partition Information
> Partition Value: [3]
> Database: default
> Table:sample_pt_rename
> CreateTime: Fri Jan 20 15:42:30 EST 2017
> LastAccessTime: UNKNOWN
> Location:
> file:/user/hive/warehouse/apache/sample_pt_rename/dummy=3
> Partition Parameters:
> COLUMN_STATS_ACCURATE
> {\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"code\":\"true\",\"description\":\"true\",\"salary\":\"true\",\"total_emp\":\"true\"}}
> last_modified_byctang
> last_modified_time 1485217063
> numFiles1
> numRows 100
> rawDataSize 5143
> totalSize 5243
> transient_lastDdlTime 1488842358
> {code}
> describe formatted default.sample_pt_rename partition (dummy = 3) salary: the
> column stats have been dropped.
> {code}
> # col_namedata_type comment
>
>
>
> salaryint from deserializer
>
> Time taken: 0.131 seconds, Fetched: 3 row(s)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13811) Constant not removed in index_auto_unused.q.out
[
https://issues.apache.org/jira/browse/HIVE-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990336#comment-15990336
]
Remus Rusanu commented on HIVE-13811:
-
Original plant (before HIVE-13608):
{noformat}
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
-TableScan
- alias: srcpart
- filterExpr: (UDFToDouble(key) < 10.0) (type: boolean)
- Statistics: Num rows: 500 Data size: 5312 Basic stats: COMPLETE
Column stats: NONE
- Filter Operator
-predicate: (UDFToDouble(key) < 10.0) (type: boolean)
-Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE
Column stats: NONE
-Select Operator
- expressions: key (type: string), value (type: string),
'2008-04-09' (type: string), hr (type: string)
- outputColumnNames: _col0, _col1, _col2, _col3
- Statistics: Num rows: 166 Data size: 1763 Basic stats: COMPLETE
Column stats: NONE
- ListSink
{noformat}
> Constant not removed in index_auto_unused.q.out
> ---
>
> Key: HIVE-13811
> URL: https://issues.apache.org/jira/browse/HIVE-13811
> Project: Hive
> Issue Type: Sub-task
> Components: CBO
>Affects Versions: 2.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Remus Rusanu
>
> Follow-up on HIVE-13068.
> In test file ql/src/test/results/clientpositive/index_auto_unused.q.out.
> After HIVE-13068 goes in, the following filter is not folded after
> PartitionPruning is done:
> {{filterExpr: ((ds = '2008-04-09') and (12.0 = 12.0) and (UDFToDouble(key) <
> 10.0)) (type: boolean)}}
> Further, SimpleFetchOptimizer got disabled.
> All this needs further investigation.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-15160) Can't order by an unselected column
[
https://issues.apache.org/jira/browse/HIVE-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990359#comment-15990359
]
Hive QA commented on HIVE-15160:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12865714/HIVE-15160.13.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 31 failed/errored test(s), 10638 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_index]
(batchId=225)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join8] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join8] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_2]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_windowing1]
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin4] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_25]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_unqualcolumnrefs]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_null_projection]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_duplicate]
(batchId=30)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_groupby]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[limit_pushdown3]
(batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[limit_pushdown]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[selectDistinctStar]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_coalesce]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_date_1]
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_2]
(batchId=148)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_round]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_grouping_sets_grouping]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_grouping_sets_limit]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_interval_1]
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_interval_arithmetic]
(batchId=142)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join8]
(batchId=134)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join8] (batchId=118)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_25]
(batchId=102)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_short_regress]
(batchId=118)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4949/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4949/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4949/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 31 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12865714 - PreCommit-HIVE-Build
> Can't order by an unselected column
> ---
>
> Key: HIVE-15160
> URL: https://issues.apache.org/jira/browse/HIVE-15160
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15160.01.patch, HIVE-15160.02.patch,
> HIVE-15160.04.patch, HIVE-15160.05.patch, HIVE-15160.06.patch,
> HIVE-15160.07.patch, HIVE-15160.08.patch, HIVE-15160.09.patch,
> HIVE-15160.09.patch, HIVE-15160.10.patch, HIVE-15160.11.patch,
> HIVE-15160.12.patch, HIVE-15160.13.patch
>
>
> If a grouping key hasn't been selected, Hive complains. For comparison,
> Postgres does not.
> Example. Notice i_item_id is not selected:
> {code}
> select i_item_desc
>
[jira] [Updated] (HIVE-16369) Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only)
[ https://issues.apache.org/jira/browse/HIVE-16369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16369: Attachment: HIVE-16369.09.patch > Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only) > - > > Key: HIVE-16369 > URL: https://issues.apache.org/jira/browse/HIVE-16369 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16369.01.patch, HIVE-16369.02.patch, > HIVE-16369.04.patch, HIVE-16369.05.patch.tar.gz, HIVE-16369.06.patch, > HIVE-16369.07.patch, HIVE-16369.09.patch > > > Support vectorization of PTF that doesn't include custom PRECEDING / > FOLLOWING window frame clauses. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16369) Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only)
[ https://issues.apache.org/jira/browse/HIVE-16369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16369: Status: Patch Available (was: In Progress) > Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only) > - > > Key: HIVE-16369 > URL: https://issues.apache.org/jira/browse/HIVE-16369 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16369.01.patch, HIVE-16369.02.patch, > HIVE-16369.04.patch, HIVE-16369.05.patch.tar.gz, HIVE-16369.06.patch, > HIVE-16369.07.patch, HIVE-16369.09.patch > > > Support vectorization of PTF that doesn't include custom PRECEDING / > FOLLOWING window frame clauses. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16369) Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only)
[
https://issues.apache.org/jira/browse/HIVE-16369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990388#comment-15990388
]
Hive QA commented on HIVE-16369:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12865724/HIVE-16369.09.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 49 failed/errored test(s), 10647 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_index]
(batchId=225)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[outer_reference_windowed]
(batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_ptf_part_simple]
(batchId=148)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_expressions]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_gby2]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_gby]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_multipartitioning]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_range_multiorder]
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_rank]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_streaming]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_windowspec4]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_ptf]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join1]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join2]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join3]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join4]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join5]
(batchId=167)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_resolution]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ptf] (batchId=105)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ptf_streaming]
(batchId=120)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_in]
(batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_6_subq]
(batchId=114)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_between_in]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_count_distinct]
(batchId=109)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_data_types]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_decimal_aggregate]
(batchId=106)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_distinct_2]
(batchId=120)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_groupby_3]
(batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_left_outer_join]
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_mapjoin_reduce]
(batchId=132)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_orderby_5]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_string_concat]
(batchId=112)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_0]
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_5]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_nested_udf]
(batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_part]
(batchId=111)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_short_regress]
(batchId=118)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_case]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_nested_mapjoin]
(batchId=105)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf]
(batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_timestamp_funcs]
(batchId=111)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[windowing]
(batchId=120)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=235)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValid (batchId=235)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValidNeg (batchId=235)
org.apache.hive.minikdc.T
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Attachment: (was: HIVE-16557.01.patch.tar.gz) > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Attachment: HIVE-16557.02.patch > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Status: In Progress (was: Patch Available) > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Status: Patch Available (was: In Progress) > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[
https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990534#comment-15990534
]
Hive QA commented on HIVE-16557:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12865727/HIVE-16557.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4951/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4951/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4951/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-05-01 01:21:38.966
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-4951/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-05-01 01:21:38.969
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at dac3786 HIVE-16527 Support outer and mixed reference aggregates
in windowed functions (Remus Rusanu, reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ptf/
Removing ql/src/java/org/apache/hadoop/hive/ql/plan/VectorPTFDesc.java
Removing ql/src/java/org/apache/hadoop/hive/ql/plan/VectorPTFInfo.java
Removing ql/src/test/queries/clientpositive/vector_windowing.q
Removing ql/src/test/queries/clientpositive/vector_windowing_expressions.q
Removing ql/src/test/queries/clientpositive/vector_windowing_gby.q
Removing ql/src/test/queries/clientpositive/vector_windowing_gby2.q
Removing ql/src/test/queries/clientpositive/vector_windowing_multipartitioning.q
Removing ql/src/test/queries/clientpositive/vector_windowing_order_null.q
Removing ql/src/test/queries/clientpositive/vector_windowing_range_multiorder.q
Removing ql/src/test/queries/clientpositive/vector_windowing_rank.q
Removing ql/src/test/queries/clientpositive/vector_windowing_streaming.q
Removing ql/src/test/queries/clientpositive/vector_windowing_windowspec.q
Removing ql/src/test/queries/clientpositive/vector_windowing_windowspec4.q
Removing ql/src/test/results/clientpositive/llap/vector_windowing.q.out
Removing
ql/src/test/results/clientpositive/llap/vector_windowing_expressions.q.out
Removing ql/src/test/results/clientpositive/llap/vector_windowing_gby.q.out
Removing ql/src/test/results/clientpositive/llap/vector_windowing_gby2.q.out
Removing
ql/src/test/results/clientpositive/llap/vector_windowing_multipartitioning.q.out
Removing
ql/src/test/results/clientpositive/llap/vector_windowing_order_null.q.out
Removing
ql/src/test/results/clientpositive/llap/vector_windowing_range_multiorder.q.out
Removing ql/src/test/results/clientpositive/llap/vector_windowing_rank.q.out
Removing
ql/src/test/results/clientpositive/llap/vector_windowing_streaming.q.out
Removing
ql/src/test/results/clientpositive/llap/vector_windowing_windowspec.q.out
Removing
ql/src/test/results/clientpositive/llap/vector_windowing_windowspec4.q.out
Removing ql/src/test/results/clientpositive/llap/windowing_windowspec.q.out
Removing ql/src/test/results/clientpositive/vector_ptf_part_simple.q.out
Removing ql/src/test/results/clientpositive/vector_windowing.q.out
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at dac3786 HIVE-16527 Support outer and mixed reference aggregates
in windowed functions (Remus Rusanu, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-05-01 01:21:39.855
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with:
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Status: In Progress (was: Patch Available) > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Attachment: HIVE-16557.03.patch > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch, > HIVE-16557.03.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Status: Patch Available (was: In Progress) > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch, > HIVE-16557.03.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[
https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990536#comment-15990536
]
Hive QA commented on HIVE-16557:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12865728/HIVE-16557.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4952/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4952/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4952/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-05-01 01:38:43.241
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-4952/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-05-01 01:38:43.244
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at dac3786 HIVE-16527 Support outer and mixed reference aggregates
in windowed functions (Remus Rusanu, reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkEmptyKeyOperator.java
Removing ql/src/test/queries/clientpositive/vector_windowing_navfn.q
Removing ql/src/test/results/clientpositive/llap/vector_windowing_navfn.q.out
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at dac3786 HIVE-16527 Support outer and mixed reference aggregates
in windowed functions (Remus Rusanu, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-05-01 01:38:43.824
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: patch -p0
patching file itests/src/test/resources/testconfiguration.properties
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkCommonOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkEmptyKeyOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkLongOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkMultiKeyOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkObjectHashOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkStringOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/reducesink/VectorReduceSinkUniformHashOperator.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java
patching file ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
patching file ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java
patching file
ql/src/java/org/apache/hadoop/hive/ql/plan/VectorReduceSinkDesc.java
patching file ql/src/test/queries/clientpositive/vector_windowing_navfn.q
patching file ql/src/test/queries/clientpositive/vectorization_limit.q
patching file ql/src/test/queries/clientpositive/windowing_navfn.q
patching file
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_id3.q.out
patching file ql/src/test/results/clientpositive/llap/vector_inner_join.q.out
patching file
ql/src/test/results/clientpositive/llap/vector_join_part_col_char.q.out
patching file
ql/src/test/results/clientpositive/llap/vector_leftsemi_mapjoin.q.out
patching file
ql/src/test/results/clientpositive/llap/vector_non_constant_in_expr.q.out
patching file ql/src/test
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Status: Patch Available (was: In Progress) > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch, > HIVE-16557.03.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Status: In Progress (was: Patch Available) > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch, > HIVE-16557.03.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-15160) Can't order by an unselected column
[
https://issues.apache.org/jira/browse/HIVE-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-15160:
---
Status: Patch Available (was: Open)
> Can't order by an unselected column
> ---
>
> Key: HIVE-15160
> URL: https://issues.apache.org/jira/browse/HIVE-15160
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15160.01.patch, HIVE-15160.02.patch,
> HIVE-15160.04.patch, HIVE-15160.05.patch, HIVE-15160.06.patch,
> HIVE-15160.07.patch, HIVE-15160.08.patch, HIVE-15160.09.patch,
> HIVE-15160.09.patch, HIVE-15160.10.patch, HIVE-15160.11.patch,
> HIVE-15160.12.patch, HIVE-15160.13.patch, HIVE-15160.14.patch
>
>
> If a grouping key hasn't been selected, Hive complains. For comparison,
> Postgres does not.
> Example. Notice i_item_id is not selected:
> {code}
> select i_item_desc
>,i_category
>,i_class
>,i_current_price
>,sum(cs_ext_sales_price) as itemrevenue
>,sum(cs_ext_sales_price)*100/sum(sum(cs_ext_sales_price)) over
>(partition by i_class) as revenueratio
> from catalog_sales
> ,item
> ,date_dim
> where cs_item_sk = i_item_sk
>and i_category in ('Jewelry', 'Sports', 'Books')
>and cs_sold_date_sk = d_date_sk
> and d_date between cast('2001-01-12' as date)
> and (cast('2001-01-12' as date) + 30 days)
> group by i_item_id
> ,i_item_desc
> ,i_category
> ,i_class
> ,i_current_price
> order by i_category
> ,i_class
> ,i_item_id
> ,i_item_desc
> ,revenueratio
> limit 100;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15160) Can't order by an unselected column
[
https://issues.apache.org/jira/browse/HIVE-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-15160:
---
Attachment: HIVE-15160.14.patch
> Can't order by an unselected column
> ---
>
> Key: HIVE-15160
> URL: https://issues.apache.org/jira/browse/HIVE-15160
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15160.01.patch, HIVE-15160.02.patch,
> HIVE-15160.04.patch, HIVE-15160.05.patch, HIVE-15160.06.patch,
> HIVE-15160.07.patch, HIVE-15160.08.patch, HIVE-15160.09.patch,
> HIVE-15160.09.patch, HIVE-15160.10.patch, HIVE-15160.11.patch,
> HIVE-15160.12.patch, HIVE-15160.13.patch, HIVE-15160.14.patch
>
>
> If a grouping key hasn't been selected, Hive complains. For comparison,
> Postgres does not.
> Example. Notice i_item_id is not selected:
> {code}
> select i_item_desc
>,i_category
>,i_class
>,i_current_price
>,sum(cs_ext_sales_price) as itemrevenue
>,sum(cs_ext_sales_price)*100/sum(sum(cs_ext_sales_price)) over
>(partition by i_class) as revenueratio
> from catalog_sales
> ,item
> ,date_dim
> where cs_item_sk = i_item_sk
>and i_category in ('Jewelry', 'Sports', 'Books')
>and cs_sold_date_sk = d_date_sk
> and d_date between cast('2001-01-12' as date)
> and (cast('2001-01-12' as date) + 30 days)
> group by i_item_id
> ,i_item_desc
> ,i_category
> ,i_class
> ,i_current_price
> order by i_category
> ,i_class
> ,i_item_id
> ,i_item_desc
> ,revenueratio
> limit 100;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-15160) Can't order by an unselected column
[
https://issues.apache.org/jira/browse/HIVE-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengcheng Xiong updated HIVE-15160:
---
Status: Open (was: Patch Available)
> Can't order by an unselected column
> ---
>
> Key: HIVE-15160
> URL: https://issues.apache.org/jira/browse/HIVE-15160
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15160.01.patch, HIVE-15160.02.patch,
> HIVE-15160.04.patch, HIVE-15160.05.patch, HIVE-15160.06.patch,
> HIVE-15160.07.patch, HIVE-15160.08.patch, HIVE-15160.09.patch,
> HIVE-15160.09.patch, HIVE-15160.10.patch, HIVE-15160.11.patch,
> HIVE-15160.12.patch, HIVE-15160.13.patch, HIVE-15160.14.patch
>
>
> If a grouping key hasn't been selected, Hive complains. For comparison,
> Postgres does not.
> Example. Notice i_item_id is not selected:
> {code}
> select i_item_desc
>,i_category
>,i_class
>,i_current_price
>,sum(cs_ext_sales_price) as itemrevenue
>,sum(cs_ext_sales_price)*100/sum(sum(cs_ext_sales_price)) over
>(partition by i_class) as revenueratio
> from catalog_sales
> ,item
> ,date_dim
> where cs_item_sk = i_item_sk
>and i_category in ('Jewelry', 'Sports', 'Books')
>and cs_sold_date_sk = d_date_sk
> and d_date between cast('2001-01-12' as date)
> and (cast('2001-01-12' as date) + 30 days)
> group by i_item_id
> ,i_item_desc
> ,i_category
> ,i_class
> ,i_current_price
> order by i_category
> ,i_class
> ,i_item_id
> ,i_item_desc
> ,revenueratio
> limit 100;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Status: In Progress (was: Patch Available) > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch, > HIVE-16557.03.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Attachment: HIVE-16557.04.patch > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch, > HIVE-16557.03.patch, HIVE-16557.04.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[ https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16557: Status: Patch Available (was: In Progress) > Vectorization: Specialize ReduceSink empty key case > --- > > Key: HIVE-16557 > URL: https://issues.apache.org/jira/browse/HIVE-16557 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch, > HIVE-16557.03.patch, HIVE-16557.04.patch > > > Gopal pointed out that native Vectorization of ReduceSink is missing the > empty key case. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15160) Can't order by an unselected column
[
https://issues.apache.org/jira/browse/HIVE-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990577#comment-15990577
]
Hive QA commented on HIVE-15160:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12865730/HIVE-15160.14.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 35 failed/errored test(s), 10638 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_index]
(batchId=225)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cp_sel] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_basic2]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynamic_rdd_cache]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets_grouping]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order3] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_by_expr_1]
(batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_by_expr_2]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcr] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pointlookup2]
(batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pointlookup3] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_udf_case]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_date_1]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_round]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_interval_1]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_interval_arithmetic]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[view_alias] (batchId=78)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query31]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query36]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query39]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query42]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query52]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query64]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query66]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query70]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query75]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query81]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query85]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query86]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query89]
(batchId=229)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query91]
(batchId=229)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[dynamic_rdd_cache]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[limit_pushdown]
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[pcr] (batchId=123)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4953/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4953/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4953/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 35 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12865730 - PreCommit-HIVE-Build
> Can't order by an unselected column
> ---
>
> Key: HIVE-15160
> URL: https://issues.apache.org/jira/browse/HIVE-15160
> Project: Hive
> Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-15160.01.patch, HIVE-15160.02.patch,
> HIVE-15160.04.patch, HIVE-15160.05.patch, HIVE-15160.06.patch,
> HIVE-15160.07.patch, HIVE-15160.08.patch, HIVE-15160.09.patch,
> HIVE-15160.09.patch, HIVE-15160.10.patch, HIVE-15160.11.patch,
> HIVE-15160.12.patch, HIVE-15160.13.patch, HIVE-15160.14.patch
>
>
> If a grouping key hasn't been selected, Hive complains. For comparison,
> Postgres does not.
> Example. Notice i_item_id is not selected:
> {code}
> select i_item_desc
>,i
[jira] [Commented] (HIVE-16557) Vectorization: Specialize ReduceSink empty key case
[
https://issues.apache.org/jira/browse/HIVE-16557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990591#comment-15990591
]
Hive QA commented on HIVE-16557:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12865733/HIVE-16557.04.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 10638 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_index]
(batchId=225)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[outer_reference_windowed]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ptf_matchpath]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[quotedid_basic]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_windowing_navfn]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_limit]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_gby2]
(batchId=34)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_windowing]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_resolution]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage3]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ptf_matchpath]
(batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[windowing]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[windowing_gby]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join1]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join2]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
(batchId=96)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=96)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vectorization_limit]
(batchId=96)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_resolution]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ptf_matchpath]
(batchId=104)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_ptf]
(batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[windowing]
(batchId=120)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4954/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4954/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4954/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12865733 - PreCommit-HIVE-Build
> Vectorization: Specialize ReduceSink empty key case
> ---
>
> Key: HIVE-16557
> URL: https://issues.apache.org/jira/browse/HIVE-16557
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-16557.01.patch, HIVE-16557.02.patch,
> HIVE-16557.03.patch, HIVE-16557.04.patch
>
>
> Gopal pointed out that native Vectorization of ReduceSink is missing the
> empty key case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HIVE-16369) Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only)
[ https://issues.apache.org/jira/browse/HIVE-16369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16369: Status: In Progress (was: Patch Available) > Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only) > - > > Key: HIVE-16369 > URL: https://issues.apache.org/jira/browse/HIVE-16369 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16369.01.patch, HIVE-16369.02.patch, > HIVE-16369.04.patch, HIVE-16369.05.patch.tar.gz, HIVE-16369.06.patch, > HIVE-16369.07.patch, HIVE-16369.091.patch, HIVE-16369.09.patch > > > Support vectorization of PTF that doesn't include custom PRECEDING / > FOLLOWING window frame clauses. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16369) Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only)
[ https://issues.apache.org/jira/browse/HIVE-16369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16369: Attachment: HIVE-16369.091.patch > Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only) > - > > Key: HIVE-16369 > URL: https://issues.apache.org/jira/browse/HIVE-16369 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16369.01.patch, HIVE-16369.02.patch, > HIVE-16369.04.patch, HIVE-16369.05.patch.tar.gz, HIVE-16369.06.patch, > HIVE-16369.07.patch, HIVE-16369.091.patch, HIVE-16369.09.patch > > > Support vectorization of PTF that doesn't include custom PRECEDING / > FOLLOWING window frame clauses. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (HIVE-16369) Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only)
[ https://issues.apache.org/jira/browse/HIVE-16369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-16369: Status: Patch Available (was: In Progress) > Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only) > - > > Key: HIVE-16369 > URL: https://issues.apache.org/jira/browse/HIVE-16369 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-16369.01.patch, HIVE-16369.02.patch, > HIVE-16369.04.patch, HIVE-16369.05.patch.tar.gz, HIVE-16369.06.patch, > HIVE-16369.07.patch, HIVE-16369.091.patch, HIVE-16369.09.patch > > > Support vectorization of PTF that doesn't include custom PRECEDING / > FOLLOWING window frame clauses. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-16527) Support outer and mixed reference aggregates in windowed functions
[
https://issues.apache.org/jira/browse/HIVE-16527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990614#comment-15990614
]
Lefty Leverenz commented on HIVE-16527:
---
Does this need to be documented in the wiki, or is it just a bug fix?
* [Windowing and Analytics Functions |
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+WindowingAndAnalytics]
> Support outer and mixed reference aggregates in windowed functions
> --
>
> Key: HIVE-16527
> URL: https://issues.apache.org/jira/browse/HIVE-16527
> Project: Hive
> Issue Type: Improvement
> Components: Query Processor
>Reporter: Remus Rusanu
>Assignee: Remus Rusanu
> Fix For: 3.0.0
>
> Attachments: HIVE-16527.00.patch, HIVE-16527.02.patch,
> HIVE-16527.03.patch
>
>
> {noformat}
> select sum(sum(c1)) over() from e011_01;
> select sum(sum(c1)) over(partition by c2 order by c1) from e011_01 group by
> e011_01.c1, e011_01.c2;
> select sum(sum(e011_01.c1)) over(partition by e011_01.c2 order by e011_01.c1)
> from e011_01 join e011_03 on e011_01.c1 = e011_03.c1 group by e011_01.c1,
> e011_01.c2;
> select sum(sum(e011_01.c1)) over(partition by e011_03.c2 order by e011_03.c1)
> from e011_01 join e011_03 on e011_01.c1 = e011_03.c1 group by e011_03.c1,
> e011_03.c2;
> select sum(corr(e011_01.c1, e011_03.c1)) over(partition by e011_01.c2 order
> by e011_03.c2) from e011_01 join e011_03 on e011_01.c1 = e011_03.c1 group by
> e011_03.c2, e011_01.c2;
> {noformat}
> We fail to generate a plan for any of the above. The issue is that in
> {{SemanticAnalyzer.doPhase1GetAllAggregations}}, for {{TOK_WINDOWSPEC}} we
> ignore all children except the last (the window spec child). Additionally the
> typecheck processor is not prepared to encounter UDAF expressions
> ({{TypeCheckProcFactory.DefaultExpreProcessor.validateUDF}},
> {{getXpathOrFuncExprNodeDesc}}).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16346) inheritPerms should be conditional based on the target filesystem
[
https://issues.apache.org/jira/browse/HIVE-16346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990620#comment-15990620
]
Lefty Leverenz commented on HIVE-16346:
---
Does this need to be documented in the wiki?
> inheritPerms should be conditional based on the target filesystem
> -
>
> Key: HIVE-16346
> URL: https://issues.apache.org/jira/browse/HIVE-16346
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
> Fix For: 2.4.0
>
> Attachments: HIVE-16346.1-branch-2.patch,
> HIVE-16346.2-branch-2.patch, HIVE-16346.3-branch-2.patch
>
>
> Right now, a lot of the logic in {{Hive.java}} attempts to set permissions of
> different files that have been moved / copied. This is only triggered if
> {{hive.warehouse.subdir.inherit.perms}} is set to true.
> However, on blobstores such as S3, there is no concept of file permissions so
> these calls are unnecessary, which can hurt performance.
> One solution would be to set {{hive.warehouse.subdir.inherit.perms}} to
> false, but this would be a global change that affects an entire HS2 instance.
> So HDFS tables will no longer have permissions inheritance.
> A better solution would be to make the inheritance of permissions conditional
> on the target filesystem.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16520) Cache hive metadata in metastore
[ https://issues.apache.org/jira/browse/HIVE-16520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990622#comment-15990622 ] Thejas M Nair commented on HIVE-16520: -- manually triggered another UT run. Build #4956 in https://builds.apache.org/job/PreCommit-HIVE-Build/ > Cache hive metadata in metastore > > > Key: HIVE-16520 > URL: https://issues.apache.org/jira/browse/HIVE-16520 > Project: Hive > Issue Type: New Feature > Components: Metastore >Reporter: Daniel Dai >Assignee: Daniel Dai > Attachments: HIVE-16520-1.patch, HIVE-16520-proto-2.patch, > HIVE-16520-proto.patch > > > During Hive 2 benchmark, we find Hive metastore operation take a lot of time > and thus slow down Hive compilation. In some extreme case, it takes much > longer than the actual query run time. Especially, we find the latency of > cloud db is very high and 90% of total query runtime is waiting for metastore > SQL database operations. Based on this observation, the metastore operation > performance will be greatly enhanced if we have a memory structure which > cache the database query result. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
