[jira] [Updated] (HIVE-23231) LLAP: Consider removing llap-server IndexCache from Hive codebase
[ https://issues.apache.org/jira/browse/HIVE-23231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-23231: Description: Hive has a copy of IndexCache class: hive: https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/shufflehandler/IndexCache.java tez: https://github.com/apache/tez/blob/master/tez-plugins/tez-aux-services/src/main/java/org/apache/tez/auxservices/IndexCache.java As far as I can see, the Hive version doesn't add any value into it (compared by eyes so far) and maintaining is extra effort, for instance: HIVE-23029 copied improvements of TEZ-3887 which is almost 2 years old HIVE-23190 is blocked on a tez release which contains TEZ-4145 > LLAP: Consider removing llap-server IndexCache from Hive codebase > - > > Key: HIVE-23231 > URL: https://issues.apache.org/jira/browse/HIVE-23231 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Priority: Major > > Hive has a copy of IndexCache class: > hive: > https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/shufflehandler/IndexCache.java > tez: > https://github.com/apache/tez/blob/master/tez-plugins/tez-aux-services/src/main/java/org/apache/tez/auxservices/IndexCache.java > As far as I can see, the Hive version doesn't add any value into it (compared > by eyes so far) and maintaining is extra effort, for instance: > HIVE-23029 copied improvements of TEZ-3887 which is almost 2 years old > HIVE-23190 is blocked on a tez release which contains TEZ-4145 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23230) HiveWarehouseConnector executeQuery api with query having "LIMIT" clause returns more rows
[
https://issues.apache.org/jira/browse/HIVE-23230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adesh Kumar Rao updated HIVE-23230:
---
Description:
The issue is reproducible when number of llap daemons is greater than 1.
How to reproduce:
run below commands via beeline to create and populate the table
{noformat}
create table test (id int);
insert into table test values (1);
insert into table test values (2);
insert into table test values (3);
insert into table test values (4);
insert into table test values (5);
insert into table test values (6);
insert into table test values (7);
delete from test where id = 7;{noformat}
now running below query via spark-shell
{noformat}
import com.hortonworks.hwc.HiveWarehouseSession
val hive = HiveWarehouseSession.session(spark).build()
hive.executeQuery("select * from test limit 1").show(){noformat}
will return more than 1 rows.
was:
The issue is reproducible when number of llap daemons is greater than 1.
How to reproduce:
```
create table test (id int);
insert into table test values (1);
insert into table test values (2);
insert into table test values (3);
insert into table test values (4);
insert into table test values (5);
insert into table test values (6);
insert into table test values (7);
delete from test where id = 7;
```
now running `select * from test limit 1;` will return more than 1 rows.
> HiveWarehouseConnector executeQuery api with query having "LIMIT" clause
> returns more rows
> --
>
> Key: HIVE-23230
> URL: https://issues.apache.org/jira/browse/HIVE-23230
> Project: Hive
> Issue Type: Bug
>Reporter: Adesh Kumar Rao
>Priority: Major
>
> The issue is reproducible when number of llap daemons is greater than 1.
>
> How to reproduce:
> run below commands via beeline to create and populate the table
>
> {noformat}
> create table test (id int);
> insert into table test values (1);
> insert into table test values (2);
> insert into table test values (3);
> insert into table test values (4);
> insert into table test values (5);
> insert into table test values (6);
> insert into table test values (7);
> delete from test where id = 7;{noformat}
> now running below query via spark-shell
> {noformat}
> import com.hortonworks.hwc.HiveWarehouseSession
> val hive = HiveWarehouseSession.session(spark).build()
> hive.executeQuery("select * from test limit 1").show(){noformat}
> will return more than 1 rows.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23231) LLAP: Consider removing llap-server IndexCache from Hive codebase
[ https://issues.apache.org/jira/browse/HIVE-23231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-23231: Summary: LLAP: Consider removing llap-server IndexCache from Hive codebase (was: Consider removing llap-server IndexCache from Hive codebase) > LLAP: Consider removing llap-server IndexCache from Hive codebase > - > > Key: HIVE-23231 > URL: https://issues.apache.org/jira/browse/HIVE-23231 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23211) Fix metastore schema differences between init scripts, and upgrade scripts
[
https://issues.apache.org/jira/browse/HIVE-23211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085452#comment-17085452
]
Hive QA commented on HIVE-23211:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000166/HIVE-23211.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17128 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=102)
org.apache.hive.jdbc.TestJdbcWithServiceDiscovery.testKillQueryWithDifferentServerZKTurnedOff
(batchId=215)
org.apache.hive.service.auth.TestImproperTrustDomainAuthenticationBinary.org.apache.hive.service.auth.TestImproperTrustDomainAuthenticationBinary
(batchId=208)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21713/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21713/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21713/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000166 - PreCommit-HIVE-Build
> Fix metastore schema differences between init scripts, and upgrade scripts
> --
>
> Key: HIVE-23211
> URL: https://issues.apache.org/jira/browse/HIVE-23211
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Barnabas Maidics
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-23211.1.patch
>
>
> There are some differences (character encoding, defaults etc..) in metastore
> schema if we initialize using the init scripts, or upgrade using the upgrade
> scripts. The schema should be identical.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23156) NPE if -f is used with HCatCLI
[ https://issues.apache.org/jira/browse/HIVE-23156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085444#comment-17085444 ] Ivan Suller commented on HIVE-23156: [~rameshkumar] just don't use -e in CLI and you'll get a NPE at https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/HCatCli.java#L181 - it's very telling that the execString variable is checked for being null a few lines later. > NPE if -f is used with HCatCLI > -- > > Key: HIVE-23156 > URL: https://issues.apache.org/jira/browse/HIVE-23156 > Project: Hive > Issue Type: Bug > Components: HCatalog >Reporter: Ivan Suller >Priority: Critical > > After HIVE-22889 if there is no -e cli parameter then a NPE is thrown. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23195) set hive.cluster.delegation.token.gc-interval to 15 minutes instead of an hour
[ https://issues.apache.org/jira/browse/HIVE-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Zhang updated HIVE-23195: - Component/s: Hive > set hive.cluster.delegation.token.gc-interval to 15 minutes instead of an hour > -- > > Key: HIVE-23195 > URL: https://issues.apache.org/jira/browse/HIVE-23195 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Richard Zhang >Assignee: Richard Zhang >Priority: Major > Attachments: HIVE-23195.1.patch > > > the config "hive.cluster.delegation.token.gc-interval" is set as long > duration, 1 hour. This created some issues in a heavy loaded cluster in which > the tokens may not be cleaned up fast enough and the cleaner thread may fail > to clean the tokens. This may cause issues like eating too much space or LLAP > startup failures, or slow system startup. > If this hive.cluster.delegation.token.gc-interval” is reduced from 1 hour to > a relatively shorter period such as 15 mins, then the zookeeper tokens will > be cleaned more timely and mitigate these issues. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23195) set hive.cluster.delegation.token.gc-interval to 15 minutes instead of an hour
[ https://issues.apache.org/jira/browse/HIVE-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Zhang updated HIVE-23195: - Attachment: (was: HIVE-23195.1.patch) > set hive.cluster.delegation.token.gc-interval to 15 minutes instead of an hour > -- > > Key: HIVE-23195 > URL: https://issues.apache.org/jira/browse/HIVE-23195 > Project: Hive > Issue Type: Bug >Reporter: Richard Zhang >Assignee: Richard Zhang >Priority: Major > Attachments: HIVE-23195.1.patch > > > the config "hive.cluster.delegation.token.gc-interval" is set as long > duration, 1 hour. This created some issues in a heavy loaded cluster in which > the tokens may not be cleaned up fast enough and the cleaner thread may fail > to clean the tokens. This may cause issues like eating too much space or LLAP > startup failures, or slow system startup. > If this hive.cluster.delegation.token.gc-interval” is reduced from 1 hour to > a relatively shorter period such as 15 mins, then the zookeeper tokens will > be cleaned more timely and mitigate these issues. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23195) set hive.cluster.delegation.token.gc-interval to 15 minutes instead of an hour
[ https://issues.apache.org/jira/browse/HIVE-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Zhang updated HIVE-23195: - Attachment: HIVE-23195.1.patch > set hive.cluster.delegation.token.gc-interval to 15 minutes instead of an hour > -- > > Key: HIVE-23195 > URL: https://issues.apache.org/jira/browse/HIVE-23195 > Project: Hive > Issue Type: Bug >Reporter: Richard Zhang >Assignee: Richard Zhang >Priority: Major > Attachments: HIVE-23195.1.patch > > > the config "hive.cluster.delegation.token.gc-interval" is set as long > duration, 1 hour. This created some issues in a heavy loaded cluster in which > the tokens may not be cleaned up fast enough and the cleaner thread may fail > to clean the tokens. This may cause issues like eating too much space or LLAP > startup failures, or slow system startup. > If this hive.cluster.delegation.token.gc-interval” is reduced from 1 hour to > a relatively shorter period such as 15 mins, then the zookeeper tokens will > be cleaned more timely and mitigate these issues. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-23195) set hive.cluster.delegation.token.gc-interval to 15 minutes instead of an hour
[ https://issues.apache.org/jira/browse/HIVE-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-23195 started by Richard Zhang. > set hive.cluster.delegation.token.gc-interval to 15 minutes instead of an hour > -- > > Key: HIVE-23195 > URL: https://issues.apache.org/jira/browse/HIVE-23195 > Project: Hive > Issue Type: Bug >Reporter: Richard Zhang >Assignee: Richard Zhang >Priority: Major > Attachments: HIVE-23195.1.patch > > > the config "hive.cluster.delegation.token.gc-interval" is set as long > duration, 1 hour. This created some issues in a heavy loaded cluster in which > the tokens may not be cleaned up fast enough and the cleaner thread may fail > to clean the tokens. This may cause issues like eating too much space or LLAP > startup failures, or slow system startup. > If this hive.cluster.delegation.token.gc-interval” is reduced from 1 hour to > a relatively shorter period such as 15 mins, then the zookeeper tokens will > be cleaned more timely and mitigate these issues. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23100) Create RexNode factory and use it in CalcitePlanner
[ https://issues.apache.org/jira/browse/HIVE-23100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-23100: --- Attachment: HIVE-23100.10.patch > Create RexNode factory and use it in CalcitePlanner > --- > > Key: HIVE-23100 > URL: https://issues.apache.org/jira/browse/HIVE-23100 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-23100.01.patch, HIVE-23100.02.patch, > HIVE-23100.03.patch, HIVE-23100.04.patch, HIVE-23100.05.patch, > HIVE-23100.06.patch, HIVE-23100.07.patch, HIVE-23100.08.patch, > HIVE-23100.09.patch, HIVE-23100.10.patch, HIVE-23100.patch > > > Follow-up of HIVE-22746. > This will allow us to generate directly the RexNode from the AST nodes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23228) Missed optimization opportunity with equals and not equals
[
https://issues.apache.org/jira/browse/HIVE-23228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-23228:
--
> Missed optimization opportunity with equals and not equals
> --
>
> Key: HIVE-23228
> URL: https://issues.apache.org/jira/browse/HIVE-23228
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-23100 went in, there was a missed opportunity on the
> simplification of an AND predicate containing equals and not equals clause,
> which can be seen in ql/src/test/results/clientpositive/pcs.q.out .
> {code}
> filterExpr: ((key = 3) or (ds = '2000-04-08') or key is not
> null) and (key = 2)) or ((ds <> '2000-04-08') and (key = 3))) and ((key + 5)
> > 0))) (type: boolean)
> {code}
> was replaced by:
> {code}
> filterExpr: ((key = 3) or (ds = '2000-04-08') or key is not
> null) and (key = 2)) or ((ds <> '2000-04-08') and (key <> 2) and (key = 3)))
> and ((key + 5) > 0))) (type: boolean)
> {code}
> Note the additional {{key <> 2}} in predicate below.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23211) Fix metastore schema differences between init scripts, and upgrade scripts
[
https://issues.apache.org/jira/browse/HIVE-23211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085436#comment-17085436
]
Hive QA commented on HIVE-23211:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
59s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21713/dev-support/hive-personality.sh
|
| git revision | master / 661e40a |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21713/yetus/patch-asflicense-problems.txt
|
| modules | C: metastore standalone-metastore/metastore-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21713/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix metastore schema differences between init scripts, and upgrade scripts
> --
>
> Key: HIVE-23211
> URL: https://issues.apache.org/jira/browse/HIVE-23211
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Barnabas Maidics
>Assignee: Barnabas Maidics
>Priority: Major
> Attachments: HIVE-23211.1.patch
>
>
> There are some differences (character encoding, defaults etc..) in metastore
> schema if we initialize using the init scripts, or upgrade using the upgrade
> scripts. The schema should be identical.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23227) Refactor RexConverter and move some of its functionality into HiveFunctionHelper
[
https://issues.apache.org/jira/browse/HIVE-23227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085421#comment-17085421
]
Jesus Camacho Rodriguez commented on HIVE-23227:
Contract for those functions could be changed too, since many of those methods
just rewrite children blindly instead of creating the RexNode replacement,
which would be more interesting and could be easily reused within other rules.
> Refactor RexConverter and move some of its functionality into
> HiveFunctionHelper
>
>
> Key: HIVE-23227
> URL: https://issues.apache.org/jira/browse/HIVE-23227
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-23100, {{HiveFunctionHelper}} makes a few calls to methods that
> are in {{RexConverter}}. Those methods do not need to be there anymore but
> were not moved as part of that patch to avoid further changes in it. This
> issue is to tackle that refactoring.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23099) Improve Logger for Operation Child Classes
[
https://issues.apache.org/jira/browse/HIVE-23099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085420#comment-17085420
]
Hive QA commented on HIVE-23099:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000157/HIVE-23099.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21712/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21712/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21712/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-04-17 04:09:33.643
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-21712/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-04-17 04:09:33.645
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 661e40a HIVE-23212 :
SemanticAnalyzer::getStagingDirectoryPathname should check for encryption zone
only when needed (Rajesh Balamohan via Gopal V)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 661e40a HIVE-23212 :
SemanticAnalyzer::getStagingDirectoryPathname should check for encryption zone
only when needed (Rajesh Balamohan via Gopal V)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-04-17 04:09:34.741
+ rm -rf ../yetus_PreCommit-HIVE-Build-21712
+ mkdir ../yetus_PreCommit-HIVE-Build-21712
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-21712
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-21712/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error:
a/service/src/java/org/apache/hive/service/cli/operation/GetCrossReferenceOperation.java:
does not exist in index
error:
a/service/src/java/org/apache/hive/service/cli/operation/HiveCommandOperation.java:
does not exist in index
error: a/service/src/java/org/apache/hive/service/cli/operation/Operation.java:
does not exist in index
error:
a/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java:
does not exist in index
Trying to apply the patch with -p1
Going to apply patch with: git apply -p1
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc8204293244640801986.exe, --version]
libprotoc 2.6.1
protoc-jar: executing: [/tmp/protoc8204293244640801986.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc404644427364101653.exe, --version]
libprotoc 2.6.1
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
[jira] [Commented] (HIVE-23213) HiveStrictManagedMigration should handle legacy Kudu tables
[
https://issues.apache.org/jira/browse/HIVE-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085416#comment-17085416
]
Hive QA commented on HIVE-23213:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000154/HIVE-23213.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17128 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21711/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21711/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21711/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000154 - PreCommit-HIVE-Build
> HiveStrictManagedMigration should handle legacy Kudu tables
> ---
>
> Key: HIVE-23213
> URL: https://issues.apache.org/jira/browse/HIVE-23213
> Project: Hive
> Issue Type: Improvement
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-23213.0.patch, HIVE-23213.1.patch,
> HIVE-23213.2.patch
>
>
> As storage handler for Kudu backed legacy tables might be
> "com.cloudera.kudu.hive.KuduStorageHandler" set. This needs to be upgraded to
> org.apache.hadoop.hive.kudu.KuduStorageHandler as that's the actual class
> shipped with Hive 3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23227) Refactor RexConverter and move some of its functionality into HiveFunctionHelper
[
https://issues.apache.org/jira/browse/HIVE-23227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-23227:
--
> Refactor RexConverter and move some of its functionality into
> HiveFunctionHelper
>
>
> Key: HIVE-23227
> URL: https://issues.apache.org/jira/browse/HIVE-23227
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-23100, {{HiveFunctionHelper}} makes a few calls to methods that
> are in {{RexConverter}}. Those methods do not need to be there anymore but
> were not moved as part of that patch to avoid further changes in it. This
> issue is to tackle that refactoring.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23225) Simplify ExprFactory, ExprNodeDescExprFactory and RexNodeExprFactory
[
https://issues.apache.org/jira/browse/HIVE-23225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085400#comment-17085400
]
Jesus Camacho Rodriguez commented on HIVE-23225:
In addition, some methods should simply not exist since the logic does not
belong to the {{ExprFactory}}.
> Simplify ExprFactory, ExprNodeDescExprFactory and RexNodeExprFactory
>
>
> Key: HIVE-23225
> URL: https://issues.apache.org/jira/browse/HIVE-23225
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> The new {{ExprFactory}} was created based on existing calls from
> {{TypeCheckProcFactory}}. Now that we have the {{ExprNodeDesc}} and
> {{RexNode}} implementations, it seems we could do some work consolidating
> those methods, simplifying the super/subclasses, etc. For instance, the
> handling of literal values seems quite convoluted (handled by many different
> method) and could possibly be abstracted in a different way.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23213) HiveStrictManagedMigration should handle legacy Kudu tables
[
https://issues.apache.org/jira/browse/HIVE-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085397#comment-17085397
]
Hive QA commented on HIVE-23213:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21711/dev-support/hive-personality.sh
|
| git revision | master / 661e40a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21711/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21711/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HiveStrictManagedMigration should handle legacy Kudu tables
> ---
>
> Key: HIVE-23213
> URL: https://issues.apache.org/jira/browse/HIVE-23213
> Project: Hive
> Issue Type: Improvement
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-23213.0.patch, HIVE-23213.1.patch,
> HIVE-23213.2.patch
>
>
> As storage handler for Kudu backed legacy tables might be
> "com.cloudera.kudu.hive.KuduStorageHandler" set. This needs to be upgraded to
> org.apache.hadoop.hive.kudu.KuduStorageHandler as that's the actual class
> shipped with Hive 3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23048) Use sequences for TXN_ID generation
[
https://issues.apache.org/jira/browse/HIVE-23048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085382#comment-17085382
]
Hive QA commented on HIVE-23048:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000143/HIVE-23048.7.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21710/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21710/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21710/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-04-17 02:51:06.585
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-21710/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-04-17 02:51:06.589
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 661e40a HIVE-23212 :
SemanticAnalyzer::getStagingDirectoryPathname should check for encryption zone
only when needed (Rajesh Balamohan via Gopal V)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 661e40a HIVE-23212 :
SemanticAnalyzer::getStagingDirectoryPathname should check for encryption zone
only when needed (Rajesh Balamohan via Gopal V)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-04-17 02:51:07.795
+ rm -rf ../yetus_PreCommit-HIVE-Build-21710
+ mkdir ../yetus_PreCommit-HIVE-Build-21710
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-21710
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-21710/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:1079
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java'
with conflicts.
Going to apply patch with: git apply -p0
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:1079
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java'
with conflicts.
U
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-21710
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000143 - PreCommit-HIVE-Build
> Use sequences for TXN_ID generation
> ---
>
> Key: HIVE-23048
> URL: https://issues.apache.org/jira/browse/HIVE-23048
> Project: Hive
> Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Varga
>Priority: Major
> Attachments: HIVE-23048.1.patch, HIVE-23048.2.patch,
> HIVE-23048.3.patch, HIVE-23048.4.patch, HIVE-23048.5.patch,
> HIVE-23048.6.patch, HIVE-23048.7.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[
https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085381#comment-17085381
]
Hive QA commented on HIVE-23210:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000196/HIVE-23210.01.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17131 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparatorParallelism
(batchId=294)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21709/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21709/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21709/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000196 - PreCommit-HIVE-Build
> Fix shortestjobcomparator when jobs submitted have 1 task their vertices
>
>
> Key: HIVE-23210
> URL: https://issues.apache.org/jira/browse/HIVE-23210
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In latency sensitive queries, lots of jobs can have vertices with 1 task.
> Currently shortestjobcomparator does not work correctly and returns tasks in
> random order.
> [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51]
> This causes delay in the job runtime. I will attach a simple test case
> shortly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23224) Literals in CBO plan could show less information
[
https://issues.apache.org/jira/browse/HIVE-23224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-23224:
--
> Literals in CBO plan could show less information
>
>
> Key: HIVE-23224
> URL: https://issues.apache.org/jira/browse/HIVE-23224
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Currently, they are very verbose. For all char and varchar literals it will
> show the encoding, thought it is always the same. For varchar literals, it
> prints type and length, which seems unnecessary.
> For instance:
> {code}
>HiveFilter(condition=[AND(IN($10,
> _UTF-16LE'wallpaper':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'parenting':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'musical':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'womens':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'birdal':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'pants':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"), IN($12,
> _UTF-16LE'Home':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Books':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Electronics':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Shoes':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Jewelry':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Men':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"), OR(AND(IN($12,
> _UTF-16LE'Home':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Books':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Electronics':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"), IN($10,
> _UTF-16LE'wallpaper':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'parenting':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'musical':VARCHAR(2147483647) CHARACTER SET "UTF-16LE")),
> AND(IN($12, _UTF-16LE'Shoes':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Jewelry':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'Men':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"), IN($10,
> _UTF-16LE'womens':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'birdal':VARCHAR(2147483647) CHARACTER SET "UTF-16LE",
> _UTF-16LE'pants':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"])
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23134) Hive & Kudu interaction not available on ARM
[ https://issues.apache.org/jira/browse/HIVE-23134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-23134: Attachment: HIVE-23134.2.patch Assignee: Zhenyu Zheng Status: Patch Available (was: Open) > Hive & Kudu interaction not available on ARM > > > Key: HIVE-23134 > URL: https://issues.apache.org/jira/browse/HIVE-23134 > Project: Hive > Issue Type: Sub-task >Reporter: Zhenyu Zheng >Assignee: Zhenyu Zheng >Priority: Major > Attachments: HIVE-23134.1.patch, HIVE-23134.2.patch > > > Currently, we have set up an ARM CI to test out how Hive works on ARM > platform: > https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/ > According to the results, Hive & Kudu interaction is not available on ARM > platform: > https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.kudu/ > this is because that we use Kudu version 1.10 and that version does not come > with ARM workable packages. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23134) Hive & Kudu interaction not available on ARM
[ https://issues.apache.org/jira/browse/HIVE-23134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-23134: Status: Open (was: Patch Available) > Hive & Kudu interaction not available on ARM > > > Key: HIVE-23134 > URL: https://issues.apache.org/jira/browse/HIVE-23134 > Project: Hive > Issue Type: Sub-task >Reporter: Zhenyu Zheng >Priority: Major > Attachments: HIVE-23134.1.patch > > > Currently, we have set up an ARM CI to test out how Hive works on ARM > platform: > https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/ > According to the results, Hive & Kudu interaction is not available on ARM > platform: > https://builds.apache.org/view/H-L/view/Hive/job/Hive-linux-ARM-trunk/25/testReport/org.apache.hadoop.hive.kudu/ > this is because that we use Kudu version 1.10 and that version does not come > with ARM workable packages. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[
https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085362#comment-17085362
]
Hive QA commented on HIVE-23210:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
43s{color} | {color:blue} llap-server in master has 89 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} llap-server: The patch generated 4 new + 10 unchanged
- 3 fixed = 14 total (was 13) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21709/dev-support/hive-personality.sh
|
| git revision | master / 661e40a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21709/yetus/diff-checkstyle-llap-server.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21709/yetus/patch-asflicense-problems.txt
|
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21709/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix shortestjobcomparator when jobs submitted have 1 task their vertices
>
>
> Key: HIVE-23210
> URL: https://issues.apache.org/jira/browse/HIVE-23210
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In latency sensitive queries, lots of jobs can have vertices with 1 task.
> Currently shortestjobcomparator does not work correctly and returns tasks in
> random order.
> [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51]
> This causes delay in the job runtime. I will attach a simple test case
> shortly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23223) Unnecessary CAST to decimal around CASE statement
[
https://issues.apache.org/jira/browse/HIVE-23223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-23223:
--
> Unnecessary CAST to decimal around CASE statement
> -
>
> Key: HIVE-23223
> URL: https://issues.apache.org/jira/browse/HIVE-23223
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-23100 went in, there was a missed opportunity on the
> simplification of a CAST statement on top of a CASE clause, which can be seen
> in ql/src/test/results/clientpositive/vector_case_when_2.q.out .
> {code}
>expressions: q548284 (type: int), CASE WHEN ((q548284 = 4))
> THEN (0.8) WHEN ((q548284 = 5)) THEN (1) ELSE (8) END (type: decimal(2,1))
> {code}
> was replaced by:
> {code}
>expressions: q548284 (type: int), CAST( CASE WHEN ((q548284 =
> 4)) THEN (0.8) WHEN ((q548284 = 5)) THEN (1) ELSE (8) END AS decimal(11,1))
> (type: decimal(11,1))
> {code}
> The type of the CASE expression could be inferred and enforce without the
> CAST.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23103) Oracle statement batching
[
https://issues.apache.org/jira/browse/HIVE-23103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085357#comment-17085357
]
Hive QA commented on HIVE-23103:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000177/HIVE-23103.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17128 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck]
(batchId=33)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query27]
(batchId=228)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21708/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21708/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21708/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000177 - PreCommit-HIVE-Build
> Oracle statement batching
> -
>
> Key: HIVE-23103
> URL: https://issues.apache.org/jira/browse/HIVE-23103
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-23103.02.patch, HIVE-23103.03.patch,
> HIVE-23103.patch
>
>
> Examine how to really get better performance for oracle statement batches.
> [Oracle JDBC
> doc|https://docs.oracle.com/cd/E11882_01/java.112/e16548/oraperf.htm#JJDBC28752]
> describes:
> {quote}The Oracle implementation of standard update batching does not
> implement true batching for generic statements and callable statements. Even
> though Oracle JDBC supports the use of standard batching for {{Statement}}
> and {{CallableStatement}} objects, you are unlikely to see performance
> improvement.
> {quote}
> I would look for connection properties to set, so it is handled anyway, or if
> not, then use:
> {code}
> begin
> query1;
> query2;
> query3;
> end;
> {code}
> to we will have only a single roundtrip for the db.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23222) Missed opportunity in IN merge
[
https://issues.apache.org/jira/browse/HIVE-23222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-23222:
--
> Missed opportunity in IN merge
> --
>
> Key: HIVE-23222
> URL: https://issues.apache.org/jira/browse/HIVE-23222
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-23100 went in, there was a missed opportunity merging IN clauses,
> which can be seen in
> ql/src/test/results/clientpositive/llap/vector_between_in.q.out .
> {code}
>filterExpr: (cdecimal1) IN (2365.8945945946,
> 881.0135135135, -3367.6517567568) (type: boolean)
> {code}
> was replaced by:
> {code}
>filterExpr: ((cdecimal1) IN (2365.8945945946,
> -3367.6517567568) or (cdecimal1) IN (881.0135135135)) (type: boolean)
> {code}
> The problem seems to be that with decimal type, we are considering values
> with different precision/scale as a different type, thus we do not merge them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23103) Oracle statement batching
[
https://issues.apache.org/jira/browse/HIVE-23103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085347#comment-17085347
]
Hive QA commented on HIVE-23103:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
13s{color} | {color:blue} standalone-metastore/metastore-server in master has
191 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 517 unchanged - 0 fixed = 518 total (was 517) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21708/dev-support/hive-personality.sh
|
| git revision | master / 661e40a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21708/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21708/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21708/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Oracle statement batching
> -
>
> Key: HIVE-23103
> URL: https://issues.apache.org/jira/browse/HIVE-23103
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-23103.02.patch, HIVE-23103.03.patch,
> HIVE-23103.patch
>
>
> Examine how to really get better performance for oracle statement batches.
> [Oracle JDBC
> doc|https://docs.oracle.com/cd/E11882_01/java.112/e16548/oraperf.htm#JJDBC28752]
> describes:
> {quote}The Oracle implementation of standard update batching does not
> implement true batching for generic statements and callable statements. Even
> though Oracle JDBC supports the use of standard batching for {{Statement}}
> and {{CallableStatement}} objects, you are unlikely to see performance
> improvement.
> {quote}
> I would look for connection properties to set, so it is handled anyway, or if
> not, then use:
> {code}
> begin
> query1;
> query2;
> query3;
> end;
> {code}
> to we will have only a single roundtrip for the db.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21304) Show Bucketing version for ReduceSinkOp in explain extended plan
[
https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085328#comment-17085328
]
Hive QA commented on HIVE-21304:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000131/HIVE-21304.25.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 110 failed/errored test(s), 17129 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_10]
(batchId=228)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_11]
(batchId=228)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_12]
(batchId=228)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_13]
(batchId=228)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_16]
(batchId=228)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_1]
(batchId=228)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_2]
(batchId=228)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_3]
(batchId=228)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_7]
(batchId=228)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_directory]
(batchId=230)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join32] (batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_1]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_2]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_3]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_4]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_6]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_7]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_8]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_4]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_5]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_8]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_rearrange]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_map_operators]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_join]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin9]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_46]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_47]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sort_merge_join_desc_1]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sort_merge_join_desc_2]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sort_merge_join_desc_3]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sort_merge_join_desc_5]
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sort_merge_join_desc_8]
(batchId=7)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2]
(batchId=88)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_smb_mapjoin_14]
(batchId=109)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_10]
(batchId=113)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_11]
(batchId=118)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_12]
(batchId=101)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_14]
(batchId=94)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_15]
(batchId=94)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=108)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_1]
(batchId=105)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=106)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_3]
(batchId=91)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_4]
(batchId=110)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_5]
(batchId=119)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_6]
(batchId=95)
[jira] [Commented] (HIVE-21304) Show Bucketing version for ReduceSinkOp in explain extended plan
[
https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085320#comment-17085320
]
Hive QA commented on HIVE-21304:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
35s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
54s{color} | {color:red} ql: The patch generated 5 new + 1293 unchanged - 10
fixed = 1298 total (was 1303) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
54s{color} | {color:red} ql generated 3 new + 1530 unchanged - 0 fixed = 1533
total (was 1530) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
16s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Suspicious comparison of Integer references in
org.apache.hadoop.hive.ql.optimizer.BucketVersionPopulator$BucketingVersionResult.merge(BucketVersionPopulator$BucketingVersionResult)
At BucketVersionPopulator.java:in
org.apache.hadoop.hive.ql.optimizer.BucketVersionPopulator$BucketingVersionResult.merge(BucketVersionPopulator$BucketingVersionResult)
At BucketVersionPopulator.java:[line 64] |
| | Suspicious comparison of Integer references in
org.apache.hadoop.hive.ql.optimizer.BucketVersionPopulator$BucketingVersionResult.merge2(BucketVersionPopulator$BucketingVersionResult)
At BucketVersionPopulator.java:in
org.apache.hadoop.hive.ql.optimizer.BucketVersionPopulator$BucketingVersionResult.merge2(BucketVersionPopulator$BucketingVersionResult)
At BucketVersionPopulator.java:[line 74] |
| | Nullcheck of table_desc at line 8211 of value previously dereferenced in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.createFileSinkDesc(String,
TableDesc, Partition, Path, int, boolean, boolean, boolean, Path,
SemanticAnalyzer$SortBucketRSCtx, DynamicPartitionCtx, ListBucketingCtx,
RowSchema, boolean, Table, Long, boolean, Integer, QB, boolean) At
SemanticAnalyzer.java:8211 of value previously dereferenced in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.createFileSinkDesc(String,
TableDesc, Partition, Path, int, boolean, boolean, boolean, Path,
SemanticAnalyzer$SortBucketRSCtx, DynamicPartitionCtx, ListBucketingCtx,
RowSchema, boolean, Table, Long, boolean, Integer, QB, boolean) At
SemanticAnalyzer.java:[line 8204] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21707/dev-support/hive-personality.sh
|
| git revision | master /
[jira] [Commented] (HIVE-23196) Reduce number of delete calls to NN during Context::clear
[
https://issues.apache.org/jira/browse/HIVE-23196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085307#comment-17085307
]
Hive QA commented on HIVE-23196:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000132/HIVE-23196.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 17134 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[results_cache_diff_fs]
(batchId=86)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[results_cache_with_auth]
(batchId=89)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
(batchId=111)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation2]
(batchId=102)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_invalidation]
(batchId=102)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_quoted_identifiers]
(batchId=105)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_transactional]
(batchId=95)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21706/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21706/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21706/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000132 - PreCommit-HIVE-Build
> Reduce number of delete calls to NN during Context::clear
> -
>
> Key: HIVE-23196
> URL: https://issues.apache.org/jira/browse/HIVE-23196
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-23196.1.patch
>
>
> {{Context::clear()}} ends up deleting same directories (or its subdirs)
> multiple times. It would be good to reduce the number of delete calls to NN
> for latency sensitive queries. This also has an impact on concurrent queries.
> {noformat}
> 2020-04-14T04:22:28,703 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting result dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-1
> 2020-04-14T04:22:28,721 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13
> 2020-04-14T04:22:28,737 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-1/.hive-staging_hive_2020-04-14_04-22-24_335_8573832618972595103-13{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23196) Reduce number of delete calls to NN during Context::clear
[
https://issues.apache.org/jira/browse/HIVE-23196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085298#comment-17085298
]
Hive QA commented on HIVE-23196:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
32s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 42 new + 65 unchanged - 18
fixed = 107 total (was 83) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21706/dev-support/hive-personality.sh
|
| git revision | master / 661e40a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21706/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21706/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21706/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce number of delete calls to NN during Context::clear
> -
>
> Key: HIVE-23196
> URL: https://issues.apache.org/jira/browse/HIVE-23196
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-23196.1.patch
>
>
> {{Context::clear()}} ends up deleting same directories (or its subdirs)
> multiple times. It would be good to reduce the number of delete calls to NN
> for latency sensitive queries. This also has an impact on concurrent queries.
> {noformat}
> 2020-04-14T04:22:28,703 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting result dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-1
> 2020-04-14T04:22:28,721 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13
> 2020-04-14T04:22:28,737 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir:
>
[jira] [Commented] (HIVE-23207) Create integration tests for TxnManager for different rdbms metastores
[
https://issues.apache.org/jira/browse/HIVE-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085293#comment-17085293
]
Hive QA commented on HIVE-23207:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
17s{color} | {color:blue} standalone-metastore/metastore-server in master has
191 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
20s{color} | {color:blue} itests/qtest-druid in master has 7 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
50s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} standalone-metastore/metastore-server: The patch
generated 0 new + 507 unchanged - 12 fixed = 507 total (was 519) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 6 new + 90 unchanged - 80
fixed = 96 total (was 170) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
7s{color} | {color:red} root: The patch generated 6 new + 597 unchanged - 92
fixed = 603 total (was 689) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hive-blobstore passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch qtest-accumulo passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch qtest-druid passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch qtest-kudu passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch util passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
6s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
18s{color} | {color:green} standalone-metastore/metastore-server generated 0
new + 190 unchanged - 1 fixed = 190 total (was 191) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} ql in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s{color} | {color:green} qtest-druid in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} util in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 13m
14s{color} |
[jira] [Commented] (HIVE-23196) Reduce number of delete calls to NN during Context::clear
[
https://issues.apache.org/jira/browse/HIVE-23196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085288#comment-17085288
]
Rajesh Balamohan commented on HIVE-23196:
-
Another possibility: Context::clear can just enqueue the paths and gets deleted
by another thread. This would free up delete call from query execution path.
> Reduce number of delete calls to NN during Context::clear
> -
>
> Key: HIVE-23196
> URL: https://issues.apache.org/jira/browse/HIVE-23196
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-23196.1.patch
>
>
> {{Context::clear()}} ends up deleting same directories (or its subdirs)
> multiple times. It would be good to reduce the number of delete calls to NN
> for latency sensitive queries. This also has an impact on concurrent queries.
> {noformat}
> 2020-04-14T04:22:28,703 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting result dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-1
> 2020-04-14T04:22:28,721 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13
> 2020-04-14T04:22:28,737 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-1/.hive-staging_hive_2020-04-14_04-22-24_335_8573832618972595103-13{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23207) Create integration tests for TxnManager for different rdbms metastores
[
https://issues.apache.org/jira/browse/HIVE-23207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085286#comment-17085286
]
Hive QA commented on HIVE-23207:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000120/HIVE-23207.5.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 452 failed/errored test(s), 16916 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=220)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions]
(batchId=230)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_insert_overwrite]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_numbuckets_partitioned_table2_h23]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[archive_excludeHadoop20]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_create_temp_table]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5a]
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join7] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join8] (batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_schema_evolution_native]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avrotblsjoin]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bool_unknown]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_4]
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin11]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[case_sensitivity]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cast_datetime_with_sql_2016_format]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_input26] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join1]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_join0]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_simple_select]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_stats]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_udaf_percentile_approx_23]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_udf_percentile2]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_union] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_windowing]
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[char_cast] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[char_nested_types]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[char_udf1] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[char_union1] (batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[char_varchar_udf]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_tbllvl]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[combine2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[combine3] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_binary]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[confirm_initial_tbl_stats]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constGby] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer10]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer5]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer7]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer9]
(batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_insert_outputformat]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_transactional_full_acid]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_udaf] (batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view_disable_cbo]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cross_join_merge]
[jira] [Updated] (HIVE-23201) Improve logging in locking
[ https://issues.apache.org/jira/browse/HIVE-23201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod updated HIVE-23201: -- Attachment: HIVE-23201.3.patch > Improve logging in locking > -- > > Key: HIVE-23201 > URL: https://issues.apache.org/jira/browse/HIVE-23201 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Attachments: HIVE-23201.1.patch, HIVE-23201.1.patch, > HIVE-23201.2.patch, HIVE-23201.2.patch, HIVE-23201.3.patch > > > Currently it can be quite difficult to troubleshoot issues related to > locking. To understand why a particular txn gave up after a while on > acquiring a lock, you have to connect directly to the backend DB, since we > are not logging right now which exact locks the txn is waiting for. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23221) Ignore flaky test testHouseKeepingThreadExistence in TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader
[ https://issues.apache.org/jira/browse/HIVE-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085278#comment-17085278 ] Jesus Camacho Rodriguez commented on HIVE-23221: +1 [~mgergely], can you create the corresponding follow-up to enable it back and link it here? Thanks > Ignore flaky test testHouseKeepingThreadExistence in > TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader > --- > > Key: HIVE-23221 > URL: https://issues.apache.org/jira/browse/HIVE-23221 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23221.01.patch, > testHouseKeepingThreadExistence-error.txt > > > These tests are failing in an indeterministic fashion with such error > messages as the one attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23093) Create new metastore config value for jdbc max batch size
[ https://issues.apache.org/jira/browse/HIVE-23093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod updated HIVE-23093: -- Attachment: HIVE-23093.4.patch > Create new metastore config value for jdbc max batch size > - > > Key: HIVE-23093 > URL: https://issues.apache.org/jira/browse/HIVE-23093 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Attachments: HIVE-23093.1.patch, HIVE-23093.2.patch, > HIVE-23093.2.patch, HIVE-23093.3.patch, HIVE-23093.4.patch > > > In order to reduce the number of network calls to the metastore DB, there is > an effort to batch acid/locking DB calls where it makes sense. Existing > metastore config params do not provide a suitable, generic option to control > the jdbc statement max batch size. Solution is to create a new config param, > and use that in the appropriate batching calls in TxnHandler. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23169) Probe runtime support for LLAP
[ https://issues.apache.org/jira/browse/HIVE-23169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23169: -- Attachment: HIVE-23169.02.patch > Probe runtime support for LLAP > -- > > Key: HIVE-23169 > URL: https://issues.apache.org/jira/browse/HIVE-23169 > Project: Hive > Issue Type: Sub-task >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23169.01.patch, HIVE-23169.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23093) Create new metastore config value for jdbc max batch size
[
https://issues.apache.org/jira/browse/HIVE-23093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085239#comment-17085239
]
Hive QA commented on HIVE-23093:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000145/HIVE-23093.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21703/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21703/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21703/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-04-16 20:26:35.173
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-21703/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-04-16 20:26:35.176
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 661e40a HIVE-23212 :
SemanticAnalyzer::getStagingDirectoryPathname should check for encryption zone
only when needed (Rajesh Balamohan via Gopal V)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 661e40a HIVE-23212 :
SemanticAnalyzer::getStagingDirectoryPathname should check for encryption zone
only when needed (Rajesh Balamohan via Gopal V)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-04-16 20:26:36.246
+ rm -rf ../yetus_PreCommit-HIVE-Build-21703
+ mkdir ../yetus_PreCommit-HIVE-Build-21703
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-21703
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-21703/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:1347
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java'
with conflicts.
Going to apply patch with: git apply -p0
error: patch failed:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java:1347
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java'
with conflicts.
U
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-21703
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000145 - PreCommit-HIVE-Build
> Create new metastore config value for jdbc max batch size
> -
>
> Key: HIVE-23093
> URL: https://issues.apache.org/jira/browse/HIVE-23093
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Attachments: HIVE-23093.1.patch, HIVE-23093.2.patch,
> HIVE-23093.2.patch, HIVE-23093.3.patch
>
>
> In order to reduce the number of network calls to the metastore DB, there is
> an effort to batch acid/locking DB calls where it makes sense. Existing
> metastore config params do not provide a suitable, generic option to control
> the jdbc statement max batch size. Solution is to create a new config param,
> and use that in the appropriate batching calls in TxnHandler.
[jira] [Updated] (HIVE-23221) Ignore flaky test testHouseKeepingThreadExistence in TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader
[ https://issues.apache.org/jira/browse/HIVE-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23221: -- Attachment: HIVE-23221.01.patch > Ignore flaky test testHouseKeepingThreadExistence in > TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader > --- > > Key: HIVE-23221 > URL: https://issues.apache.org/jira/browse/HIVE-23221 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23221.01.patch, > testHouseKeepingThreadExistence-error.txt > > > These tests are failing in an indeterministic fashion with such error > messages as the one attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23221) Ignore flaky test testHouseKeepingThreadExistence in TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader
[ https://issues.apache.org/jira/browse/HIVE-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23221: -- Attachment: (was: HIVE-23221.01.patch) > Ignore flaky test testHouseKeepingThreadExistence in > TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader > --- > > Key: HIVE-23221 > URL: https://issues.apache.org/jira/browse/HIVE-23221 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23221.01.patch, > testHouseKeepingThreadExistence-error.txt > > > These tests are failing in an indeterministic fashion with such error > messages as the one attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23169) Probe runtime support for LLAP
[
https://issues.apache.org/jira/browse/HIVE-23169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085238#comment-17085238
]
Hive QA commented on HIVE-23169:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/1348/HIVE-23169.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17128 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestNoSaslAuth.testConnection (batchId=217)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21702/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21702/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21702/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 1348 - PreCommit-HIVE-Build
> Probe runtime support for LLAP
> --
>
> Key: HIVE-23169
> URL: https://issues.apache.org/jira/browse/HIVE-23169
> Project: Hive
> Issue Type: Sub-task
>Reporter: Panagiotis Garefalakis
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23169.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23221) Ignore flaky test testHouseKeepingThreadExistence in TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader
[ https://issues.apache.org/jira/browse/HIVE-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23221: -- Attachment: testHouseKeepingThreadExistence-error.txt > Ignore flaky test testHouseKeepingThreadExistence in > TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader > --- > > Key: HIVE-23221 > URL: https://issues.apache.org/jira/browse/HIVE-23221 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23221.01.patch, > testHouseKeepingThreadExistence-error.txt > > > These tests are failing in an indeterministic fashion with such error > messages as the one attached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23221) Ignore flaky test testHouseKeepingThreadExistence in TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader
[
https://issues.apache.org/jira/browse/HIVE-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-23221:
--
Description: These tests are failing in an indeterministic fashion with
such error messages as the one attached. (was: These tests are failing in an
indeterministic fashion with such error messages:
{code:java}
Error MessageNo thread with name metastore_task_thread_test_impl_1
found.Stacktracejava.lang.AssertionError: No thread with name
metastore_task_thread_test_impl_1 found.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at
org.apache.hadoop.hive.metastore.TestMetastoreHousekeepingLeaderEmptyConfig.testHouseKeepingThreadExistence(TestMetastoreHousekeepingLeaderEmptyConfig.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
Standard OutputFormatting using clusterid: testClusterID
Starting MetaStore Server on port 57893
Standard ErrorSLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/hiveptest/35.224.55.83-hiveptest-2/maven/org/apache/logging/log4j/log4j-slf4j-impl/2.12.1/log4j-slf4j-impl-2.12.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/hiveptest/35.224.55.83-hiveptest-2/maven/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
DEBUG StatusLogger Using ShutdownCallbackRegistry class
org.apache.logging.log4j.core.util.DefaultShutdownCallbackRegistry
DEBUG StatusLogger Took 0.073789 seconds to load 222 plugins from
sun.misc.Launcher$AppClassLoader@135fbaa4
DEBUG StatusLogger PluginManager 'Converter' found 44 plugins
DEBUG StatusLogger Starting OutputStreamManager SYSTEM_OUT.false.false-1
DEBUG StatusLogger Starting LoggerContext[name=135fbaa4,
org.apache.logging.log4j.core.LoggerContext@223191a6]...
DEBUG StatusLogger Reconfiguration started for context[name=135fbaa4] at URI
null (org.apache.logging.log4j.core.LoggerContext@223191a6) with optional
ClassLoader: null
DEBUG StatusLogger PluginManager 'ConfigurationFactory' found 4 plugins
DEBUG StatusLogger Missing dependencies for Yaml support, ConfigurationFactory
org.apache.logging.log4j.core.config.yaml.YamlConfigurationFactory is inactive
DEBUG StatusLogger Using configurationFactory
org.apache.logging.log4j.core.config.ConfigurationFactory$Factory@4135c3b
DEBUG StatusLogger Apache Log4j Core 2.12.1 initializing configuration
[jira] [Updated] (HIVE-23221) Ignore flaky test testHouseKeepingThreadExistence in TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader
[ https://issues.apache.org/jira/browse/HIVE-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23221: -- Attachment: HIVE-23221.01.patch > Ignore flaky test testHouseKeepingThreadExistence in > TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader > --- > > Key: HIVE-23221 > URL: https://issues.apache.org/jira/browse/HIVE-23221 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23221.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23221) Ignore flaky test testHouseKeepingThreadExistence in TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader
[ https://issues.apache.org/jira/browse/HIVE-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23221: -- Status: Patch Available (was: Open) > Ignore flaky test testHouseKeepingThreadExistence in > TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader > --- > > Key: HIVE-23221 > URL: https://issues.apache.org/jira/browse/HIVE-23221 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23221.01.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085219#comment-17085219 ] Panagiotis Garefalakis edited comment on HIVE-23210 at 4/16/20, 8:17 PM: - [~gopalv] thanks for the comment! You are right, the negative sign can indeed complicate things there. Reworked the patch to handle single-task DAGs as a separate case and added an extra test for multi- vs single-task comparison: [https://github.com/apache/hive/pull/984/files#diff-ae97ef12823d4e9f10f596a95340cb3cR318] Please let me know what you think. was (Author: pgaref): [~gopalv] thanks for the comment! You are right, the negative sign can indeed complicate things there. Reworked the patch to handle single-task DAGs as a separate case and added an extra test for multi- vs single-task comparison: [https://github.com/apache/hive/pull/984/files#diff-ae97ef12823d4e9f10f596a95340cb3cR318] Please let me know what you think. > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23221) Ignore flaky test testHouseKeepingThreadExistence in TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader
[ https://issues.apache.org/jira/browse/HIVE-23221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-23221: - > Ignore flaky test testHouseKeepingThreadExistence in > TestMetastoreHousekeepingLeaderEmptyConfig and TestMetastoreHousekeepingLeader > --- > > Key: HIVE-23221 > URL: https://issues.apache.org/jira/browse/HIVE-23221 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23210: -- Attachment: (was: HIVE-23210.01.patch) > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23210: -- Attachment: HIVE-23210.01.patch > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23040) Checkpointing for repl dump incremental phase
[ https://issues.apache.org/jira/browse/HIVE-23040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PRAVIN KUMAR SINHA updated HIVE-23040: -- Attachment: (was: HIVE-23040.01.patch) > Checkpointing for repl dump incremental phase > - > > Key: HIVE-23040 > URL: https://issues.apache.org/jira/browse/HIVE-23040 > Project: Hive > Issue Type: Improvement >Reporter: Aasha Medhi >Assignee: PRAVIN KUMAR SINHA >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23040.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23040) Checkpointing for repl dump incremental phase
[ https://issues.apache.org/jira/browse/HIVE-23040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PRAVIN KUMAR SINHA updated HIVE-23040: -- Attachment: HIVE-23040.01.patch > Checkpointing for repl dump incremental phase > - > > Key: HIVE-23040 > URL: https://issues.apache.org/jira/browse/HIVE-23040 > Project: Hive > Issue Type: Improvement >Reporter: Aasha Medhi >Assignee: PRAVIN KUMAR SINHA >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23040.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085219#comment-17085219 ] Panagiotis Garefalakis edited comment on HIVE-23210 at 4/16/20, 8:02 PM: - [~gopalv] thanks for the comment! You are right, the negative sign can indeed complicate things there. Reworked the patch to handle single-task DAGs as a separate case and added an extra test for multi- vs single-task comparison: [https://github.com/apache/hive/pull/984/files#diff-ae97ef12823d4e9f10f596a95340cb3cR318] Please let me know what you think. was (Author: pgaref): [~gopalv] thanks for the comment! You are right, the negative sign can indeed complicate things there. Reworked the patch to handle single-task DAGs as a separate case and added an extra test for multi- vs single-task comparison: [https://github.com/apache/hive/pull/984/files#diff-ae97ef12823d4e9f10f596a95340cb3cR318] Please let me know what you think. > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085219#comment-17085219 ] Panagiotis Garefalakis commented on HIVE-23210: --- [~gopalv] thanks for the comment! You are right, the negative sign can indeed complicate things there. Reworked the patch to handle single-task DAGs as a separate case and added an extra test for multi- vs single-task comparison: [https://github.com/apache/hive/pull/984/files#diff-ae97ef12823d4e9f10f596a95340cb3cR318] Please let me know what you think. > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23210: -- Attachment: HIVE-23210.01.patch > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23210: -- Attachment: (was: HIVE-23210.01.patch) > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23210: -- Attachment: HIVE-23210.01.patch > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23210: -- Attachment: (was: HIVE-23210.01.patch) > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23169) Probe runtime support for LLAP
[
https://issues.apache.org/jira/browse/HIVE-23169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085200#comment-17085200
]
Hive QA commented on HIVE-23169:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} llap-server in master has 89 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} llap-server: The patch generated 1 new + 99 unchanged
- 0 fixed = 100 total (was 99) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21702/dev-support/hive-personality.sh
|
| git revision | master / 661e40a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21702/yetus/diff-checkstyle-llap-server.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21702/yetus/patch-asflicense-problems.txt
|
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21702/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Probe runtime support for LLAP
> --
>
> Key: HIVE-23169
> URL: https://issues.apache.org/jira/browse/HIVE-23169
> Project: Hive
> Issue Type: Sub-task
>Reporter: Panagiotis Garefalakis
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23169.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23217) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is unchanged
[ https://issues.apache.org/jira/browse/HIVE-23217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23217: -- Attachment: (was: HIVE-23217.02.patch) > Move q tests to TestMiniLlapLocal from TestCliDriver where the output is > unchanged > -- > > Key: HIVE-23217 > URL: https://issues.apache.org/jira/browse/HIVE-23217 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23217.01.patch, HIVE-23217.02.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23217) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is unchanged
[ https://issues.apache.org/jira/browse/HIVE-23217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23217: -- Attachment: HIVE-23217.02.patch > Move q tests to TestMiniLlapLocal from TestCliDriver where the output is > unchanged > -- > > Key: HIVE-23217 > URL: https://issues.apache.org/jira/browse/HIVE-23217 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23217.01.patch, HIVE-23217.02.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23217) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is unchanged
[
https://issues.apache.org/jira/browse/HIVE-23217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085194#comment-17085194
]
Hive QA commented on HIVE-23217:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000121/HIVE-23217.02.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 17128 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multi_insert_partitioned]
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_diff_part_cols]
(batchId=74)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_par_vec_table_dictionary_encoding]
(batchId=77)
org.apache.hadoop.hive.common.metrics.metrics2.TestCodahaleMetrics.testFileReporting
(batchId=238)
org.apache.hadoop.hive.ql.parse.TestScheduledReplicationScenarios.testAcidTablesReplLoadBootstrapIncr
(batchId=205)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21701/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21701/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21701/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000121 - PreCommit-HIVE-Build
> Move q tests to TestMiniLlapLocal from TestCliDriver where the output is
> unchanged
> --
>
> Key: HIVE-23217
> URL: https://issues.apache.org/jira/browse/HIVE-23217
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23217.01.patch, HIVE-23217.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23201) Improve logging in locking
[ https://issues.apache.org/jira/browse/HIVE-23201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod updated HIVE-23201: -- Attachment: HIVE-23201.2.patch > Improve logging in locking > -- > > Key: HIVE-23201 > URL: https://issues.apache.org/jira/browse/HIVE-23201 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Attachments: HIVE-23201.1.patch, HIVE-23201.1.patch, > HIVE-23201.2.patch, HIVE-23201.2.patch > > > Currently it can be quite difficult to troubleshoot issues related to > locking. To understand why a particular txn gave up after a while on > acquiring a lock, you have to connect directly to the backend DB, since we > are not logging right now which exact locks the txn is waiting for. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085184#comment-17085184 ] Gopal Vijayaraghavan commented on HIVE-23210: - This approach might have issues outside of the 1 task setup, because the ratios for one scenario is -ve (i.e 1/1), while the almost equivalent (12/13) gets a +ve number or 0, even though both have only 1 task pending. > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23217) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is unchanged
[
https://issues.apache.org/jira/browse/HIVE-23217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085162#comment-17085162
]
Hive QA commented on HIVE-23217:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
2s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
10s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 3m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21701/dev-support/hive-personality.sh
|
| git revision | master / 661e40a |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21701/yetus/patch-asflicense-problems.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21701/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Move q tests to TestMiniLlapLocal from TestCliDriver where the output is
> unchanged
> --
>
> Key: HIVE-23217
> URL: https://issues.apache.org/jira/browse/HIVE-23217
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23217.01.patch, HIVE-23217.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23213) HiveStrictManagedMigration should handle legacy Kudu tables
[ https://issues.apache.org/jira/browse/HIVE-23213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085161#comment-17085161 ] Peter Vary commented on HIVE-23213: --- +1 pending tests > HiveStrictManagedMigration should handle legacy Kudu tables > --- > > Key: HIVE-23213 > URL: https://issues.apache.org/jira/browse/HIVE-23213 > Project: Hive > Issue Type: Improvement >Reporter: Ádám Szita >Assignee: Ádám Szita >Priority: Major > Attachments: HIVE-23213.0.patch, HIVE-23213.1.patch, > HIVE-23213.2.patch > > > As storage handler for Kudu backed legacy tables might be > "com.cloudera.kudu.hive.KuduStorageHandler" set. This needs to be upgraded to > org.apache.hadoop.hive.kudu.KuduStorageHandler as that's the actual class > shipped with Hive 3. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23123) Disable export/import of views and materialized views
[
https://issues.apache.org/jira/browse/HIVE-23123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-23123:
--
Attachment: HIVE-23123.03.patch
> Disable export/import of views and materialized views
> -
>
> Key: HIVE-23123
> URL: https://issues.apache.org/jira/browse/HIVE-23123
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23123.01.patch, HIVE-23123.02.patch,
> HIVE-23123.03.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> According to
> [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ImportExport]
> import and export can be done by using the
> {code:java}
> export table ...
> import table ...
> {code}
> commands. The document doesn't mention views or materialized views at all,
> and in fact we don't support commands like
> {code:java}
> export view ...
> import view ...
> export materialized view ...
> import materialized view ...
> {code}
> they can not be parsed at all. The word table is often used though in a
> broader sense, when it means all table like entities, including views and
> materialized views. For example the various Table classes may represent any
> of these as well.
> If I try to export a view with the export table ... command, it goes fine. A
> _metadata file will be created, but no data directory, which is what we'd
> expect. If I try to import it back, an exception is thrown due to the lack of
> the data dir:
> {code:java}
> java.lang.AssertionError: null==getPath() for exim_view
> at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:3088)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:419)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:364)
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:335)
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:722)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:491)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:485)
> {code}
> Still the view gets imported successfully, as data movement wasn't even
> necessary.
> If we try to export a materialized view which is transactional, then this
> exception occurs:
> {code:java}
> org.apache.hadoop.hive.ql.parse.SemanticException:
> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
> exim_materialized_view_da21d41a_9fe4_4446_9c72_d251496abf9d
> at
> org.apache.hadoop.hive.ql.parse.AcidExportSemanticAnalyzer.analyzeAcidExport(AcidExportSemanticAnalyzer.java:163)
> at
> org.apache.hadoop.hive.ql.parse.AcidExportSemanticAnalyzer.analyze(AcidExportSemanticAnalyzer.java:71)
> at
> org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer.analyzeInternal(RewriteSemanticAnalyzer.java:72)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:220)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:183)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:601)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:547)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:541)
> {code}
> So the export process can not handle it, as the temporary table is not
> getting created.
>
> The import command handling have a lot of codes dedicated to importing views
> and materialized views, which suggests that we support the importing (and
> thus also suggests implicitly that we support the exporting) of views and
> materialiezed views.
>
> So the conclusion is that we have to decide if we support exporting/importing
> of views and materialized views.
> If we decide not to support them then:
> - export process should throw an exception if a view or materialized view is
> the subject
> - the codes specific to view imports should be removed
> If we decide to support them, then:
> - the commands mentioned above above should be introduced
> - exception should be thrown if not the proper command used (e.g. export
> view on a table)
> - the exceptions mentioned above should be fixed
>
> I prefer not to support them, I don't think we should support the exporting /
> importing of views. The point of exporting / importing is the transfer of
>
[jira] [Updated] (HIVE-23123) Disable export/import of views and materialized views
[
https://issues.apache.org/jira/browse/HIVE-23123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-23123:
--
Attachment: (was: HIVE-23123.03.patch)
> Disable export/import of views and materialized views
> -
>
> Key: HIVE-23123
> URL: https://issues.apache.org/jira/browse/HIVE-23123
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23123.01.patch, HIVE-23123.02.patch,
> HIVE-23123.03.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> According to
> [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ImportExport]
> import and export can be done by using the
> {code:java}
> export table ...
> import table ...
> {code}
> commands. The document doesn't mention views or materialized views at all,
> and in fact we don't support commands like
> {code:java}
> export view ...
> import view ...
> export materialized view ...
> import materialized view ...
> {code}
> they can not be parsed at all. The word table is often used though in a
> broader sense, when it means all table like entities, including views and
> materialized views. For example the various Table classes may represent any
> of these as well.
> If I try to export a view with the export table ... command, it goes fine. A
> _metadata file will be created, but no data directory, which is what we'd
> expect. If I try to import it back, an exception is thrown due to the lack of
> the data dir:
> {code:java}
> java.lang.AssertionError: null==getPath() for exim_view
> at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:3088)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:419)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:364)
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:335)
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:722)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:491)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:485)
> {code}
> Still the view gets imported successfully, as data movement wasn't even
> necessary.
> If we try to export a materialized view which is transactional, then this
> exception occurs:
> {code:java}
> org.apache.hadoop.hive.ql.parse.SemanticException:
> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
> exim_materialized_view_da21d41a_9fe4_4446_9c72_d251496abf9d
> at
> org.apache.hadoop.hive.ql.parse.AcidExportSemanticAnalyzer.analyzeAcidExport(AcidExportSemanticAnalyzer.java:163)
> at
> org.apache.hadoop.hive.ql.parse.AcidExportSemanticAnalyzer.analyze(AcidExportSemanticAnalyzer.java:71)
> at
> org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer.analyzeInternal(RewriteSemanticAnalyzer.java:72)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:220)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:183)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:601)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:547)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:541)
> {code}
> So the export process can not handle it, as the temporary table is not
> getting created.
>
> The import command handling have a lot of codes dedicated to importing views
> and materialized views, which suggests that we support the importing (and
> thus also suggests implicitly that we support the exporting) of views and
> materialiezed views.
>
> So the conclusion is that we have to decide if we support exporting/importing
> of views and materialized views.
> If we decide not to support them then:
> - export process should throw an exception if a view or materialized view is
> the subject
> - the codes specific to view imports should be removed
> If we decide to support them, then:
> - the commands mentioned above above should be introduced
> - exception should be thrown if not the proper command used (e.g. export
> view on a table)
> - the exceptions mentioned above should be fixed
>
> I prefer not to support them, I don't think we should support the exporting /
> importing of views. The point of exporting / importing is the transfer
[jira] [Commented] (HIVE-23123) Disable export/import of views and materialized views
[
https://issues.apache.org/jira/browse/HIVE-23123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085136#comment-17085136
]
Hive QA commented on HIVE-23123:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/1336/HIVE-23123.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21700/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21700/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21700/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/1336/HIVE-23123.03.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 1336 - PreCommit-HIVE-Build
> Disable export/import of views and materialized views
> -
>
> Key: HIVE-23123
> URL: https://issues.apache.org/jira/browse/HIVE-23123
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23123.01.patch, HIVE-23123.02.patch,
> HIVE-23123.03.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> According to
> [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ImportExport]
> import and export can be done by using the
> {code:java}
> export table ...
> import table ...
> {code}
> commands. The document doesn't mention views or materialized views at all,
> and in fact we don't support commands like
> {code:java}
> export view ...
> import view ...
> export materialized view ...
> import materialized view ...
> {code}
> they can not be parsed at all. The word table is often used though in a
> broader sense, when it means all table like entities, including views and
> materialized views. For example the various Table classes may represent any
> of these as well.
> If I try to export a view with the export table ... command, it goes fine. A
> _metadata file will be created, but no data directory, which is what we'd
> expect. If I try to import it back, an exception is thrown due to the lack of
> the data dir:
> {code:java}
> java.lang.AssertionError: null==getPath() for exim_view
> at org.apache.hadoop.hive.ql.metadata.Hive.loadTable(Hive.java:3088)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:419)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:364)
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:335)
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:722)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:491)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:485)
> {code}
> Still the view gets imported successfully, as data movement wasn't even
> necessary.
> If we try to export a materialized view which is transactional, then this
> exception occurs:
> {code:java}
> org.apache.hadoop.hive.ql.parse.SemanticException:
> org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found
> exim_materialized_view_da21d41a_9fe4_4446_9c72_d251496abf9d
> at
> org.apache.hadoop.hive.ql.parse.AcidExportSemanticAnalyzer.analyzeAcidExport(AcidExportSemanticAnalyzer.java:163)
> at
> org.apache.hadoop.hive.ql.parse.AcidExportSemanticAnalyzer.analyze(AcidExportSemanticAnalyzer.java:71)
> at
> org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer.analyzeInternal(RewriteSemanticAnalyzer.java:72)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:220)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:183)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:601)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:547)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:541)
> {code}
> So the export process can not handle it, as the temporary table is not
> getting created.
>
> The import command handling have a lot of codes dedicated to importing views
> and materialized views, which suggests that we support the importing (and
> thus also
[jira] [Commented] (HIVE-23192) "default" database locationUri should be external warehouse root.
[
https://issues.apache.org/jira/browse/HIVE-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085134#comment-17085134
]
Hive QA commented on HIVE-23192:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000106/HIVE-23192.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17128 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck]
(batchId=33)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21699/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21699/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21699/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000106 - PreCommit-HIVE-Build
> "default" database locationUri should be external warehouse root.
> -
>
> Key: HIVE-23192
> URL: https://issues.apache.org/jira/browse/HIVE-23192
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-23192.patch, HIVE-23192.patch
>
>
> When creating the default database, the database locationUri should be set to
> external warehouse.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23218) LlapRecordReader queue limit computation is not optimal
[
https://issues.apache.org/jira/browse/HIVE-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramesh Kumar Thangarajan updated HIVE-23218:
Status: Open (was: Patch Available)
> LlapRecordReader queue limit computation is not optimal
> ---
>
> Key: HIVE-23218
> URL: https://issues.apache.org/jira/browse/HIVE-23218
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-23218.2.patch, HIVE-23218.3.patch
>
>
> After decoding {{OrcEncodedDataConsumer::decodeBatch}}, data is enqueued into
> a queue in LlapRecordReader. Queue limit for this queue is determined in
> LlapRecordReader. If it is minimal, it ends up waiting for 100ms until it
> gets capacity.
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L168
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L590
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L260
> {{determineQueueLimit}} takes into consideration all columns though only few
> columns are needed for projection. Here is an example.
> {noformat}
> create table test_acid(a1 string, a2 string, a3 string, a4 string, a5 string,
> a6 string, a7 string, a8 string, a9 string, a10 string,
> a11 string, a22 string, a33 string, a44 string, a55 string, a66 string, a77
> string, a88 string, a99 string, a100 string,
> a111 decimal(25,2), a222 decimal(25,2), a333 decimal(25,2), a444
> decimal(25,2), a555 decimal(25,2), a666 decimal(25,2), a777 decimal(25,2),
> a888 decimal(25,2), a999 decimal(25,2), a1000 decimal(25,2)) stored as orc;
> insert into table test_acid values
> ("a1","a2","a3","a4","a5","a6","a7","a8","a9","a10",
> "a11","a22","a33","a44","a55","a66","a77","a88","a99","a100",
> 10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23
> );
> select a44, count(*) from test_acid where a44 like "a4%" group by a44 order
> by a44;
> {noformat}
> For this query, queue size predicted would be "138" as it takes into account
> all fields instead of just 2. This would causes unwanted delays in adding
> data to the queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23218) LlapRecordReader queue limit computation is not optimal
[
https://issues.apache.org/jira/browse/HIVE-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramesh Kumar Thangarajan updated HIVE-23218:
Attachment: HIVE-23218.3.patch
Status: Patch Available (was: Open)
> LlapRecordReader queue limit computation is not optimal
> ---
>
> Key: HIVE-23218
> URL: https://issues.apache.org/jira/browse/HIVE-23218
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-23218.2.patch, HIVE-23218.3.patch
>
>
> After decoding {{OrcEncodedDataConsumer::decodeBatch}}, data is enqueued into
> a queue in LlapRecordReader. Queue limit for this queue is determined in
> LlapRecordReader. If it is minimal, it ends up waiting for 100ms until it
> gets capacity.
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L168
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L590
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L260
> {{determineQueueLimit}} takes into consideration all columns though only few
> columns are needed for projection. Here is an example.
> {noformat}
> create table test_acid(a1 string, a2 string, a3 string, a4 string, a5 string,
> a6 string, a7 string, a8 string, a9 string, a10 string,
> a11 string, a22 string, a33 string, a44 string, a55 string, a66 string, a77
> string, a88 string, a99 string, a100 string,
> a111 decimal(25,2), a222 decimal(25,2), a333 decimal(25,2), a444
> decimal(25,2), a555 decimal(25,2), a666 decimal(25,2), a777 decimal(25,2),
> a888 decimal(25,2), a999 decimal(25,2), a1000 decimal(25,2)) stored as orc;
> insert into table test_acid values
> ("a1","a2","a3","a4","a5","a6","a7","a8","a9","a10",
> "a11","a22","a33","a44","a55","a66","a77","a88","a99","a100",
> 10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23
> );
> select a44, count(*) from test_acid where a44 like "a4%" group by a44 order
> by a44;
> {noformat}
> For this query, queue size predicted would be "138" as it takes into account
> all fields instead of just 2. This would causes unwanted delays in adding
> data to the queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23212) SemanticAnalyzer::getStagingDirectoryPathname should check for encryption zone only when needed
[ https://issues.apache.org/jira/browse/HIVE-23212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23212: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks, Rajesh! > SemanticAnalyzer::getStagingDirectoryPathname should check for encryption > zone only when needed > --- > > Key: HIVE-23212 > URL: https://issues.apache.org/jira/browse/HIVE-23212 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-23212.1.patch > > > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L2572] > > When cluster does not have encryption zones configured, this ends up making 2 > calls to NN unnecessarily. It would be good to guard it with config or check > for the KMS config from HDFS and invoke it on need basis. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23192) "default" database locationUri should be external warehouse root.
[
https://issues.apache.org/jira/browse/HIVE-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085106#comment-17085106
]
Hive QA commented on HIVE-23192:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
191 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21699/dev-support/hive-personality.sh
|
| git revision | master / 46365ef |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21699/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21699/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> "default" database locationUri should be external warehouse root.
> -
>
> Key: HIVE-23192
> URL: https://issues.apache.org/jira/browse/HIVE-23192
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-23192.patch, HIVE-23192.patch
>
>
> When creating the default database, the database locationUri should be set to
> external warehouse.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23218) LlapRecordReader queue limit computation is not optimal
[
https://issues.apache.org/jira/browse/HIVE-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085094#comment-17085094
]
Ashutosh Chauhan commented on HIVE-23218:
-
+1
[~rameshkumar] patch contains changes from another issue, so you may have to
update it .
> LlapRecordReader queue limit computation is not optimal
> ---
>
> Key: HIVE-23218
> URL: https://issues.apache.org/jira/browse/HIVE-23218
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-23218.2.patch
>
>
> After decoding {{OrcEncodedDataConsumer::decodeBatch}}, data is enqueued into
> a queue in LlapRecordReader. Queue limit for this queue is determined in
> LlapRecordReader. If it is minimal, it ends up waiting for 100ms until it
> gets capacity.
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L168
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L590
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L260
> {{determineQueueLimit}} takes into consideration all columns though only few
> columns are needed for projection. Here is an example.
> {noformat}
> create table test_acid(a1 string, a2 string, a3 string, a4 string, a5 string,
> a6 string, a7 string, a8 string, a9 string, a10 string,
> a11 string, a22 string, a33 string, a44 string, a55 string, a66 string, a77
> string, a88 string, a99 string, a100 string,
> a111 decimal(25,2), a222 decimal(25,2), a333 decimal(25,2), a444
> decimal(25,2), a555 decimal(25,2), a666 decimal(25,2), a777 decimal(25,2),
> a888 decimal(25,2), a999 decimal(25,2), a1000 decimal(25,2)) stored as orc;
> insert into table test_acid values
> ("a1","a2","a3","a4","a5","a6","a7","a8","a9","a10",
> "a11","a22","a33","a44","a55","a66","a77","a88","a99","a100",
> 10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23
> );
> select a44, count(*) from test_acid where a44 like "a4%" group by a44 order
> by a44;
> {noformat}
> For this query, queue size predicted would be "138" as it takes into account
> all fields instead of just 2. This would causes unwanted delays in adding
> data to the queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23210) Fix shortestjobcomparator when jobs submitted have 1 task their vertices
[ https://issues.apache.org/jira/browse/HIVE-23210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085092#comment-17085092 ] Ashutosh Chauhan commented on HIVE-23210: - [~gopalv] or [~rajesh.balamohan] can you please review Panos's patch? > Fix shortestjobcomparator when jobs submitted have 1 task their vertices > > > Key: HIVE-23210 > URL: https://issues.apache.org/jira/browse/HIVE-23210 > Project: Hive > Issue Type: Improvement >Reporter: Rajesh Balamohan >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23210.01.patch, TestShortestJobFirstComparator.java > > Time Spent: 10m > Remaining Estimate: 0h > > In latency sensitive queries, lots of jobs can have vertices with 1 task. > Currently shortestjobcomparator does not work correctly and returns tasks in > random order. > [https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/comparator/ShortestJobFirstComparator.java#L51] > This causes delay in the job runtime. I will attach a simple test case > shortly. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23196) Reduce number of delete calls to NN during Context::clear
[
https://issues.apache.org/jira/browse/HIVE-23196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085084#comment-17085084
]
Ashutosh Chauhan commented on HIVE-23196:
-
instead of intercepting delete calls in Context won't it be better to just
remove those delete calls from caller side to eliminate unneeded delete ?
> Reduce number of delete calls to NN during Context::clear
> -
>
> Key: HIVE-23196
> URL: https://issues.apache.org/jira/browse/HIVE-23196
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-23196.1.patch
>
>
> {{Context::clear()}} ends up deleting same directories (or its subdirs)
> multiple times. It would be good to reduce the number of delete calls to NN
> for latency sensitive queries. This also has an impact on concurrent queries.
> {noformat}
> 2020-04-14T04:22:28,703 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting result dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-1
> 2020-04-14T04:22:28,721 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13
> 2020-04-14T04:22:28,737 DEBUG [7c6a6b09-ab37-4bc8-93a5-5da6fb154899
> HiveServer2-Handler-Pool: Thread-378] ql.Context: Deleting scratch dir:
> hdfs://nn1:8020/tmp/hive/xyz/7c6a6b09-ab37-4bc8-93a5-5da6fb154899/hive_2020-04-14_04-22-24_335_8573832618972595103-13/-mr-1/.hive-staging_hive_2020-04-14_04-22-24_335_8573832618972595103-13{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23201) Improve logging in locking
[
https://issues.apache.org/jira/browse/HIVE-23201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085081#comment-17085081
]
Hive QA commented on HIVE-23201:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000170/HIVE-23201.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17128 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck]
(batchId=33)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query30]
(batchId=228)
org.apache.hadoop.hive.ql.schq.TestScheduledQueryStatements.testExecuteImmediate
(batchId=279)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21698/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21698/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21698/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000170 - PreCommit-HIVE-Build
> Improve logging in locking
> --
>
> Key: HIVE-23201
> URL: https://issues.apache.org/jira/browse/HIVE-23201
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Attachments: HIVE-23201.1.patch, HIVE-23201.1.patch,
> HIVE-23201.2.patch
>
>
> Currently it can be quite difficult to troubleshoot issues related to
> locking. To understand why a particular txn gave up after a while on
> acquiring a lock, you have to connect directly to the backend DB, since we
> are not logging right now which exact locks the txn is waiting for.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22458) Add more constraints on showing partitions
[
https://issues.apache.org/jira/browse/HIVE-22458?focusedWorklogId=423561=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-423561
]
ASF GitHub Bot logged work on HIVE-22458:
-
Author: ASF GitHub Bot
Created on: 16/Apr/20 16:29
Start Date: 16/Apr/20 16:29
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on pull request #980: HIVE-22458:
Add more constraints on showing partitions
URL: https://github.com/apache/hive/pull/980#discussion_r409690515
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/partition/show/ShowPartitionAnalyzer.java
##
@@ -89,7 +89,8 @@ public void analyzeInternal(ASTNode ast) throws
SemanticException {
setFetchTask(createFetchTask(ShowPartitionsDesc.SCHEMA));
}
- private ExprNodeDesc getShowPartitionsFilter(Table table, ASTNode command)
throws SemanticException {
+
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 423561)
Time Spent: 1h 10m (was: 1h)
> Add more constraints on showing partitions
> --
>
> Key: HIVE-22458
> URL: https://issues.apache.org/jira/browse/HIVE-22458
> Project: Hive
> Issue Type: Improvement
>Reporter: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22458.2.patch, HIVE-22458.3.patch,
> HIVE-22458.4.patch, HIVE-22458.5.patch, HIVE-22458.branch-1.02.patch,
> HIVE-22458.branch-1.patch, HIVE-22458.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When we showing partitions of a table with thousands of partitions, all the
> partitions will be returned and it's not easy to catch the specified one from
> them, this make showing partitions hard to use. We can add where/limit/order
> by constraints to show partitions like:
> show partitions table_name [partition_specs] where partition_key >= value
> order by partition_key desc limit n;
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23103) Oracle statement batching
[
https://issues.apache.org/jira/browse/HIVE-23103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-23103:
--
Attachment: HIVE-23103.03.patch
> Oracle statement batching
> -
>
> Key: HIVE-23103
> URL: https://issues.apache.org/jira/browse/HIVE-23103
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-23103.02.patch, HIVE-23103.03.patch,
> HIVE-23103.patch
>
>
> Examine how to really get better performance for oracle statement batches.
> [Oracle JDBC
> doc|https://docs.oracle.com/cd/E11882_01/java.112/e16548/oraperf.htm#JJDBC28752]
> describes:
> {quote}The Oracle implementation of standard update batching does not
> implement true batching for generic statements and callable statements. Even
> though Oracle JDBC supports the use of standard batching for {{Statement}}
> and {{CallableStatement}} objects, you are unlikely to see performance
> improvement.
> {quote}
> I would look for connection properties to set, so it is handled anyway, or if
> not, then use:
> {code}
> begin
> query1;
> query2;
> query3;
> end;
> {code}
> to we will have only a single roundtrip for the db.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23218) LlapRecordReader queue limit computation is not optimal
[
https://issues.apache.org/jira/browse/HIVE-23218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramesh Kumar Thangarajan updated HIVE-23218:
Attachment: HIVE-23218.2.patch
Status: Patch Available (was: Open)
> LlapRecordReader queue limit computation is not optimal
> ---
>
> Key: HIVE-23218
> URL: https://issues.apache.org/jira/browse/HIVE-23218
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-23218.2.patch
>
>
> After decoding {{OrcEncodedDataConsumer::decodeBatch}}, data is enqueued into
> a queue in LlapRecordReader. Queue limit for this queue is determined in
> LlapRecordReader. If it is minimal, it ends up waiting for 100ms until it
> gets capacity.
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L168
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L590
> https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L260
> {{determineQueueLimit}} takes into consideration all columns though only few
> columns are needed for projection. Here is an example.
> {noformat}
> create table test_acid(a1 string, a2 string, a3 string, a4 string, a5 string,
> a6 string, a7 string, a8 string, a9 string, a10 string,
> a11 string, a22 string, a33 string, a44 string, a55 string, a66 string, a77
> string, a88 string, a99 string, a100 string,
> a111 decimal(25,2), a222 decimal(25,2), a333 decimal(25,2), a444
> decimal(25,2), a555 decimal(25,2), a666 decimal(25,2), a777 decimal(25,2),
> a888 decimal(25,2), a999 decimal(25,2), a1000 decimal(25,2)) stored as orc;
> insert into table test_acid values
> ("a1","a2","a3","a4","a5","a6","a7","a8","a9","a10",
> "a11","a22","a33","a44","a55","a66","a77","a88","a99","a100",
> 10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23,10.23
> );
> select a44, count(*) from test_acid where a44 like "a4%" group by a44 order
> by a44;
> {noformat}
> For this query, queue size predicted would be "138" as it takes into account
> all fields instead of just 2. This would causes unwanted delays in adding
> data to the queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23201) Improve logging in locking
[
https://issues.apache.org/jira/browse/HIVE-23201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085051#comment-17085051
]
Hive QA commented on HIVE-23201:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
11s{color} | {color:blue} standalone-metastore/metastore-server in master has
188 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
31s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 13 new + 492 unchanged - 12 fixed = 505 total (was 504) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 1 new + 15 unchanged - 0 fixed
= 16 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
18s{color} | {color:green} standalone-metastore/metastore-server generated 0
new + 187 unchanged - 1 fixed = 187 total (was 188) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} ql in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 49s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21698/dev-support/hive-personality.sh
|
| git revision | master / b632b2a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21698/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21698/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21698/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21698/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve logging in locking
> --
>
> Key: HIVE-23201
> URL: https://issues.apache.org/jira/browse/HIVE-23201
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Attachments:
[jira] [Assigned] (HIVE-23178) Add Tez Total Order Partitioner
[ https://issues.apache.org/jira/browse/HIVE-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan reassigned HIVE-23178: --- Assignee: Gopal Vijayaraghavan (was: Roohi Syeda) > Add Tez Total Order Partitioner > --- > > Key: HIVE-23178 > URL: https://issues.apache.org/jira/browse/HIVE-23178 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 3.1.0, 3.1.1, 3.1.2 >Reporter: Roohi Syeda >Assignee: Gopal Vijayaraghavan >Priority: Major > Labels: TODOC > Fix For: 4.0.0 > > Attachments: HIVE-23178.01.patch, HIVE-23178.addendum.patch, > HIVE-23178.addendum.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23178) Add Tez Total Order Partitioner
[ https://issues.apache.org/jira/browse/HIVE-23178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal Vijayaraghavan updated HIVE-23178: Attachment: HIVE-23178.addendum.patch > Add Tez Total Order Partitioner > --- > > Key: HIVE-23178 > URL: https://issues.apache.org/jira/browse/HIVE-23178 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 3.1.0, 3.1.1, 3.1.2 >Reporter: Roohi Syeda >Assignee: Gopal Vijayaraghavan >Priority: Major > Labels: TODOC > Fix For: 4.0.0 > > Attachments: HIVE-23178.01.patch, HIVE-23178.addendum.patch, > HIVE-23178.addendum.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23104) Minimize critical paths of TxnHandler::commitTxn and abortTxn
[ https://issues.apache.org/jira/browse/HIVE-23104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-23104: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks for the patch [~Marton Bod]! > Minimize critical paths of TxnHandler::commitTxn and abortTxn > - > > Key: HIVE-23104 > URL: https://issues.apache.org/jira/browse/HIVE-23104 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23104.1.patch, HIVE-23104.1.patch, > HIVE-23104.1.patch, HIVE-23104.2.patch, HIVE-23104.2.patch, > HIVE-23104.3.patch, HIVE-23104.4.patch, HIVE-23104.5.patch, > HIVE-23104.5.patch, HIVE-23104.6.patch > > > Investigate whether any code sections in TxnHandler::commitTxn and abortTxn > can be lifted out/executed async in order to reduce the overall execution > time of these methods. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23156) NPE if -f is used with HCatCLI
[ https://issues.apache.org/jira/browse/HIVE-23156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085032#comment-17085032 ] Ramesh Kumar Thangarajan commented on HIVE-23156: - [~isuller], Can you please post an example that is failing? > NPE if -f is used with HCatCLI > -- > > Key: HIVE-23156 > URL: https://issues.apache.org/jira/browse/HIVE-23156 > Project: Hive > Issue Type: Bug > Components: HCatalog >Reporter: Ivan Suller >Priority: Critical > > After HIVE-22889 if there is no -e cli parameter then a NPE is thrown. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22458) Add more constraints on showing partitions
[
https://issues.apache.org/jira/browse/HIVE-22458?focusedWorklogId=423530=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-423530
]
ASF GitHub Bot logged work on HIVE-22458:
-
Author: ASF GitHub Bot
Created on: 16/Apr/20 15:56
Start Date: 16/Apr/20 15:56
Worklog Time Spent: 10m
Work Description: miklosgergely commented on pull request #980:
HIVE-22458: Add more constraints on showing partitions
URL: https://github.com/apache/hive/pull/980#discussion_r409668346
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/partition/show/ShowPartitionAnalyzer.java
##
@@ -89,7 +89,8 @@ public void analyzeInternal(ASTNode ast) throws
SemanticException {
setFetchTask(createFetchTask(ShowPartitionsDesc.SCHEMA));
}
- private ExprNodeDesc getShowPartitionsFilter(Table table, ASTNode command)
throws SemanticException {
+
Review comment:
Please add @VisibleForTesting
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 423530)
Time Spent: 1h (was: 50m)
> Add more constraints on showing partitions
> --
>
> Key: HIVE-22458
> URL: https://issues.apache.org/jira/browse/HIVE-22458
> Project: Hive
> Issue Type: Improvement
>Reporter: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22458.2.patch, HIVE-22458.3.patch,
> HIVE-22458.4.patch, HIVE-22458.5.patch, HIVE-22458.branch-1.02.patch,
> HIVE-22458.branch-1.patch, HIVE-22458.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> When we showing partitions of a table with thousands of partitions, all the
> partitions will be returned and it's not easy to catch the specified one from
> them, this make showing partitions hard to use. We can add where/limit/order
> by constraints to show partitions like:
> show partitions table_name [partition_specs] where partition_key >= value
> order by partition_key desc limit n;
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22458) Add more constraints on showing partitions
[ https://issues.apache.org/jira/browse/HIVE-22458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085012#comment-17085012 ] Zhihua Deng commented on HIVE-22458: [~mgergely] done by TestShowPartitionAnalyzer. > Add more constraints on showing partitions > -- > > Key: HIVE-22458 > URL: https://issues.apache.org/jira/browse/HIVE-22458 > Project: Hive > Issue Type: Improvement >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22458.2.patch, HIVE-22458.3.patch, > HIVE-22458.4.patch, HIVE-22458.5.patch, HIVE-22458.branch-1.02.patch, > HIVE-22458.branch-1.patch, HIVE-22458.patch > > Time Spent: 50m > Remaining Estimate: 0h > > When we showing partitions of a table with thousands of partitions, all the > partitions will be returned and it's not easy to catch the specified one from > them, this make showing partitions hard to use. We can add where/limit/order > by constraints to show partitions like: > show partitions table_name [partition_specs] where partition_key >= value > order by partition_key desc limit n; > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23208) Update guaranteed capacity in ZK only when WM is enabled
[ https://issues.apache.org/jira/browse/HIVE-23208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23208: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks, Ramesh! > Update guaranteed capacity in ZK only when WM is enabled > > > Key: HIVE-23208 > URL: https://issues.apache.org/jira/browse/HIVE-23208 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Rajesh Balamohan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-23208.1.patch > > > [https://github.com/apache/hive/blob/master/llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskSchedulerService.java#L1091] > > Though WM is not enabled, it ends up updating ZK for every dag completion > event. For short running queries with concurrency, this ends up with lots of > calls to ZK. > It would be good to invoke this only when WM is enabled. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23220) PostExecOrcFileDump listing order may depend on the underlying filesystem
[ https://issues.apache.org/jira/browse/HIVE-23220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-23220: Parent: HIVE-22942 Issue Type: Sub-task (was: Bug) > PostExecOrcFileDump listing order may depend on the underlying filesystem > - > > Key: HIVE-23220 > URL: https://issues.apache.org/jira/browse/HIVE-23220 > Project: Hive > Issue Type: Sub-task > Components: Testing Infrastructure >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > > in case there are multiple files; the order might not be stable - and may > cause unstable q.outs > https://github.com/apache/hive/blob/83f917c787d60543f171b23d28ceda44d69c235d/ql/src/java/org/apache/hadoop/hive/ql/hooks/PostExecOrcFileDump.java#L104 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23104) Minimize critical paths of TxnHandler::commitTxn and abortTxn
[
https://issues.apache.org/jira/browse/HIVE-23104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084993#comment-17084993
]
Hive QA commented on HIVE-23104:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13000104/HIVE-23104.6.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17128 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21697/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21697/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21697/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13000104 - PreCommit-HIVE-Build
> Minimize critical paths of TxnHandler::commitTxn and abortTxn
> -
>
> Key: HIVE-23104
> URL: https://issues.apache.org/jira/browse/HIVE-23104
> Project: Hive
> Issue Type: Improvement
>Reporter: Marton Bod
>Assignee: Marton Bod
>Priority: Major
> Attachments: HIVE-23104.1.patch, HIVE-23104.1.patch,
> HIVE-23104.1.patch, HIVE-23104.2.patch, HIVE-23104.2.patch,
> HIVE-23104.3.patch, HIVE-23104.4.patch, HIVE-23104.5.patch,
> HIVE-23104.5.patch, HIVE-23104.6.patch
>
>
> Investigate whether any code sections in TxnHandler::commitTxn and abortTxn
> can be lifted out/executed async in order to reduce the overall execution
> time of these methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23220) PostExecOrcFileDump listing order may depend on the underlying filesystem
[ https://issues.apache.org/jira/browse/HIVE-23220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-23220: --- > PostExecOrcFileDump listing order may depend on the underlying filesystem > - > > Key: HIVE-23220 > URL: https://issues.apache.org/jira/browse/HIVE-23220 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > > in case there are multiple files; the order might not be stable - and may > cause unstable q.outs > https://github.com/apache/hive/blob/83f917c787d60543f171b23d28ceda44d69c235d/ql/src/java/org/apache/hadoop/hive/ql/hooks/PostExecOrcFileDump.java#L104 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[
https://issues.apache.org/jira/browse/HIVE-23215?focusedWorklogId=423520=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-423520
]
ASF GitHub Bot logged work on HIVE-23215:
-
Author: ASF GitHub Bot
Created on: 16/Apr/20 15:32
Start Date: 16/Apr/20 15:32
Worklog Time Spent: 10m
Work Description: omalley commented on pull request #983: HIVE-23215.
Make FilterContext an interface.
URL: https://github.com/apache/hive/pull/983#discussion_r409651334
##
File path:
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java
##
@@ -359,4 +361,62 @@ public void ensureSize(int rows) {
cols[i].ensureSize(rows, false);
}
Review comment:
Yeah, that is a good idea.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 423520)
Time Spent: 1h 10m (was: 1h)
> Make FilterContext and MutableFilterContext interfaces
> --
>
> Key: HIVE-23215
> URL: https://issues.apache.org/jira/browse/HIVE-23215
> Project: Hive
> Issue Type: Bug
> Components: storage-api
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> HIVE-22959 introduced FilterContext to support ORC-577. The duplication of
> fields between the FilterContext and VectorizedRowBatch seems likely to cause
> user confusion. This patch makes them interfaces that VectorizedRowBatch
> implements.
> Thus, there is a single copy of the data and no need to copy them back and
> forth. LLAP can make its own implementation of the interfaces if it doesn't
> want to use VectorizedRowBatch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[
https://issues.apache.org/jira/browse/HIVE-23215?focusedWorklogId=423518=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-423518
]
ASF GitHub Bot logged work on HIVE-23215:
-
Author: ASF GitHub Bot
Created on: 16/Apr/20 15:28
Start Date: 16/Apr/20 15:28
Worklog Time Spent: 10m
Work Description: omalley commented on pull request #983: HIVE-23215.
Make FilterContext an interface.
URL: https://github.com/apache/hive/pull/983#discussion_r409648595
##
File path:
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java
##
@@ -338,6 +339,7 @@ public void write(DataOutput arg0) throws IOException {
* - resets each column
* - inits each column
*/
+ @Override
public void reset() {
Review comment:
My problem is that VectorizedRowBatch currently has an implicit invariant
that selected is never null. Changing that assumption seems potentially
problematic.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 423518)
Time Spent: 1h (was: 50m)
> Make FilterContext and MutableFilterContext interfaces
> --
>
> Key: HIVE-23215
> URL: https://issues.apache.org/jira/browse/HIVE-23215
> Project: Hive
> Issue Type: Bug
> Components: storage-api
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> HIVE-22959 introduced FilterContext to support ORC-577. The duplication of
> fields between the FilterContext and VectorizedRowBatch seems likely to cause
> user confusion. This patch makes them interfaces that VectorizedRowBatch
> implements.
> Thus, there is a single copy of the data and no need to copy them back and
> forth. LLAP can make its own implementation of the interfaces if it doesn't
> want to use VectorizedRowBatch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23201) Improve logging in locking
[ https://issues.apache.org/jira/browse/HIVE-23201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod updated HIVE-23201: -- Attachment: HIVE-23201.2.patch > Improve logging in locking > -- > > Key: HIVE-23201 > URL: https://issues.apache.org/jira/browse/HIVE-23201 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Attachments: HIVE-23201.1.patch, HIVE-23201.1.patch, > HIVE-23201.2.patch > > > Currently it can be quite difficult to troubleshoot issues related to > locking. To understand why a particular txn gave up after a while on > acquiring a lock, you have to connect directly to the backend DB, since we > are not logging right now which exact locks the txn is waiting for. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23208) Update guaranteed capacity in ZK only when WM is enabled
[ https://issues.apache.org/jira/browse/HIVE-23208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17084981#comment-17084981 ] Ashutosh Chauhan commented on HIVE-23208: - +1 > Update guaranteed capacity in ZK only when WM is enabled > > > Key: HIVE-23208 > URL: https://issues.apache.org/jira/browse/HIVE-23208 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Rajesh Balamohan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-23208.1.patch > > > [https://github.com/apache/hive/blob/master/llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskSchedulerService.java#L1091] > > Though WM is not enabled, it ends up updating ZK for every dag completion > event. For short running queries with concurrency, this ends up with lots of > calls to ZK. > It would be good to invoke this only when WM is enabled. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23201) Improve logging in locking
[ https://issues.apache.org/jira/browse/HIVE-23201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Bod updated HIVE-23201: -- Attachment: (was: HIVE-23201.2.patch) > Improve logging in locking > -- > > Key: HIVE-23201 > URL: https://issues.apache.org/jira/browse/HIVE-23201 > Project: Hive > Issue Type: Improvement >Reporter: Marton Bod >Assignee: Marton Bod >Priority: Major > Attachments: HIVE-23201.1.patch, HIVE-23201.1.patch > > > Currently it can be quite difficult to troubleshoot issues related to > locking. To understand why a particular txn gave up after a while on > acquiring a lock, you have to connect directly to the backend DB, since we > are not logging right now which exact locks the txn is waiting for. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23211) Fix metastore schema differences between init scripts, and upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-23211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barnabas Maidics updated HIVE-23211: Attachment: HIVE-23211.1.patch Status: Patch Available (was: In Progress) > Fix metastore schema differences between init scripts, and upgrade scripts > -- > > Key: HIVE-23211 > URL: https://issues.apache.org/jira/browse/HIVE-23211 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Reporter: Barnabas Maidics >Assignee: Barnabas Maidics >Priority: Major > Attachments: HIVE-23211.1.patch > > > There are some differences (character encoding, defaults etc..) in metastore > schema if we initialize using the init scripts, or upgrade using the upgrade > scripts. The schema should be identical. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23211) Fix metastore schema differences between init scripts, and upgrade scripts
[ https://issues.apache.org/jira/browse/HIVE-23211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barnabas Maidics updated HIVE-23211: Attachment: (was: HIVE-23211.1.patch) > Fix metastore schema differences between init scripts, and upgrade scripts > -- > > Key: HIVE-23211 > URL: https://issues.apache.org/jira/browse/HIVE-23211 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Reporter: Barnabas Maidics >Assignee: Barnabas Maidics >Priority: Major > Attachments: HIVE-23211.1.patch > > > There are some differences (character encoding, defaults etc..) in metastore > schema if we initialize using the init scripts, or upgrade using the upgrade > scripts. The schema should be identical. -- This message was sent by Atlassian Jira (v8.3.4#803005)
