[jira] [Commented] (HIVE-23492) Remove unnecessary FileSystem#exists calls from ql module
[ https://issues.apache.org/jira/browse/HIVE-23492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113810#comment-17113810 ] Peter Vary commented on HIVE-23492: --- +1 > Remove unnecessary FileSystem#exists calls from ql module > - > > Key: HIVE-23492 > URL: https://issues.apache.org/jira/browse/HIVE-23492 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-23492.01.patch, HIVE-23492.02.patch, > HIVE-23492.03.patch, HIVE-23492.04.patch, HIVE-23492.05.patch > > > Wherever there is an exists() call before open() or delete(), remove it and > infer from the FileNotFoundException raised in open/delete that the file does > not exist. Exists() just checks for a FileNotFoundException so it's a waste > of time, especially on clunkier FSes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23526) Out of sequence seen in Beeline may swallow the real problem
[
https://issues.apache.org/jira/browse/HIVE-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113811#comment-17113811
]
Hive QA commented on HIVE-23526:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
44s{color} | {color:green} beeline generated 0 new + 47 unchanged - 1 fixed =
47 total (was 48) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22540/dev-support/hive-personality.sh
|
| git revision | master / 716f1f9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: beeline U: beeline |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22540/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Out of sequence seen in Beeline may swallow the real problem
> -
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23526.patch, outofsequence.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage messages to figured it out, this makes problem
>
[jira] [Updated] (HIVE-23525) TestAcidTxnCleanerService is unstable
[
https://issues.apache.org/jira/browse/HIVE-23525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Varga updated HIVE-23525:
---
Status: Patch Available (was: In Progress)
> TestAcidTxnCleanerService is unstable
> -
>
> Key: HIVE-23525
> URL: https://issues.apache.org/jira/browse/HIVE-23525
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Peter Varga
>Priority: Major
> Attachments: HIVE-23525.1.patch
>
>
> from time to time this exception happens
> http://34.66.156.144:8080/job/hive-c/7/console
> {code}
> 15:03:41 [INFO]
> 15:03:41 [INFO] ---
> 15:03:41 [INFO] T E S T S
> 15:03:41 [INFO] ---
> 15:03:42 [INFO] Running
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 25.582 s <<< FAILURE! - in
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR]
> cleansAllCommittedTxns(org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService)
> Time elapsed: 9.952 s <<< FAILURE!
> 15:04:10 java.lang.AssertionError: expected:<6> but was:<7>
> 15:04:10 at
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService.cleansAllCommittedTxns(TestAcidTxnCleanerService.java:107)
> 15:04:10
> 15:04:10 [INFO]
> 15:04:10 [INFO] Results:
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Failures:
> 15:04:10 [ERROR] TestAcidTxnCleanerService.cleansAllCommittedTxns:107
> expected:<6> but was:<7>
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on
> project hive-standalone-metastore-server: There are test failures.
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] Please refer to
> /home/jenkins/agent/workspace/hive-c/standalone-metastore/metastore-server/target/surefire-reports
> for the individual test results.
> 15:04:10 [ERROR] Please refer to dump files (if any exist)
> [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
> 15:04:10 [ERROR] -> [Help 1]
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] To see the full stack trace of the errors, re-run Maven
> with the -e switch.
> 15:04:10 [ERROR] Re-run Maven using the -X switch to enable full debug
> logging.
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] For more information about the errors and possible
> solutions, please read the following articles:
> 15:04:10 [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23525) TestAcidTxnCleanerService is unstable
[
https://issues.apache.org/jira/browse/HIVE-23525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Varga updated HIVE-23525:
---
Status: In Progress (was: Patch Available)
> TestAcidTxnCleanerService is unstable
> -
>
> Key: HIVE-23525
> URL: https://issues.apache.org/jira/browse/HIVE-23525
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Peter Varga
>Priority: Major
> Attachments: HIVE-23525.1.patch
>
>
> from time to time this exception happens
> http://34.66.156.144:8080/job/hive-c/7/console
> {code}
> 15:03:41 [INFO]
> 15:03:41 [INFO] ---
> 15:03:41 [INFO] T E S T S
> 15:03:41 [INFO] ---
> 15:03:42 [INFO] Running
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 25.582 s <<< FAILURE! - in
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR]
> cleansAllCommittedTxns(org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService)
> Time elapsed: 9.952 s <<< FAILURE!
> 15:04:10 java.lang.AssertionError: expected:<6> but was:<7>
> 15:04:10 at
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService.cleansAllCommittedTxns(TestAcidTxnCleanerService.java:107)
> 15:04:10
> 15:04:10 [INFO]
> 15:04:10 [INFO] Results:
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Failures:
> 15:04:10 [ERROR] TestAcidTxnCleanerService.cleansAllCommittedTxns:107
> expected:<6> but was:<7>
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on
> project hive-standalone-metastore-server: There are test failures.
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] Please refer to
> /home/jenkins/agent/workspace/hive-c/standalone-metastore/metastore-server/target/surefire-reports
> for the individual test results.
> 15:04:10 [ERROR] Please refer to dump files (if any exist)
> [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
> 15:04:10 [ERROR] -> [Help 1]
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] To see the full stack trace of the errors, re-run Maven
> with the -e switch.
> 15:04:10 [ERROR] Re-run Maven using the -X switch to enable full debug
> logging.
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] For more information about the errors and possible
> solutions, please read the following articles:
> 15:04:10 [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23525) TestAcidTxnCleanerService is unstable
[
https://issues.apache.org/jira/browse/HIVE-23525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Varga updated HIVE-23525:
---
Attachment: (was: HIVE-23525.1.patch)
> TestAcidTxnCleanerService is unstable
> -
>
> Key: HIVE-23525
> URL: https://issues.apache.org/jira/browse/HIVE-23525
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Peter Varga
>Priority: Major
> Attachments: HIVE-23525.1.patch
>
>
> from time to time this exception happens
> http://34.66.156.144:8080/job/hive-c/7/console
> {code}
> 15:03:41 [INFO]
> 15:03:41 [INFO] ---
> 15:03:41 [INFO] T E S T S
> 15:03:41 [INFO] ---
> 15:03:42 [INFO] Running
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 25.582 s <<< FAILURE! - in
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR]
> cleansAllCommittedTxns(org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService)
> Time elapsed: 9.952 s <<< FAILURE!
> 15:04:10 java.lang.AssertionError: expected:<6> but was:<7>
> 15:04:10 at
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService.cleansAllCommittedTxns(TestAcidTxnCleanerService.java:107)
> 15:04:10
> 15:04:10 [INFO]
> 15:04:10 [INFO] Results:
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Failures:
> 15:04:10 [ERROR] TestAcidTxnCleanerService.cleansAllCommittedTxns:107
> expected:<6> but was:<7>
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on
> project hive-standalone-metastore-server: There are test failures.
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] Please refer to
> /home/jenkins/agent/workspace/hive-c/standalone-metastore/metastore-server/target/surefire-reports
> for the individual test results.
> 15:04:10 [ERROR] Please refer to dump files (if any exist)
> [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
> 15:04:10 [ERROR] -> [Help 1]
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] To see the full stack trace of the errors, re-run Maven
> with the -e switch.
> 15:04:10 [ERROR] Re-run Maven using the -X switch to enable full debug
> logging.
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] For more information about the errors and possible
> solutions, please read the following articles:
> 15:04:10 [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23525) TestAcidTxnCleanerService is unstable
[
https://issues.apache.org/jira/browse/HIVE-23525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Varga updated HIVE-23525:
---
Attachment: HIVE-23525.1.patch
> TestAcidTxnCleanerService is unstable
> -
>
> Key: HIVE-23525
> URL: https://issues.apache.org/jira/browse/HIVE-23525
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Peter Varga
>Priority: Major
> Attachments: HIVE-23525.1.patch
>
>
> from time to time this exception happens
> http://34.66.156.144:8080/job/hive-c/7/console
> {code}
> 15:03:41 [INFO]
> 15:03:41 [INFO] ---
> 15:03:41 [INFO] T E S T S
> 15:03:41 [INFO] ---
> 15:03:42 [INFO] Running
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 25.582 s <<< FAILURE! - in
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR]
> cleansAllCommittedTxns(org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService)
> Time elapsed: 9.952 s <<< FAILURE!
> 15:04:10 java.lang.AssertionError: expected:<6> but was:<7>
> 15:04:10 at
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService.cleansAllCommittedTxns(TestAcidTxnCleanerService.java:107)
> 15:04:10
> 15:04:10 [INFO]
> 15:04:10 [INFO] Results:
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Failures:
> 15:04:10 [ERROR] TestAcidTxnCleanerService.cleansAllCommittedTxns:107
> expected:<6> but was:<7>
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on
> project hive-standalone-metastore-server: There are test failures.
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] Please refer to
> /home/jenkins/agent/workspace/hive-c/standalone-metastore/metastore-server/target/surefire-reports
> for the individual test results.
> 15:04:10 [ERROR] Please refer to dump files (if any exist)
> [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
> 15:04:10 [ERROR] -> [Help 1]
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] To see the full stack trace of the errors, re-run Maven
> with the -e switch.
> 15:04:10 [ERROR] Re-run Maven using the -X switch to enable full debug
> logging.
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] For more information about the errors and possible
> solutions, please read the following articles:
> 15:04:10 [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22077) Inserting overwrite partitions clause does not clean directories while partitions' info is not stored in metadata
[
https://issues.apache.org/jira/browse/HIVE-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113829#comment-17113829
]
Jeffrey(Xilang) Yan commented on HIVE-22077:
We meet exactly same issue on production. Insert overwrite sql failed due to
hive metastore lock, retry the sql doesn't remove old data which make many many
duplicate data left in hdfs. It is a nightmare now, we have to find all
partition which have duplicate data.
Could someone help to revew this patch?
[~kgyrtkirk] [~jcamachorodriguez]
> Inserting overwrite partitions clause does not clean directories while
> partitions' info is not stored in metadata
> -
>
> Key: HIVE-22077
> URL: https://issues.apache.org/jira/browse/HIVE-22077
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1, 4.0.0, 2.3.4
>Reporter: Hui An
>Assignee: Hui An
>Priority: Major
> Attachments: HIVE-22077.patch.1
>
>
> Inserting overwrite static partitions may not clean related HDFS location if
> partitions' info is not stored in metadata.
> Steps to reproduce this issue :
>
> 1. Create a managed table :
>
> {code:sql}
> CREATE TABLE `test`(
>`id` string)
> PARTITIONED BY (
>`dayno` string)
> ROW FORMAT SERDE
>'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION
>'hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test'
> TBLPROPERTIES (
>'transient_lastDdlTime'='1564731656')
> {code}
>
> 2. Create partition's directory and put some data in it
>
> {code:java}
> hdfs dfs -mkdir
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
> hdfs dfs -put test.data
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
> {code}
>
> 3. Insert overwrite partition dayno=20190802
>
> {code:sql}
> INSERT OVERWRITE TABLE test PARTITION(dayno='20190802')
> SELECT "some value";
> {code}
>
> 4. We could see the test.data under partition directory is not deleted.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22077) Inserting overwrite partitions clause does not clean directories while partitions' info is not stored in metadata
[
https://issues.apache.org/jira/browse/HIVE-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113829#comment-17113829
]
Jeffrey(Xilang) Yan edited comment on HIVE-22077 at 5/22/20, 7:57 AM:
--
We meet exactly same issue on production. Insert overwrite sql failed due to
hive metastore lock, retry the sql doesn't remove old data which make many many
duplicate data left in hdfs. It is a nightmare now, we have to find all
partition which have duplicate data.
Could someone help to revew this patch?
[~kgyrtkirk] [~jcamachorodriguez] [~mgergely] [~ashutoshc]
was (Author: xilangyan):
We meet exactly same issue on production. Insert overwrite sql failed due to
hive metastore lock, retry the sql doesn't remove old data which make many many
duplicate data left in hdfs. It is a nightmare now, we have to find all
partition which have duplicate data.
Could someone help to revew this patch?
[~kgyrtkirk] [~jcamachorodriguez]
> Inserting overwrite partitions clause does not clean directories while
> partitions' info is not stored in metadata
> -
>
> Key: HIVE-22077
> URL: https://issues.apache.org/jira/browse/HIVE-22077
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1, 4.0.0, 2.3.4
>Reporter: Hui An
>Assignee: Hui An
>Priority: Major
> Attachments: HIVE-22077.patch.1
>
>
> Inserting overwrite static partitions may not clean related HDFS location if
> partitions' info is not stored in metadata.
> Steps to reproduce this issue :
>
> 1. Create a managed table :
>
> {code:sql}
> CREATE TABLE `test`(
>`id` string)
> PARTITIONED BY (
>`dayno` string)
> ROW FORMAT SERDE
>'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION
>'hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test'
> TBLPROPERTIES (
>'transient_lastDdlTime'='1564731656')
> {code}
>
> 2. Create partition's directory and put some data in it
>
> {code:java}
> hdfs dfs -mkdir
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
> hdfs dfs -put test.data
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
> {code}
>
> 3. Insert overwrite partition dayno=20190802
>
> {code:sql}
> INSERT OVERWRITE TABLE test PARTITION(dayno='20190802')
> SELECT "some value";
> {code}
>
> 4. We could see the test.data under partition directory is not deleted.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-1941) support explicit view partitioning
[ https://issues.apache.org/jira/browse/HIVE-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113834#comment-17113834 ] congli commented on HIVE-1941: -- This feature is still not covered in the DDL doc. > support explicit view partitioning > -- > > Key: HIVE-1941 > URL: https://issues.apache.org/jira/browse/HIVE-1941 > Project: Hive > Issue Type: New Feature > Components: Query Processor, Views >Affects Versions: 0.6.0 >Reporter: John Sichi >Assignee: John Sichi >Priority: Major > Fix For: 0.8.0 > > Attachments: HIVE-1941.1.patch, HIVE-1941.2.patch, HIVE-1941.3.patch, > HIVE-1941.4.patch, HIVE-1941.5.patch > > > Allow creation of a view with an explicit partitioning definition, and > support ALTER VIEW ADD/DROP PARTITION for instantiating partitions. > For more information, see > (obsolete: http://wiki.apache.org/hadoop/Hive/PartitionedViews) > https://cwiki.apache.org/confluence/display/Hive/PartitionedViews -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23526) Out of sequence seen in Beeline may swallow the real problem
[
https://issues.apache.org/jira/browse/HIVE-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113837#comment-17113837
]
Hive QA commented on HIVE-23526:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003638/HIVE-23526.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17279 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.beeline.TestBeeLineWithArgs.testRowsAffected (batchId=211)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22540/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22540/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22540/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003638 - PreCommit-HIVE-Build
> Out of sequence seen in Beeline may swallow the real problem
> -
>
> Key: HIVE-23526
> URL: https://issues.apache.org/jira/browse/HIVE-23526
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
> Environment: Hive 1.2.2
>Reporter: Zhihua Deng
>Assignee: Zhihua Deng
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23526.patch, outofsequence.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Sometimes we can see 'out of sequence response' message in beeline, for
> example:
> Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of
> sequence response (state=08S01,code=0)
> java.sql.SQLException: org.apache.thrift.TApplicationException:
> CloseOperation failed: out of sequence response
> at
> org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198)
> at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217)
> at org.apache.hive.beeline.Commands.execute(Commands.java:891)
> at org.apache.hive.beeline.Commands.sql(Commands.java:713)
> at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976)
> at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816)
> at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774)
> at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487)
> at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470)
> and there is no other usage messages to figured it out, this makes problem
> puzzled as beeline does not have concurrency problem on underlying thrift
> transport.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23513) Fix Json output for SHOW TABLES and SHOW MATERIALIZED VIEWS
[ https://issues.apache.org/jira/browse/HIVE-23513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23513: -- Attachment: HIVE-23513.02.patch > Fix Json output for SHOW TABLES and SHOW MATERIALIZED VIEWS > --- > > Key: HIVE-23513 > URL: https://issues.apache.org/jira/browse/HIVE-23513 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23513.01.patch, HIVE-23513.02.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23513) Fix Json output for SHOW TABLES and SHOW MATERIALIZED VIEWS
[ https://issues.apache.org/jira/browse/HIVE-23513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23513: -- Attachment: (was: HIVE-23513.02.patch) > Fix Json output for SHOW TABLES and SHOW MATERIALIZED VIEWS > --- > > Key: HIVE-23513 > URL: https://issues.apache.org/jira/browse/HIVE-23513 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23513.01.patch, HIVE-23513.02.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23510) TestMiniLlapLocalCliDriver should be the default driver for q tests
[ https://issues.apache.org/jira/browse/HIVE-23510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23510: -- Attachment: HIVE-23510.04.patch > TestMiniLlapLocalCliDriver should be the default driver for q tests > --- > > Key: HIVE-23510 > URL: https://issues.apache.org/jira/browse/HIVE-23510 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23510.01.patch, HIVE-23510.02.patch, > HIVE-23510.03.patch, HIVE-23510.04.patch > > > Set TestMiniLlapLocalCliDriver as the default driver. For now the few tests > still processed by TestCliDriver should be marked in the > testconfiguration.properties, until it is completely eliminated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23510) TestMiniLlapLocalCliDriver should be the default driver for q tests
[ https://issues.apache.org/jira/browse/HIVE-23510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23510: -- Attachment: (was: HIVE-23510.04.patch) > TestMiniLlapLocalCliDriver should be the default driver for q tests > --- > > Key: HIVE-23510 > URL: https://issues.apache.org/jira/browse/HIVE-23510 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23510.01.patch, HIVE-23510.02.patch, > HIVE-23510.03.patch, HIVE-23510.04.patch > > > Set TestMiniLlapLocalCliDriver as the default driver. For now the few tests > still processed by TestCliDriver should be marked in the > testconfiguration.properties, until it is completely eliminated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23281) ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to DB for ACID tables
[
https://issues.apache.org/jira/browse/HIVE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113866#comment-17113866
]
Hive QA commented on HIVE-23281:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
33s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
20s{color} | {color:red} metastore-server in master failed. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
32s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 7 new + 586 unchanged - 1 fixed = 593 total (was 587) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} metastore-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22541/dev-support/hive-personality.sh
|
| git revision | master / 716f1f9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22541/yetus/branch-findbugs-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22541/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22541/yetus/patch-findbugs-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22541/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to
> DB for ACID tables
> --
>
> Key: HIVE-23281
> URL: https://issues.apache.org/jira/browse/HIVE-23281
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-23281.1.patch, HIVE-23281.2.patch,
> HIVE-23281.3.patch, HIVE-23281.4.patch, HIVE-23281.5.patch,
> HIVE-23281.6.patch, HIVE-23281.7.patch, image-2020-04-23-13-56-17-210.png
>
>
> [https://github.com/apache/hive
[jira] [Updated] (HIVE-23519) Read Ranger Configs from Classpath
[ https://issues.apache.org/jira/browse/HIVE-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23519: --- Attachment: HIVE-23519.05.patch Status: Patch Available (was: In Progress) > Read Ranger Configs from Classpath > -- > > Key: HIVE-23519 > URL: https://issues.apache.org/jira/browse/HIVE-23519 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23519.01.patch, HIVE-23519.02.patch, > HIVE-23519.03.patch, HIVE-23519.04.patch, HIVE-23519.05.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23519) Read Ranger Configs from Classpath
[ https://issues.apache.org/jira/browse/HIVE-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23519: --- Status: In Progress (was: Patch Available) > Read Ranger Configs from Classpath > -- > > Key: HIVE-23519 > URL: https://issues.apache.org/jira/browse/HIVE-23519 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23519.01.patch, HIVE-23519.02.patch, > HIVE-23519.03.patch, HIVE-23519.04.patch, HIVE-23519.05.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23281) ObjectStore::convertToStorageDescriptor can be optimised to reduce calls to DB for ACID tables
[
https://issues.apache.org/jira/browse/HIVE-23281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113877#comment-17113877
]
Hive QA commented on HIVE-23281:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003697/HIVE-23281.7.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 51 failed/errored test(s), 17281 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
(batchId=22)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
(batchId=27)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets]
(batchId=101)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_nullscan]
(batchId=99)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_stats2]
(batchId=79)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_stats5]
(batchId=51)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_table_directories_test]
(batchId=66)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_table_stats]
(batchId=85)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[autoColumnStats_4]
(batchId=43)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[check_constraint]
(batchId=49)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[create_or_replace_view]
(batchId=71)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[create_transactional_full_acid]
(batchId=108)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[create_transactional_insert_only]
(batchId=91)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[create_view_partitioned]
(batchId=69)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=70)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[describe_formatted_view_partitioned]
(batchId=110)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[describe_formatted_view_partitioned_json]
(batchId=88)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=96)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif]
(batchId=45)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_default]
(batchId=117)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_exim]
(batchId=124)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[murmur_hash_migration2]
(batchId=79)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge_stats]
(batchId=110)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[stats_nonpart]
(batchId=43)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[stats_part2]
(batchId=51)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[stats_part]
(batchId=80)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[stats_sizebug]
(batchId=114)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_view_as_select_with_partition]
(batchId=8)
org.apache.hadoop.hive.metastore.TestEmbeddedHiveMetaStore.testAlterViewParititon
(batchId=158)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testAlterViewParititon
(batchId=160)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreCustomAuth.testAlterViewParititon
(batchId=166)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK.testAlterViewParititon
(batchId=159)
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZKBindHost.testAlterViewParititon
(batchId=166)
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testAlterViewParititon
(batchId=156)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterViewParititon
(batchId=154)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyServer.testAlterViewParititon
(batchId=165)
org.apache.hadoop.hive.metastore.client.TestAddPartitions.testAddPartitionsForViewNullPartSd[Embedded]
(batchId=155)
org.apache.hadoop.hive.metastore.client.TestAddPartitions.testAddPartitionsForViewNullPartSd[Remote]
(batchId=155)
org.apache.hadoop.hive.metastore.client.TestAddPartitionsFromPartSpec.testAddPartitionsForViewNullPartSd[Embedded]
(batchId=157)
org.apache.hadoop.hive.metastore.client.TestAddPartitionsFromPartSpec.testAddPartitionsForViewNullPartSd[Remote]
(batchId=157)
org.apache.hadoop.hive.ql.TestTxnExIm.testImport (batchId=278)
org.apache.hadoop.hive.ql.TestTxnExIm.testImportPartitioned (batchId=278)
org.apache.hadoop.hive.ql.TestTxnExIm.testImportPartitionedCreate (batchId=278)
org.apache.hadoop.hive.ql.TestTxnExIm.testImportPartitionedOrc (batch
[jira] [Updated] (HIVE-23365) Put RS deduplication optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-23365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-23365: --- Attachment: HIVE-23365.01.patch > Put RS deduplication optimization under cost based decision > --- > > Key: HIVE-23365 > URL: https://issues.apache.org/jira/browse/HIVE-23365 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Stamatis Zampetakis >Priority: Major > Attachments: HIVE-23365.01.patch > > > Currently, RS deduplication is always executed whenever it is semantically > correct. However, it could be beneficial to leave both RS operators in the > plan, e.g., if the NDV of the second RS is very low. Thus, we would like this > decision to be cost-based. We could use a simple heuristic that would work > fine for most of the cases without introducing regressions for existing > cases, e.g., if NDV for partition column is less than estimated parallelism > in the second RS, do not execute deduplication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23365) Put RS deduplication optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-23365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-23365: --- Status: Patch Available (was: Open) Add heuristic to perform deduplication only if the estimated parallelism of the final RS is not reduced after the merge. Upload patch to trigger pre-commit tests and check for regressions. > Put RS deduplication optimization under cost based decision > --- > > Key: HIVE-23365 > URL: https://issues.apache.org/jira/browse/HIVE-23365 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Stamatis Zampetakis >Priority: Major > Attachments: HIVE-23365.01.patch > > > Currently, RS deduplication is always executed whenever it is semantically > correct. However, it could be beneficial to leave both RS operators in the > plan, e.g., if the NDV of the second RS is very low. Thus, we would like this > decision to be cost-based. We could use a simple heuristic that would work > fine for most of the cases without introducing regressions for existing > cases, e.g., if NDV for partition column is less than estimated parallelism > in the second RS, do not execute deduplication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23466) ZK registry base should remove only specific instance instead of host
[ https://issues.apache.org/jira/browse/HIVE-23466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-23466: - Status: Patch Available (was: Open) > ZK registry base should remove only specific instance instead of host > - > > Key: HIVE-23466 > URL: https://issues.apache.org/jira/browse/HIVE-23466 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23466.1.patch > > > When ZKRegistryBase detects new ZK nodes it maintains path based cache and > host based cache. The host based cached already handles multiple instances > running in same host. But even if single instance is removed all instances > belonging to the host are removed. > Another issue is that, if single host has multiple instances it returns a Set > with no ordering. Ideally, we want the newest instance to be top of the set > (use TreeSet maybe?). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23466) ZK registry base should remove only specific instance instead of host
[ https://issues.apache.org/jira/browse/HIVE-23466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar reassigned HIVE-23466: Assignee: Attila Magyar (was: Prasanth Jayachandran) > ZK registry base should remove only specific instance instead of host > - > > Key: HIVE-23466 > URL: https://issues.apache.org/jira/browse/HIVE-23466 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Attila Magyar >Priority: Major > Attachments: HIVE-23466.1.patch > > > When ZKRegistryBase detects new ZK nodes it maintains path based cache and > host based cache. The host based cached already handles multiple instances > running in same host. But even if single instance is removed all instances > belonging to the host are removed. > Another issue is that, if single host has multiple instances it returns a Set > with no ordering. Ideally, we want the newest instance to be top of the set > (use TreeSet maybe?). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23510) TestMiniLlapLocalCliDriver should be the default driver for q tests
[
https://issues.apache.org/jira/browse/HIVE-23510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113907#comment-17113907
]
Hive QA commented on HIVE-23510:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
27s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
55s{color} | {color:blue} itests/util in master has 54 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22542/dev-support/hive-personality.sh
|
| git revision | master / 716f1f9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests itests/util U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22542/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestMiniLlapLocalCliDriver should be the default driver for q tests
> ---
>
> Key: HIVE-23510
> URL: https://issues.apache.org/jira/browse/HIVE-23510
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23510.01.patch, HIVE-23510.02.patch,
> HIVE-23510.03.patch, HIVE-23510.04.patch
>
>
> Set TestMiniLlapLocalCliDriver as the default driver. For now the few tests
> still processed by TestCliDriver should be marked in the
> testconfiguration.properties, until it is completely eliminated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23510) TestMiniLlapLocalCliDriver should be the default driver for q tests
[
https://issues.apache.org/jira/browse/HIVE-23510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113919#comment-17113919
]
Hive QA commented on HIVE-23510:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003725/HIVE-23510.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17277 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=186)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[special_character_in_tabnames_1]
(batchId=76)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22542/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22542/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22542/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003725 - PreCommit-HIVE-Build
> TestMiniLlapLocalCliDriver should be the default driver for q tests
> ---
>
> Key: HIVE-23510
> URL: https://issues.apache.org/jira/browse/HIVE-23510
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23510.01.patch, HIVE-23510.02.patch,
> HIVE-23510.03.patch, HIVE-23510.04.patch
>
>
> Set TestMiniLlapLocalCliDriver as the default driver. For now the few tests
> still processed by TestCliDriver should be marked in the
> testconfiguration.properties, until it is completely eliminated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21170) Wrong (no) results of cross-product query executed on LLAP
[
https://issues.apache.org/jira/browse/HIVE-21170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113945#comment-17113945
]
zhaolong commented on HIVE-21170:
-
hello, can you share a document about how to deploy llap on hive 3.1.0
> Wrong (no) results of cross-product query executed on LLAP
> ---
>
> Key: HIVE-21170
> URL: https://issues.apache.org/jira/browse/HIVE-21170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
> Environment: Hive distribution: HDP 3.1.0.
> LLAP execution engine in version:
> {code:java}
> $ beeline --version
> Hive 3.1.0.3.1.0.0-78
> Git
> git://ctr-e138-1518143905142-586755-01-15.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos7/SOURCES/hive
> -r 56673b027117d8cb3400675b1680a4d992360808
> Compiled by jenkins on Thu Dec 6 12:27:21 UTC 2018
> From source with checksum 97cc61f6acbe68b1fa988aa9f76b34cc{code}
>
>Reporter: Krzysztof Zarzycki
>Assignee: Ivan Suller
>Priority: Critical
>
> *In the environment:*
> {code:java}
> $ beeline --version
> Hive 3.1.0.3.1.0.0-78
> Git
> git://ctr-e138-1518143905142-586755-01-15.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos7/SOURCES/hive
> -r 56673b027117d8cb3400675b1680a4d992360808
> Compiled by jenkins on Thu Dec 6 12:27:21 UTC 2018
> From source with checksum 97cc61f6acbe68b1fa988aa9f76b34cc
> {code}
> *On LLAP execution engine*, the following query gives *wrong results*:
> {code:java}
> -- prepare test data
> set hive.query.results.cache.enabled=false;
> create table test1 (id int);
> insert into test1 values (1),(2),(3);
> -- query
> select * from test1 t1 cross join test1 t2;
> -- or any other cross join query, that involve 1 or more tables.{code}
> *Query result:*
> {code:java}
> 0: jdbc:hive2://hostname:> select t1.* from test1 t1 cross join test1 t2;
> INFO : Compiling
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6):
> select t1.* from test1 t1 cross join test1 t2
> INFO : Warning: Shuffle Join MERGEJOIN[9][tables = [$hdt$_0, $hdt$_1]] in
> Stage 'Reducer 2' is a cross product
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:t1.id,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6);
> Time taken: 0.229 seconds
> INFO : Executing
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6):
> select t1.* from test1 t1 cross join test1 t2
> INFO : Query ID = hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in parallel
>
> --
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
>
> --
> Map 1 .. llap SUCCEEDED 1 1 0 0 0 0
> Map 3 .. llap SUCCEEDED 1 1 0 0 0 0
> Reducer 2 llap SUCCEEDED 0 0 0 0 0 0
>
> --
> VERTICES: 02/03 [==>>] 100% ELAPSED TIME: 1.92 s
>
> --
> INFO : Completed executing
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6);
> Time taken: 2.006 seconds
> INFO : OK
> ++
> |t1.id|
> ++
> ++
> No rows selected (2.284 seconds)
> {code}
> *Expected result:*
> {code:java}
> ++
> |t1.id|
> ++
> |3|
> |3|
> |3|
> |2|
> |2|
> |2|
> |1|
> |1|
> |1|
> ++
> 9 rows selected
> {code}
> *What worked as a workaround:*
> {code:java}
> set hive.tez.cartesian-product.enabled=false; (default true)
> {code}
> Then query gave a correct result.
> *Difference in execution plans :*
> * With {{set hive.tez.cartesian-product.enabled=true;}}:
> {code:java}
> ++
> |Explain|
> ++
> |Plan optimized by CBO.|
> | |
> |Vertex dependency in root stage|
> |Reducer 2 <- Map 1 (XPROD_EDGE), Map 3 (XPROD_EDGE)|
> | |
> |Stage-0|
> |Fetch Operator|
> |limit:-1|
> |Stage-1|
> |Reducer 2 llap|
> |File Output Operator [FS_8]|
> |Merge Join Operator [MERGEJOIN_9] (rows=9 width=8)|
> |Conds:(Inner),Output:["_col0","_col1"]|
> |<-Map 1 [XPROD_EDGE] vectorized, llap|
> |XPROD_EDGE [RS_11]|
> |Select Operator [SEL_10] (rows=3 width=4)|
> |Output:["_col0"]|
> |TableScan [TS_0] (rows=3 width=4)|
> |zarz@test1,t1, ACID table,Tbl:COMPLETE,Col:CO
[jira] [Assigned] (HIVE-21170) Wrong (no) results of cross-product query executed on LLAP
[
https://issues.apache.org/jira/browse/HIVE-21170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Suller reassigned HIVE-21170:
--
Assignee: (was: Ivan Suller)
> Wrong (no) results of cross-product query executed on LLAP
> ---
>
> Key: HIVE-21170
> URL: https://issues.apache.org/jira/browse/HIVE-21170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
> Environment: Hive distribution: HDP 3.1.0.
> LLAP execution engine in version:
> {code:java}
> $ beeline --version
> Hive 3.1.0.3.1.0.0-78
> Git
> git://ctr-e138-1518143905142-586755-01-15.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos7/SOURCES/hive
> -r 56673b027117d8cb3400675b1680a4d992360808
> Compiled by jenkins on Thu Dec 6 12:27:21 UTC 2018
> From source with checksum 97cc61f6acbe68b1fa988aa9f76b34cc{code}
>
>Reporter: Krzysztof Zarzycki
>Priority: Critical
>
> *In the environment:*
> {code:java}
> $ beeline --version
> Hive 3.1.0.3.1.0.0-78
> Git
> git://ctr-e138-1518143905142-586755-01-15.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos7/SOURCES/hive
> -r 56673b027117d8cb3400675b1680a4d992360808
> Compiled by jenkins on Thu Dec 6 12:27:21 UTC 2018
> From source with checksum 97cc61f6acbe68b1fa988aa9f76b34cc
> {code}
> *On LLAP execution engine*, the following query gives *wrong results*:
> {code:java}
> -- prepare test data
> set hive.query.results.cache.enabled=false;
> create table test1 (id int);
> insert into test1 values (1),(2),(3);
> -- query
> select * from test1 t1 cross join test1 t2;
> -- or any other cross join query, that involve 1 or more tables.{code}
> *Query result:*
> {code:java}
> 0: jdbc:hive2://hostname:> select t1.* from test1 t1 cross join test1 t2;
> INFO : Compiling
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6):
> select t1.* from test1 t1 cross join test1 t2
> INFO : Warning: Shuffle Join MERGEJOIN[9][tables = [$hdt$_0, $hdt$_1]] in
> Stage 'Reducer 2' is a cross product
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:t1.id,
> type:int, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6);
> Time taken: 0.229 seconds
> INFO : Executing
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6):
> select t1.* from test1 t1 cross join test1 t2
> INFO : Query ID = hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in parallel
>
> --
> VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
>
> --
> Map 1 .. llap SUCCEEDED 1 1 0 0 0 0
> Map 3 .. llap SUCCEEDED 1 1 0 0 0 0
> Reducer 2 llap SUCCEEDED 0 0 0 0 0 0
>
> --
> VERTICES: 02/03 [==>>] 100% ELAPSED TIME: 1.92 s
>
> --
> INFO : Completed executing
> command(queryId=hive_20190125215942_7df8062c-8511-4915-a0d9-5e7ac84030f6);
> Time taken: 2.006 seconds
> INFO : OK
> ++
> |t1.id|
> ++
> ++
> No rows selected (2.284 seconds)
> {code}
> *Expected result:*
> {code:java}
> ++
> |t1.id|
> ++
> |3|
> |3|
> |3|
> |2|
> |2|
> |2|
> |1|
> |1|
> |1|
> ++
> 9 rows selected
> {code}
> *What worked as a workaround:*
> {code:java}
> set hive.tez.cartesian-product.enabled=false; (default true)
> {code}
> Then query gave a correct result.
> *Difference in execution plans :*
> * With {{set hive.tez.cartesian-product.enabled=true;}}:
> {code:java}
> ++
> |Explain|
> ++
> |Plan optimized by CBO.|
> | |
> |Vertex dependency in root stage|
> |Reducer 2 <- Map 1 (XPROD_EDGE), Map 3 (XPROD_EDGE)|
> | |
> |Stage-0|
> |Fetch Operator|
> |limit:-1|
> |Stage-1|
> |Reducer 2 llap|
> |File Output Operator [FS_8]|
> |Merge Join Operator [MERGEJOIN_9] (rows=9 width=8)|
> |Conds:(Inner),Output:["_col0","_col1"]|
> |<-Map 1 [XPROD_EDGE] vectorized, llap|
> |XPROD_EDGE [RS_11]|
> |Select Operator [SEL_10] (rows=3 width=4)|
> |Output:["_col0"]|
> |TableScan [TS_0] (rows=3 width=4)|
> |zarz@test1,t1, ACID table,Tbl:COMPLETE,Col:COMPLETE,Output:["id"]|
> |<-Map 3 [XPROD_EDGE] vectorized, llap|
> |XPROD_EDGE [RS_13]|
> |Select Operator [SEL_12]
[jira] [Commented] (HIVE-23529) CTAS is broken for uniontype when row_deserialize
[
https://issues.apache.org/jira/browse/HIVE-23529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113956#comment-17113956
]
Hive QA commented on HIVE-23529:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003699/HIVE-23529.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 17278 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=187)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row_deserialize_with_union]
(batchId=6)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=173)
org.apache.hadoop.hive.serde2.objectinspector.TestStandardObjectInspectors.testStandardUnionObjectInspector
(batchId=309)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22543/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22543/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22543/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003699 - PreCommit-HIVE-Build
> CTAS is broken for uniontype when row_deserialize
> -
>
> Key: HIVE-23529
> URL: https://issues.apache.org/jira/browse/HIVE-23529
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa Iman
>Assignee: Mustafa Iman
>Priority: Major
> Attachments: HIVE-23529.patch
>
>
> CTAS queries fail when there is a uniontype in source table and
> hive.vectorized.use.vector.serde.deserialize=false.
> ObjectInspectorUtils.copyToStandardObject in ROW_DESERIALIZE path extracts
> the value from union type. However, VectorAssignRow expects a StandardUnion
> object causing ClassCastException for any CTAS query.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-23526) Out of sequence seen in Beeline may swallow the real problem
[ https://issues.apache.org/jira/browse/HIVE-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113097#comment-17113097 ] Zhihua Deng edited comment on HIVE-23526 at 5/22/20, 11:45 AM: --- Add some codes to catch the missing exception, the attached log shows the case caused by a oom error. was (Author: dengzh): Add some codes to catch the missing exception, the attached log shows the case caused by a oom error. > Out of sequence seen in Beeline may swallow the real problem > - > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.patch, outofsequence.log > > Time Spent: 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage messages to figured it out, this makes problem > puzzled as beeline does not have concurrency problem on underlying thrift > transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23526) Out of sequence seen in Beeline may swallow the real problem
[ https://issues.apache.org/jira/browse/HIVE-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated HIVE-23526: --- Description: Sometimes we can see 'out of sequence response' message in beeline, for example: Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of sequence response (state=08S01,code=0) java.sql.SQLException: org.apache.thrift.TApplicationException: CloseOperation failed: out of sequence response at org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) at org.apache.hive.beeline.Commands.execute(Commands.java:891) at org.apache.hive.beeline.Commands.sql(Commands.java:713) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) and there is no other usage messages to figured it out, even with verbose flag, this makes problem puzzled as beeline does not have concurrency problem on underlying thrift transport. was: Sometimes we can see 'out of sequence response' message in beeline, for example: Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of sequence response (state=08S01,code=0) java.sql.SQLException: org.apache.thrift.TApplicationException: CloseOperation failed: out of sequence response at org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) at org.apache.hive.beeline.Commands.execute(Commands.java:891) at org.apache.hive.beeline.Commands.sql(Commands.java:713) at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) and there is no other usage messages to figured it out, this makes problem puzzled as beeline does not have concurrency problem on underlying thrift transport. > Out of sequence seen in Beeline may swallow the real problem > - > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.patch, outofsequence.log > > Time Spent: 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage messages to figured it out, even with verbose > flag, this makes problem puzzled as beeline does not have concurrency problem > on underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23526) Out of sequence seen in Beeline may swallow the real problem
[ https://issues.apache.org/jira/browse/HIVE-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113959#comment-17113959 ] Zhihua Deng commented on HIVE-23526: The failed test is caused by: Unknown HS2 problem when communicating with Thrift server. Enable verbose error messages (--verbose=true) for more information. Error: Failed to wait for operation to complete (state=08S01,code=0) In my environment, the failed test shows that a read timeout exception happens: Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketTimeoutException: Read timed out at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:374) at org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:451) at org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:433) at org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) at org.apache.hive.service.rpc.thrift.TCLIService$Client.recv_GetOperationStatus(TCLIService.java:475) at org.apache.hive.service.rpc.thrift.TCLIService$Client.GetOperationStatus(TCLIService.java:462) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hive.jdbc.HiveConnection$SynchronizedHandler.invoke(HiveConnection.java:1732) at com.sun.proxy.$Proxy131.GetOperationStatus(Unknown Source) at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:348) ... 52 more I will fix the problem lately. > Out of sequence seen in Beeline may swallow the real problem > - > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.patch, outofsequence.log > > Time Spent: 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage messages to figured it out, even with verbose > flag, this makes problem puzzled as beeline does not have concurrency problem > on underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23529) CTAS is broken for uniontype when row_deserialize
[
https://issues.apache.org/jira/browse/HIVE-23529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113969#comment-17113969
]
Hive QA commented on HIVE-23529:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
51s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
18s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
46s{color} | {color:red} patch/serde cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
33s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 15m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 82m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22543/dev-support/hive-personality.sh
|
| git revision | master / 716f1f9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22543/yetus/patch-findbugs-serde.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22543/yetus/patch-findbugs-ql.txt
|
| modules | C: serde ql . itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22543/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CTAS is broken for uniontype when row_deserialize
> -
>
> Key: HIVE-23529
> URL: https://issues.apache.org/jira/browse/HIVE-23529
> Project: Hive
> Issue Type: Bug
>Reporter: Mustafa Iman
>Assignee: Mustafa Iman
>Priority: Major
> Attachments: HIVE-23529.patch
>
>
> CTAS queries fail when there is a uniontype in source table and
> hive.vectorized.use.vector.serde.deserialize=false.
> ObjectInspectorUtils.copyToStandardObject in ROW_DESERIALIZE path extracts
> the value from union type. However, VectorAssignRow expects a StandardUnion
> object causing ClassCastException for any CTAS query.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22017) Keep HMS interfaces backward compatible with changes for HIVE-21637
[
https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17113984#comment-17113984
]
Hive QA commented on HIVE-22017:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} storage-api in master has 50 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
16s{color} | {color:red} metastore-common in master failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
16s{color} | {color:red} metastore-server in master failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 6 new + 409 unchanged - 0 fixed = 415 total (was 409) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
30s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 12 new + 633 unchanged - 1 fixed = 645 total (was 634) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
16s{color} | {color:red} metastore-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
17s{color} | {color:red} metastore-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22544/dev-support/hive-personality.sh
|
| git revision | master / 716f1f9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22544/yetus/branch-findbugs-standalone-metastore_metastore-common.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22544/yetus/branch-findbugs-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22544/yetus/diff-checkstyle-standalone-metastore_metastore-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22544/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22544/yetus/patch-findbugs-standalone-metastore_metastore-common.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22544/yetus/patch-findbugs-standalone-metastore_metastore-server.txt
|
| modules | C: storage-api standalone-metastore
[jira] [Updated] (HIVE-23519) Read Ranger Configs from Classpath
[ https://issues.apache.org/jira/browse/HIVE-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23519: --- Status: In Progress (was: Patch Available) > Read Ranger Configs from Classpath > -- > > Key: HIVE-23519 > URL: https://issues.apache.org/jira/browse/HIVE-23519 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23519.01.patch, HIVE-23519.02.patch, > HIVE-23519.03.patch, HIVE-23519.04.patch, HIVE-23519.05.patch, > HIVE-23519.06.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23519) Read Ranger Configs from Classpath
[ https://issues.apache.org/jira/browse/HIVE-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23519: --- Attachment: HIVE-23519.06.patch Status: Patch Available (was: In Progress) > Read Ranger Configs from Classpath > -- > > Key: HIVE-23519 > URL: https://issues.apache.org/jira/browse/HIVE-23519 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23519.01.patch, HIVE-23519.02.patch, > HIVE-23519.03.patch, HIVE-23519.04.patch, HIVE-23519.05.patch, > HIVE-23519.06.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23531) Major CRUD QB compaction failing with ClassCastException when vectorization off
[
https://issues.apache.org/jira/browse/HIVE-23531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage reassigned HIVE-23531:
> Major CRUD QB compaction failing with ClassCastException when vectorization
> off
> ---
>
> Key: HIVE-23531
> URL: https://issues.apache.org/jira/browse/HIVE-23531
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
>
> Exception:
> {code:java}
> 2020-05-22T01:33:09,944 ERROR [TezChild] tez.MapRecordSource:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:403)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
> org.apache.hadoop.io.IntWritable
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:965)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:174)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> ... 20 more
> {code}
> And some more in Tez.
> Because when vectorization is turned on, primitives in the row are wrapped in
> Writables by VectorFileSinkOperator; when it is off, they are not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23531) Major CRUD QB compaction failing with ClassCastException when vectorization off
[
https://issues.apache.org/jira/browse/HIVE-23531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-23531:
-
Attachment: HIVE-23531.01.patch
> Major CRUD QB compaction failing with ClassCastException when vectorization
> off
> ---
>
> Key: HIVE-23531
> URL: https://issues.apache.org/jira/browse/HIVE-23531
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23531.01.patch
>
>
> Exception:
> {code:java}
> 2020-05-22T01:33:09,944 ERROR [TezChild] tez.MapRecordSource:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:403)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
> org.apache.hadoop.io.IntWritable
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:965)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:174)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> ... 20 more
> {code}
> And some more in Tez.
> Because when vectorization is turned on, primitives in the row are wrapped in
> Writables by VectorFileSinkOperator; when it is off, they are not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23531) Major CRUD QB compaction failing with ClassCastException when vectorization off
[
https://issues.apache.org/jira/browse/HIVE-23531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114006#comment-17114006
]
László Pintér commented on HIVE-23531:
--
+1, pending tests
> Major CRUD QB compaction failing with ClassCastException when vectorization
> off
> ---
>
> Key: HIVE-23531
> URL: https://issues.apache.org/jira/browse/HIVE-23531
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23531.01.patch
>
>
> Exception:
> {code:java}
> 2020-05-22T01:33:09,944 ERROR [TezChild] tez.MapRecordSource:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:403)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
> org.apache.hadoop.io.IntWritable
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:965)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:174)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> ... 20 more
> {code}
> And some more in Tez.
> Because when vectorization is turned on, primitives in the row are wrapped in
> Writables by VectorFileSinkOperator; when it is off, they are not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22017) Keep HMS interfaces backward compatible with changes for HIVE-21637
[
https://issues.apache.org/jira/browse/HIVE-22017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114015#comment-17114015
]
Hive QA commented on HIVE-22017:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003705/HIVE-216371.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 26 failed/errored test(s), 17285 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionReq[Embedded]
(batchId=157)
org.apache.hadoop.hive.metastore.client.TestGetPartitions.testGetPartitionReq[Remote]
(batchId=157)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientGetPartitionsTempTable.testGetPartitionReq[Embedded]
(batchId=271)
org.apache.hadoop.hive.ql.metadata.TestSessionHiveMetastoreClientGetPartitionsTempTable.testGetPartitionReq[Remote]
(batchId=271)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testTxnTable
(batchId=259)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testCancelRenewTokenFlow
(batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testConnection
(batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testIsValid (batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testIsValidNeg
(batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeProxyAuth
(batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeTokenAuth
(batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNoKrbSASLTokenAuthNeg
(batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNonKrbSASLAuth
(batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNonKrbSASLFullNameAuth
(batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testProxyAuth
(batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testRenewDelegationToken
(batchId=238)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testTokenAuth
(batchId=238)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testCancelRenewTokenFlow
(batchId=238)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testConnection
(batchId=238)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValid (batchId=238)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValidNeg
(batchId=238)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeProxyAuth
(batchId=238)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeTokenAuth
(batchId=238)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testProxyAuth
(batchId=238)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testRenewDelegationToken
(batchId=238)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testTokenAuth
(batchId=238)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22544/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22544/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22544/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 26 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003705 - PreCommit-HIVE-Build
> Keep HMS interfaces backward compatible with changes for HIVE-21637
> ---
>
> Key: HIVE-22017
> URL: https://issues.apache.org/jira/browse/HIVE-22017
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 2.3.7
>Reporter: Daniel Dai
>Assignee: Kishen Das
>Priority: Major
> Attachments: HIVE-216371.1.patch, HIVE-216371.2.patch
>
>
> As part of HIVE-21637 we would have to introduce ValidWriteIdList in several
> methods. Also, in the long term, we should deprecate and remove all the
> methods that take direct arguments, as the service definition keeps changing
> whenever we add/remove arguments, making it hard to maintain backward
> compatibility. So, instead of adding writeId in bunch of get_xxx calls that
> take direct arguments, we will create new set of methods that take Request
> object and return Response object. We shall mark those deprecated and remove
> in future version.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23530) Use SQL functions instead of compute_stats UDAF to compute column statistics
[
https://issues.apache.org/jira/browse/HIVE-23530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114050#comment-17114050
]
Hive QA commented on HIVE-23530:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
42s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
57s{color} | {color:red} ql: The patch generated 21 new + 138 unchanged - 59
fixed = 159 total (was 197) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22545/dev-support/hive-personality.sh
|
| git revision | master / 716f1f9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22545/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22545/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use SQL functions instead of compute_stats UDAF to compute column statistics
>
>
> Key: HIVE-23530
> URL: https://issues.apache.org/jira/browse/HIVE-23530
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-23530.patch
>
>
> Currently we compute column statistics by relying on the {{compute_stats}}
> UDAF. For instance, for a given table {{tbl}}, the query to compute
> statistics for columns is translated internally into:
> {code}
> SELECT compute_stats(c1),
>compute_stats(c2),
>...
> FROM tbl;
> {code}
> {{compute_stats}} produces data for the stats available for each column type,
> e.g., struct<"max":long,"min":long,"countnulls":long,...>.
> This issue is to produce a query that relies purely on SQL functions instead:
> {code}
> SELECT max(c1), min(c1), count(case when c1 is null then 1 else null end),
>...
> FROM tbl;
> {code}
> This will allow us to deprecate the {{compute_stats}} UDAF since it mostly
> duplicates functionality found in those other functions. Additionally, many
> of those functions already provide a vectorized implementation so the
> approach can potentially improve the performance of column stats collection.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23532) NPE when fetching incomplete column statistics from the metastore
[
https://issues.apache.org/jira/browse/HIVE-23532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stamatis Zampetakis updated HIVE-23532:
---
Attachment: HIVE-23532.01.patch
> NPE when fetching incomplete column statistics from the metastore
> -
>
> Key: HIVE-23532
> URL: https://issues.apache.org/jira/browse/HIVE-23532
> Project: Hive
> Issue Type: Bug
>Reporter: Stamatis Zampetakis
>Priority: Minor
> Attachments: HIVE-23532.01.patch
>
>
> Certain operations may store in the metastore incomplete column statistics.
> Fetching those statistics back from the metastore leads to
> {{NullPointerException}} .
> For instance consider a column "name" of type string. If we do have
> statistics for this column then the following info must be available:
> * maxColLen;
> * avgColLen;
> * numNulls;
> * numDVs;
> Executing the following statement on a table with no stats updates a subset
> of the statistics for this column:
> {code:sql}
> ALTER TABLE example UPDATE STATISTICS for column name SET ('numDVs'='242',
> 'numNulls'='5');
> {code}
> Fetching this kind of statistics leads to NPE that sometimes pops up in the
> client and some other times is buried in the logs leading to incomplete
> column stats during optimization and execution of a query.
> Usually the stacktrace is similar to the one below:
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getTableColumnStatistics(Hive.java:5251)
> at
> org.apache.hadoop.hive.ql.ddl.table.info.desc.DescTableOperation.getColumnDataColPathSpecified(DescTableOperation.java:216)
> at
> org.apache.hadoop.hive.ql.ddl.table.info.desc.DescTableOperation.execute(DescTableOperation.java:94)
> at org.apache.hadoop.hive.ql.ddl.DDLTask.execute(DDLTask.java:80)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:362)
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:335)
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:723)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:492)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:486)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:164)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:230)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:353)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:730)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:700)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:170)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157)
> at
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:62)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:135)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>
[jira] [Commented] (HIVE-23532) NPE when fetching incomplete column statistics from the metastore
[
https://issues.apache.org/jira/browse/HIVE-23532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114061#comment-17114061
]
Stamatis Zampetakis commented on HIVE-23532:
The HIVE-23532.01.patch contains only a minimal reproducer.
Apart from tests the scenario should not appear very often so I am putting this
to low priority.
> NPE when fetching incomplete column statistics from the metastore
> -
>
> Key: HIVE-23532
> URL: https://issues.apache.org/jira/browse/HIVE-23532
> Project: Hive
> Issue Type: Bug
>Reporter: Stamatis Zampetakis
>Priority: Minor
> Attachments: HIVE-23532.01.patch
>
>
> Certain operations may store in the metastore incomplete column statistics.
> Fetching those statistics back from the metastore leads to
> {{NullPointerException}} .
> For instance consider a column "name" of type string. If we do have
> statistics for this column then the following info must be available:
> * maxColLen;
> * avgColLen;
> * numNulls;
> * numDVs;
> Executing the following statement on a table with no stats updates a subset
> of the statistics for this column:
> {code:sql}
> ALTER TABLE example UPDATE STATISTICS for column name SET ('numDVs'='242',
> 'numNulls'='5');
> {code}
> Fetching this kind of statistics leads to NPE that sometimes pops up in the
> client and some other times is buried in the logs leading to incomplete
> column stats during optimization and execution of a query.
> Usually the stacktrace is similar to the one below:
> {noformat}
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.metadata.Hive.getTableColumnStatistics(Hive.java:5251)
> at
> org.apache.hadoop.hive.ql.ddl.table.info.desc.DescTableOperation.getColumnDataColPathSpecified(DescTableOperation.java:216)
> at
> org.apache.hadoop.hive.ql.ddl.table.info.desc.DescTableOperation.execute(DescTableOperation.java:94)
> at org.apache.hadoop.hive.ql.ddl.DDLTask.execute(DDLTask.java:80)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:362)
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:335)
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:723)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:492)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:486)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:164)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:230)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:353)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:730)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:700)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:170)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157)
> at
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:62)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:135)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.j
[jira] [Assigned] (HIVE-23533) Remove an FS#exists call from AcidUtils#getLogicalLength
[
https://issues.apache.org/jira/browse/HIVE-23533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage reassigned HIVE-23533:
> Remove an FS#exists call from AcidUtils#getLogicalLength
>
>
> Key: HIVE-23533
> URL: https://issues.apache.org/jira/browse/HIVE-23533
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
>
> {code:java}
> Path lengths = OrcAcidUtils.getSideFile(file.getPath());
> if(!fs.exists(lengths)) {
> ...
> return file.getLen();
> }
> long len = OrcAcidUtils.getLastFlushLength(fs, file.getPath());
> {code}
> OrcAcidUtils.getLastFlushLength also has an exists() check and returns
> Long.MAX_VALUE if false.
> exists() is expensive on S3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23533) Remove an FS#exists call from AcidUtils#getLogicalLength
[
https://issues.apache.org/jira/browse/HIVE-23533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-23533:
-
Attachment: HIVE-23533.01.patch
Status: Patch Available (was: Open)
> Remove an FS#exists call from AcidUtils#getLogicalLength
>
>
> Key: HIVE-23533
> URL: https://issues.apache.org/jira/browse/HIVE-23533
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23533.01.patch
>
>
> {code:java}
> Path lengths = OrcAcidUtils.getSideFile(file.getPath());
> if(!fs.exists(lengths)) {
> ...
> return file.getLen();
> }
> long len = OrcAcidUtils.getLastFlushLength(fs, file.getPath());
> {code}
> OrcAcidUtils.getLastFlushLength also has an exists() check and returns
> Long.MAX_VALUE if false.
> exists() is expensive on S3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23531) Major CRUD QB compaction failing with ClassCastException when vectorization off
[
https://issues.apache.org/jira/browse/HIVE-23531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-23531:
-
Attachment: HIVE-23531.01.patch
Status: Patch Available (was: Open)
> Major CRUD QB compaction failing with ClassCastException when vectorization
> off
> ---
>
> Key: HIVE-23531
> URL: https://issues.apache.org/jira/browse/HIVE-23531
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23531.01.patch, HIVE-23531.01.patch
>
>
> Exception:
> {code:java}
> 2020-05-22T01:33:09,944 ERROR [TezChild] tez.MapRecordSource:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:403)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to
> org.apache.hadoop.io.IntWritable
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:965)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:174)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> ... 20 more
> {code}
> And some more in Tez.
> Because when vectorization is turned on, primitives in the row are wrapped in
> Writables by VectorFileSinkOperator; when it is off, they are not.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23534) NPE in RetryingMetaStoreClient#invoke when catching MetaException with no message
[ https://issues.apache.org/jira/browse/HIVE-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis reassigned HIVE-23534: -- > NPE in RetryingMetaStoreClient#invoke when catching MetaException with no > message > - > > Key: HIVE-23534 > URL: https://issues.apache.org/jira/browse/HIVE-23534 > Project: Hive > Issue Type: Bug >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Major > > RetryingMetaStoreClient#invoke method catches MetaException and attempts to > classify it by checking the message. However there are cases (e.g., various > places in > [ObjectStore|https://github.com/apache/hive/blob/716f1f9a945a9a11e6702754667660d27e0a5cf4/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3916]) > where the message of the MetaException is null and this leads to NPE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23534) NPE in RetryingMetaStoreClient#invoke when catching MetaException with no message
[ https://issues.apache.org/jira/browse/HIVE-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-23534: --- Priority: Trivial (was: Major) > NPE in RetryingMetaStoreClient#invoke when catching MetaException with no > message > - > > Key: HIVE-23534 > URL: https://issues.apache.org/jira/browse/HIVE-23534 > Project: Hive > Issue Type: Bug >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Trivial > > RetryingMetaStoreClient#invoke method catches MetaException and attempts to > classify it by checking the message. However there are cases (e.g., various > places in > [ObjectStore|https://github.com/apache/hive/blob/716f1f9a945a9a11e6702754667660d27e0a5cf4/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3916]) > where the message of the MetaException is null and this leads to NPE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23530) Use SQL functions instead of compute_stats UDAF to compute column statistics
[
https://issues.apache.org/jira/browse/HIVE-23530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114076#comment-17114076
]
Hive QA commented on HIVE-23530:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003709/HIVE-23530.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 591 failed/errored test(s), 17281 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
(batchId=225)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[desc_table_formatted]
(batchId=233)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_11]
(batchId=233)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_12]
(batchId=233)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_7]
(batchId=233)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[explain]
(batchId=239)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table]
(batchId=239)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_table]
(batchId=239)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[write_final_output_blobstore]
(batchId=239)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[zero_rows_blobstore]
(batchId=239)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[zero_rows_hdfs]
(batchId=239)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark1]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark2]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark3]
(batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin5]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative2]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_map_ppr] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_8]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonmr_fetch] (batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullformatCTAS]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin9] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_20]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_21]
(batchId=4)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes2]
(batchId=229)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes3]
(batchId=229)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes4]
(batchId=229)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes5]
(batchId=229)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[serde_typedbytes]
(batchId=229)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_single_sourced_multi_insert]
(batchId=16)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_semijoin_reduction_all_types]
(batchId=131)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[autoColumnStats_6]
(batchId=27)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[binary_output_format]
(batchId=29)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[bucket5]
(batchId=28)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[bucket6]
(batchId=20)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_semijoin_user_level]
(batchId=27)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynpart_cast]
(batchId=28)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2]
(batchId=27)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[input14]
(batchId=28)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[input17]
(batchId=24)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[input18]
(batchId=21)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[input20]
(batchId=20)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[input33]
(batchId=26)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[input34]
(batchId=21)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[input35]
(batchId=26)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[input36]
(batchId=21)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[input38]
(batchI
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks [~ashutoshc] for the review! > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22066.10.patch, HIVE-22066.11.patch, > HIVE-22066.12.patch, HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch, > HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23521) REPL: Optimise partition loading during bootstrap
[
https://issues.apache.org/jira/browse/HIVE-23521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114128#comment-17114128
]
Hive QA commented on HIVE-23521:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
22s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22546/dev-support/hive-personality.sh
|
| git revision | master / 716f1f9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22546/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> REPL: Optimise partition loading during bootstrap
> -
>
> Key: HIVE-23521
> URL: https://issues.apache.org/jira/browse/HIVE-23521
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-23521.1.patch
>
>
> When bootstrapping with large "REPL dump" with ~10K partitions, it starts
> executing "addPartition" in sequential manner and takes very long time as it
> communicates with HMS/registers partition etc for every call.
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadPartitions.java#L399]
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadPartitions.java#L165]
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadPartitions.java#L210]
> When bootstrap loading has to deal with DDL, it would be good to collate all
> partitions in single call to HMS. This would help in reducing overall runtime.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23535) Bump Minimum Required Version of Maven to 3.0.5
[
https://issues.apache.org/jira/browse/HIVE-23535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23535:
--
Status: Patch Available (was: Open)
> Bump Minimum Required Version of Maven to 3.0.5
> ---
>
> Key: HIVE-23535
> URL: https://issues.apache.org/jira/browse/HIVE-23535
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23535.1.patch
>
>
> {code:xml|title=pom.xml}
>
> 2.2.1
>
> {code}
> Time to upgrade to 3.x
> https://maven.apache.org/pom.html#prerequisites
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23535) Bump Minimum Required Version of Maven to 3.0.5
[
https://issues.apache.org/jira/browse/HIVE-23535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor reassigned HIVE-23535:
-
> Bump Minimum Required Version of Maven to 3.0.5
> ---
>
> Key: HIVE-23535
> URL: https://issues.apache.org/jira/browse/HIVE-23535
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23535.1.patch
>
>
> {code:xml|title=pom.xml}
>
> 2.2.1
>
> {code}
> Time to upgrade to 3.x
> https://maven.apache.org/pom.html#prerequisites
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23535) Bump Minimum Required Version of Maven to 3.0.5
[
https://issues.apache.org/jira/browse/HIVE-23535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23535:
--
Attachment: HIVE-23535.1.patch
> Bump Minimum Required Version of Maven to 3.0.5
> ---
>
> Key: HIVE-23535
> URL: https://issues.apache.org/jira/browse/HIVE-23535
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23535.1.patch
>
>
> {code:xml|title=pom.xml}
>
> 2.2.1
>
> {code}
> Time to upgrade to 3.x
> https://maven.apache.org/pom.html#prerequisites
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23535) Bump Minimum Required Version of Maven to 3.0.5
[
https://issues.apache.org/jira/browse/HIVE-23535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23535:
--
Description:
{code:xml|title=pom.xml}
2.2.1
{code}
Time to upgrade to 3.x
bq. In Maven 3, use Maven Enforcer Plugin's requireMavenVersion rule, or other
rules to check other aspects.
https://maven.apache.org/pom.html#prerequisites
was:
{code:xml|title=pom.xml}
2.2.1
{code}
Time to upgrade to 3.x
https://maven.apache.org/pom.html#prerequisites
> Bump Minimum Required Version of Maven to 3.0.5
> ---
>
> Key: HIVE-23535
> URL: https://issues.apache.org/jira/browse/HIVE-23535
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23535.1.patch
>
>
> {code:xml|title=pom.xml}
>
> 2.2.1
>
> {code}
> Time to upgrade to 3.x
> bq. In Maven 3, use Maven Enforcer Plugin's requireMavenVersion rule, or
> other rules to check other aspects.
> https://maven.apache.org/pom.html#prerequisites
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23535) Bump Minimum Required Version of Maven to 3.0.5
[
https://issues.apache.org/jira/browse/HIVE-23535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23535:
--
Description:
{code:xml|title=pom.xml}
2.2.1
{code}
Time to upgrade to 3.x
{quote}In Maven 3, use Maven Enforcer Plugin's requireMavenVersion rule, or
other rules to check other aspects.
{quote}
[https://maven.apache.org/pom.html#prerequisites]
We get the Enforcer Plugin from the Apache Parent POM.
was:
{code:xml|title=pom.xml}
2.2.1
{code}
Time to upgrade to 3.x
bq. In Maven 3, use Maven Enforcer Plugin's requireMavenVersion rule, or other
rules to check other aspects.
https://maven.apache.org/pom.html#prerequisites
> Bump Minimum Required Version of Maven to 3.0.5
> ---
>
> Key: HIVE-23535
> URL: https://issues.apache.org/jira/browse/HIVE-23535
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23535.1.patch
>
>
> {code:xml|title=pom.xml}
>
> 2.2.1
>
> {code}
> Time to upgrade to 3.x
> {quote}In Maven 3, use Maven Enforcer Plugin's requireMavenVersion rule, or
> other rules to check other aspects.
> {quote}
> [https://maven.apache.org/pom.html#prerequisites]
>
> We get the Enforcer Plugin from the Apache Parent POM.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23533) Remove an FS#exists call from AcidUtils#getLogicalLength
[
https://issues.apache.org/jira/browse/HIVE-23533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114137#comment-17114137
]
Peter Vary commented on HIVE-23533:
---
+1 pending tests
> Remove an FS#exists call from AcidUtils#getLogicalLength
>
>
> Key: HIVE-23533
> URL: https://issues.apache.org/jira/browse/HIVE-23533
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23533.01.patch
>
>
> {code:java}
> Path lengths = OrcAcidUtils.getSideFile(file.getPath());
> if(!fs.exists(lengths)) {
> ...
> return file.getLen();
> }
> long len = OrcAcidUtils.getLastFlushLength(fs, file.getPath());
> {code}
> OrcAcidUtils.getLastFlushLength also has an exists() check and returns
> Long.MAX_VALUE if false.
> exists() is expensive on S3.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23536) Provide an option to skip stats generation for major compaction
[ https://issues.apache.org/jira/browse/HIVE-23536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary reassigned HIVE-23536: - > Provide an option to skip stats generation for major compaction > --- > > Key: HIVE-23536 > URL: https://issues.apache.org/jira/browse/HIVE-23536 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > > Currently major MR compaction is regenerates stats every time if the column > stats table contains some data. Some configurations do not use stats but > because of historical reasons the column stats table can still contain some > data. > We should provide a possibility to skip stats generation in these cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23536) Provide an option to skip stats generation for major compaction
[ https://issues.apache.org/jira/browse/HIVE-23536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-23536: -- Attachment: HIVE-23536.patch > Provide an option to skip stats generation for major compaction > --- > > Key: HIVE-23536 > URL: https://issues.apache.org/jira/browse/HIVE-23536 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-23536.patch > > > Currently major MR compaction is regenerates stats every time if the column > stats table contains some data. Some configurations do not use stats but > because of historical reasons the column stats table can still contain some > data. > We should provide a possibility to skip stats generation in these cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23536) Provide an option to skip stats generation for major compaction
[ https://issues.apache.org/jira/browse/HIVE-23536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-23536: -- Status: Patch Available (was: Open) > Provide an option to skip stats generation for major compaction > --- > > Key: HIVE-23536 > URL: https://issues.apache.org/jira/browse/HIVE-23536 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-23536.patch > > > Currently major MR compaction is regenerates stats every time if the column > stats table contains some data. Some configurations do not use stats but > because of historical reasons the column stats table can still contain some > data. > We should provide a possibility to skip stats generation in these cases. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23521) REPL: Optimise partition loading during bootstrap
[
https://issues.apache.org/jira/browse/HIVE-23521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114176#comment-17114176
]
Hive QA commented on HIVE-23521:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003713/HIVE-23521.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17281 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths
(batchId=183)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22546/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22546/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22546/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003713 - PreCommit-HIVE-Build
> REPL: Optimise partition loading during bootstrap
> -
>
> Key: HIVE-23521
> URL: https://issues.apache.org/jira/browse/HIVE-23521
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-23521.1.patch
>
>
> When bootstrapping with large "REPL dump" with ~10K partitions, it starts
> executing "addPartition" in sequential manner and takes very long time as it
> communicates with HMS/registers partition etc for every call.
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadPartitions.java#L399]
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadPartitions.java#L165]
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/table/LoadPartitions.java#L210]
> When bootstrap loading has to deal with DDL, it would be good to collate all
> partitions in single call to HMS. This would help in reducing overall runtime.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23534) NPE in RetryingMetaStoreClient#invoke when catching MetaException with no message
[ https://issues.apache.org/jira/browse/HIVE-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-23534: --- Attachment: HIVE-23534.01.patch > NPE in RetryingMetaStoreClient#invoke when catching MetaException with no > message > - > > Key: HIVE-23534 > URL: https://issues.apache.org/jira/browse/HIVE-23534 > Project: Hive > Issue Type: Bug >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Trivial > Attachments: HIVE-23534.01.patch > > > RetryingMetaStoreClient#invoke method catches MetaException and attempts to > classify it by checking the message. However there are cases (e.g., various > places in > [ObjectStore|https://github.com/apache/hive/blob/716f1f9a945a9a11e6702754667660d27e0a5cf4/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3916]) > where the message of the MetaException is null and this leads to NPE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23534) NPE in RetryingMetaStoreClient#invoke when catching MetaException with no message
[ https://issues.apache.org/jira/browse/HIVE-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stamatis Zampetakis updated HIVE-23534: --- Status: Patch Available (was: Open) # Extract retrying check of MetaException into method. # Add check for null message. # Precompile pattern for MetaException message. > NPE in RetryingMetaStoreClient#invoke when catching MetaException with no > message > - > > Key: HIVE-23534 > URL: https://issues.apache.org/jira/browse/HIVE-23534 > Project: Hive > Issue Type: Bug >Reporter: Stamatis Zampetakis >Assignee: Stamatis Zampetakis >Priority: Trivial > Attachments: HIVE-23534.01.patch > > > RetryingMetaStoreClient#invoke method catches MetaException and attempts to > classify it by checking the message. However there are cases (e.g., various > places in > [ObjectStore|https://github.com/apache/hive/blob/716f1f9a945a9a11e6702754667660d27e0a5cf4/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L3916]) > where the message of the MetaException is null and this leads to NPE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23519) Read Ranger Configs from Classpath
[ https://issues.apache.org/jira/browse/HIVE-23519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114195#comment-17114195 ] Pravin Sinha commented on HIVE-23519: - +1 > Read Ranger Configs from Classpath > -- > > Key: HIVE-23519 > URL: https://issues.apache.org/jira/browse/HIVE-23519 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23519.01.patch, HIVE-23519.02.patch, > HIVE-23519.03.patch, HIVE-23519.04.patch, HIVE-23519.05.patch, > HIVE-23519.06.patch > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23269) Unsafe comparing bigints and strings
[
https://issues.apache.org/jira/browse/HIVE-23269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114200#comment-17114200
]
Hive QA commented on HIVE-23269:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
18s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
55s{color} | {color:red} ql: The patch generated 1 new + 94 unchanged - 1 fixed
= 95 total (was 95) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22547/dev-support/hive-personality.sh
|
| git revision | master / e649562 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22547/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22547/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22547/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Unsafe comparing bigints and strings
>
>
> Key: HIVE-23269
> URL: https://issues.apache.org/jira/browse/HIVE-23269
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Major
> Attachments: HIVE-23269.2.patch, HIVE-23269.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Comparing bigints and varchars or chars may result to wrong result, for
> example:
> CREATE TABLE test_a (appid1 varchar(256), appid2 char(20));
> INSERT INTO test_a VALUES ('2882303761517473127', '2882303761517473127'),
> ('2882303761517473276','2882303761517473276');
> SET hive.strict.checks.type.safety=false;
> SELECT appid1 FROM test_a WHERE appid1 = 2882303761517473127;
> SELECT appid2 FROM test_a WHERE appid2 = 2882303761517473127;
> Bo
[jira] [Updated] (HIVE-23529) CTAS is broken for uniontype when row_deserialize
[ https://issues.apache.org/jira/browse/HIVE-23529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-23529: Attachment: HIVE-23529.patch Status: Patch Available (was: In Progress) > CTAS is broken for uniontype when row_deserialize > - > > Key: HIVE-23529 > URL: https://issues.apache.org/jira/browse/HIVE-23529 > Project: Hive > Issue Type: Bug >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Attachments: HIVE-23529.patch, HIVE-23529.patch > > > CTAS queries fail when there is a uniontype in source table and > hive.vectorized.use.vector.serde.deserialize=false. > ObjectInspectorUtils.copyToStandardObject in ROW_DESERIALIZE path extracts > the value from union type. However, VectorAssignRow expects a StandardUnion > object causing ClassCastException for any CTAS query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23529) CTAS is broken for uniontype when row_deserialize
[ https://issues.apache.org/jira/browse/HIVE-23529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-23529: Status: In Progress (was: Patch Available) > CTAS is broken for uniontype when row_deserialize > - > > Key: HIVE-23529 > URL: https://issues.apache.org/jira/browse/HIVE-23529 > Project: Hive > Issue Type: Bug >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Attachments: HIVE-23529.patch, HIVE-23529.patch > > > CTAS queries fail when there is a uniontype in source table and > hive.vectorized.use.vector.serde.deserialize=false. > ObjectInspectorUtils.copyToStandardObject in ROW_DESERIALIZE path extracts > the value from union type. However, VectorAssignRow expects a StandardUnion > object causing ClassCastException for any CTAS query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23269) Unsafe comparing bigints and strings
[
https://issues.apache.org/jira/browse/HIVE-23269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114231#comment-17114231
]
Hive QA commented on HIVE-23269:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003715/HIVE-23269.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 17284 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[avrotblsjoin]
(batchId=62)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer8]
(batchId=43)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[filter_join_breaktask2]
(batchId=109)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_literals]
(batchId=122)
org.apache.hive.jdbc.TestActivePassiveHA.testActivePassiveHA (batchId=218)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22547/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22547/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22547/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003715 - PreCommit-HIVE-Build
> Unsafe comparing bigints and strings
>
>
> Key: HIVE-23269
> URL: https://issues.apache.org/jira/browse/HIVE-23269
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Major
> Attachments: HIVE-23269.2.patch, HIVE-23269.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Comparing bigints and varchars or chars may result to wrong result, for
> example:
> CREATE TABLE test_a (appid1 varchar(256), appid2 char(20));
> INSERT INTO test_a VALUES ('2882303761517473127', '2882303761517473127'),
> ('2882303761517473276','2882303761517473276');
> SET hive.strict.checks.type.safety=false;
> SELECT appid1 FROM test_a WHERE appid1 = 2882303761517473127;
> SELECT appid2 FROM test_a WHERE appid2 = 2882303761517473127;
> Both queries will output the row:
> ('2882303761517473276','2882303761517473276')
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23537) RecordReader support for row-filtering
[ https://issues.apache.org/jira/browse/HIVE-23537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-23537: - > RecordReader support for row-filtering > -- > > Key: HIVE-23537 > URL: https://issues.apache.org/jira/browse/HIVE-23537 > Project: Hive > Issue Type: Sub-task > Components: llap, Reader >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > > ORC-577 enables row-level filtering for the ORC format while HIVE-23167 is > aiming to extend the existing compiler logic and push filters further down > the pipeline wherever possible. > In this jira we extend the HIVE Record readers to utilize the above filtering > functionality (similar to what we already do for PPD). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23494) Upgrade Apache parent POM to version 23
[ https://issues.apache.org/jira/browse/HIVE-23494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23494: -- Attachment: HIVE-23494.1.patch > Upgrade Apache parent POM to version 23 > --- > > Key: HIVE-23494 > URL: https://issues.apache.org/jira/browse/HIVE-23494 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23494.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23353) Atlas metadata replication scheduling
[
https://issues.apache.org/jira/browse/HIVE-23353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114294#comment-17114294
]
Hive QA commented on HIVE-23353:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003716/HIVE-23353.07.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17285 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.exec.spark.status.TestSparkJobMonitor.testProgress
(batchId=293)
org.apache.hadoop.hive.ql.metadata.TestHive.testMetaStoreApiTiming (batchId=272)
org.apache.hadoop.hive.ql.metadata.TestHiveRemote.testMetaStoreApiTiming
(batchId=273)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22548/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22548/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22548/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003716 - PreCommit-HIVE-Build
> Atlas metadata replication scheduling
> -
>
> Key: HIVE-23353
> URL: https://issues.apache.org/jira/browse/HIVE-23353
> Project: Hive
> Issue Type: Task
>Reporter: Pravin Sinha
>Assignee: Pravin Sinha
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23353.01.patch, HIVE-23353.02.patch,
> HIVE-23353.03.patch, HIVE-23353.04.patch, HIVE-23353.05.patch,
> HIVE-23353.06.patch, HIVE-23353.07.patch, HIVE-23353.07.patch
>
> Time Spent: 4h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23353) Atlas metadata replication scheduling
[
https://issues.apache.org/jira/browse/HIVE-23353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114310#comment-17114310
]
Hive QA commented on HIVE-23353:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
34s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
40s{color} | {color:red} patch/common cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 5m
53s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
19s{color} | {color:red} patch/itests/hive-unit cannot run setBugDatabaseInfo
from findbugs {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 12m
7s{color} | {color:red} root in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 87m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22548/dev-support/hive-personality.sh
|
| git revision | master / e649562 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22548/yetus/patch-findbugs-common.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22548/yetus/patch-findbugs-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22548/yetus/patch-findbugs-itests_hive-unit.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22548/yetus/patch-javadoc-root.txt
|
| modules | C: common ql . itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22548/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Atlas metadata replication scheduling
> -
>
> Key: HIVE-23
[jira] [Work logged] (HIVE-23537) RecordReader support for row-filtering
[ https://issues.apache.org/jira/browse/HIVE-23537?focusedWorklogId=436636&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-436636 ] ASF GitHub Bot logged work on HIVE-23537: - Author: ASF GitHub Bot Created on: 22/May/20 18:32 Start Date: 22/May/20 18:32 Worklog Time Spent: 10m Work Description: pgaref opened a new pull request #1032: URL: https://github.com/apache/hive/pull/1032 Change-Id: I3273949120b50605c257a6e1bc9384ca17f63a22 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 436636) Remaining Estimate: 0h Time Spent: 10m > RecordReader support for row-filtering > -- > > Key: HIVE-23537 > URL: https://issues.apache.org/jira/browse/HIVE-23537 > Project: Hive > Issue Type: Sub-task > Components: llap, Reader >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ORC-577 enables row-level filtering for the ORC format while HIVE-23167 is > aiming to extend the existing compiler logic and push filters further down > the pipeline wherever possible. > In this jira we extend the HIVE Record readers to utilize the above filtering > functionality (similar to what we already do for PPD). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23537) RecordReader support for row-filtering
[ https://issues.apache.org/jira/browse/HIVE-23537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23537: -- Labels: pull-request-available (was: ) > RecordReader support for row-filtering > -- > > Key: HIVE-23537 > URL: https://issues.apache.org/jira/browse/HIVE-23537 > Project: Hive > Issue Type: Sub-task > Components: llap, Reader >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > ORC-577 enables row-level filtering for the ORC format while HIVE-23167 is > aiming to extend the existing compiler logic and push filters further down > the pipeline wherever possible. > In this jira we extend the HIVE Record readers to utilize the above filtering > functionality (similar to what we already do for PPD). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23353) Atlas metadata replication scheduling
[ https://issues.apache.org/jira/browse/HIVE-23353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha updated HIVE-23353: Attachment: HIVE-23353.07.patch > Atlas metadata replication scheduling > - > > Key: HIVE-23353 > URL: https://issues.apache.org/jira/browse/HIVE-23353 > Project: Hive > Issue Type: Task >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23353.01.patch, HIVE-23353.02.patch, > HIVE-23353.03.patch, HIVE-23353.04.patch, HIVE-23353.05.patch, > HIVE-23353.06.patch, HIVE-23353.07.patch, HIVE-23353.07.patch, > HIVE-23353.07.patch > > Time Spent: 4h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23361) Optimising privilege synchroniser
[
https://issues.apache.org/jira/browse/HIVE-23361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114329#comment-17114329
]
Hive QA commented on HIVE-23361:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
31s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
57s{color} | {color:red} ql: The patch generated 1 new + 82 unchanged - 0 fixed
= 83 total (was 82) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 39m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22549/dev-support/hive-personality.sh
|
| git revision | master / e649562 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22549/yetus/diff-checkstyle-ql.txt
|
| modules | C: metastore ql itests itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22549/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Optimising privilege synchroniser
> -
>
> Key: HIVE-23361
> URL: https://issues.apache.org/jira/browse/HIVE-23361
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Simhadri G
>Assignee: Simhadri G
>Priority: Minor
> Attachments: hive-23361.1.patch, hive-23361.2.patch,
> hive-23361.3.patch, hive-23361.4.patch, hive-23361.5.patch,
> hive-23361.6.patch, hive-23361.7.patch, hive-23361.8.patch,
> hive-23361.9.patch, hive-23361.patch, hive-23361.patch
>
>
> Privilege synchronizer pulls the list of databases, tables and columns from
> the Hive Metastore. For each of these objects it fetches the privilege
> information and invokes HMS API to refresh the privilege information in HMS.
> This patch store the privilege information as bit string. This is done to
> reduce the size of the tbl_col_privs tables in metastore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23494) Upgrade Apache parent POM to version 23
[ https://issues.apache.org/jira/browse/HIVE-23494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-23494: Status: Patch Available (was: Open) > Upgrade Apache parent POM to version 23 > --- > > Key: HIVE-23494 > URL: https://issues.apache.org/jira/browse/HIVE-23494 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23494.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23361) Optimising privilege synchroniser
[
https://issues.apache.org/jira/browse/HIVE-23361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114355#comment-17114355
]
Hive QA commented on HIVE-23361:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003717/hive-23361.9.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17283 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22549/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22549/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22549/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003717 - PreCommit-HIVE-Build
> Optimising privilege synchroniser
> -
>
> Key: HIVE-23361
> URL: https://issues.apache.org/jira/browse/HIVE-23361
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Simhadri G
>Assignee: Simhadri G
>Priority: Minor
> Attachments: hive-23361.1.patch, hive-23361.2.patch,
> hive-23361.3.patch, hive-23361.4.patch, hive-23361.5.patch,
> hive-23361.6.patch, hive-23361.7.patch, hive-23361.8.patch,
> hive-23361.9.patch, hive-23361.patch, hive-23361.patch
>
>
> Privilege synchronizer pulls the list of databases, tables and columns from
> the Hive Metastore. For each of these objects it fetches the privilege
> information and invokes HMS API to refresh the privilege information in HMS.
> This patch store the privilege information as bit string. This is done to
> reduce the size of the tbl_col_privs tables in metastore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23530) Use SQL functions instead of compute_stats UDAF to compute column statistics
[
https://issues.apache.org/jira/browse/HIVE-23530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-23530:
---
Attachment: HIVE-23530.01.patch
> Use SQL functions instead of compute_stats UDAF to compute column statistics
>
>
> Key: HIVE-23530
> URL: https://issues.apache.org/jira/browse/HIVE-23530
> Project: Hive
> Issue Type: Improvement
> Components: Statistics
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-23530.01.patch, HIVE-23530.patch
>
>
> Currently we compute column statistics by relying on the {{compute_stats}}
> UDAF. For instance, for a given table {{tbl}}, the query to compute
> statistics for columns is translated internally into:
> {code}
> SELECT compute_stats(c1),
>compute_stats(c2),
>...
> FROM tbl;
> {code}
> {{compute_stats}} produces data for the stats available for each column type,
> e.g., struct<"max":long,"min":long,"countnulls":long,...>.
> This issue is to produce a query that relies purely on SQL functions instead:
> {code}
> SELECT max(c1), min(c1), count(case when c1 is null then 1 else null end),
>...
> FROM tbl;
> {code}
> This will allow us to deprecate the {{compute_stats}} UDAF since it mostly
> duplicates functionality found in those other functions. Additionally, many
> of those functions already provide a vectorized implementation so the
> approach can potentially improve the performance of column stats collection.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23527) CUME_DIST should be threated as a function returning double
[
https://issues.apache.org/jira/browse/HIVE-23527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114369#comment-17114369
]
Hive QA commented on HIVE-23527:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
31s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} ql: The patch generated 0 new + 2 unchanged - 2
fixed = 2 total (was 4) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22550/dev-support/hive-personality.sh
|
| git revision | master / e649562 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22550/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CUME_DIST should be threated as a function returning double
> ---
>
> Key: HIVE-23527
> URL: https://issues.apache.org/jira/browse/HIVE-23527
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23527.01.patch, HIVE-23527.01.patch
>
>
> right now it is handled as integer in calcite - which might cause some
> trouble (at least for me)
> Interestingly enough it seems like only percent_rank has double type
> https://github.com/apache/hive/blob/b047cfae872244e769f5d1c3c11811d1c49c19ad/ql/src/java/org/apache/hadoop/hive/ql/parse/type/HiveFunctionHelper.java#L392
> since this change goes back to HIVE-9133 and seems to me that removing this
> tweak is just works - I'll submit a patch to remove this conditional
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23527) CUME_DIST should be threated as a function returning double
[
https://issues.apache.org/jira/browse/HIVE-23527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114386#comment-17114386
]
Hive QA commented on HIVE-23527:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003718/HIVE-23527.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 32 failed/errored test(s), 17281 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_windowing]
(batchId=71)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_windowing_2]
(batchId=58)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join46]
(batchId=31)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in]
(batchId=96)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_notin]
(batchId=103)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_unqualcolumnrefs]
(batchId=48)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[topnkey_windowing]
(batchId=57)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[topnkey_windowing_order]
(batchId=48)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_rank]
(batchId=90)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing_streaming]
(batchId=84)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[windowing_filter]
(batchId=79)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[windowing_streaming]
(batchId=100)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query47]
(batchId=231)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query49]
(batchId=231)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query57]
(batchId=231)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query44]
(batchId=231)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query47]
(batchId=231)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query49]
(batchId=231)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query57]
(batchId=231)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query67]
(batchId=231)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query70]
(batchId=231)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query47]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query49]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query57]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[mv_query44]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[mv_query67]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query44]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query47]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query49]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query57]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query67]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query70]
(batchId=230)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22550/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22550/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22550/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 32 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003718 - PreCommit-HIVE-Build
> CUME_DIST should be threated as a function returning double
> ---
>
> Key: HIVE-23527
> URL: https://issues.apache.org/jira/browse/HIVE-23527
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23527.01.patch, HIVE-23527.01.patch
>
>
> right now it is handled as integer in calcite - which might cause some
> trouble (at least for me)
> Interestingly enough it seems like only percent_rank has double type
> https
[jira] [Assigned] (HIVE-23538) Cannot run setBugDatabaseInfo from findbugs during preCommit
[ https://issues.apache.org/jira/browse/HIVE-23538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha reassigned HIVE-23538: --- > Cannot run setBugDatabaseInfo from findbugs during preCommit > > > Key: HIVE-23538 > URL: https://issues.apache.org/jira/browse/HIVE-23538 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: David Mollitor >Priority: Major > > During the preCommit of the patch HIVE-23353 this is seen. > -1findbugs1m 5s patch/common cannot run setBugDatabaseInfo from > findbugs > -1findbugs10m 27s patch/ql cannot run setBugDatabaseInfo > from findbugs > -1findbugs1m 51s patch/itests/hive-unit cannot run > setBugDatabaseInfo from findbugs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23525) TestAcidTxnCleanerService is unstable
[
https://issues.apache.org/jira/browse/HIVE-23525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114404#comment-17114404
]
Hive QA commented on HIVE-23525:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
19s{color} | {color:red} metastore-server in master failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
27s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 456 unchanged - 0 fixed = 457 total (was 456) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
16s{color} | {color:red} metastore-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22551/dev-support/hive-personality.sh
|
| git revision | master / e649562 |
| Default Java | 1.8.0_111 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22551/yetus/branch-findbugs-standalone-metastore_metastore-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22551/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22551/yetus/patch-findbugs-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22551/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestAcidTxnCleanerService is unstable
> -
>
> Key: HIVE-23525
> URL: https://issues.apache.org/jira/browse/HIVE-23525
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Peter Varga
>Priority: Major
> Attachments: HIVE-23525.1.patch
>
>
> from time to time this exception happens
> http://34.66.156.144:8080/job/hive-c/7/console
> {code}
> 15:03:41 [INFO]
> 15:03:41 [INFO] ---
> 15:03:41 [INFO] T E S T S
> 15:03:41 [INFO] ---
> 15:03:42 [INFO] Running
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 25.582 s <<< FAILURE! - in
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR]
> cleansAllCommittedTxns(org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService)
> Time elapsed: 9.952 s <<< FAILURE!
> 15:04:10 java.lang.AssertionError: expe
[jira] [Assigned] (HIVE-23539) Optimize data copy during repl load operation for HDFS based staging location
[ https://issues.apache.org/jira/browse/HIVE-23539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha reassigned HIVE-23539: --- > Optimize data copy during repl load operation for HDFS based staging location > - > > Key: HIVE-23539 > URL: https://issues.apache.org/jira/browse/HIVE-23539 > Project: Hive > Issue Type: Improvement >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23538) Cannot run setBugDatabaseInfo from findbugs during preCommit
[ https://issues.apache.org/jira/browse/HIVE-23538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-23538: - Assignee: David Mollitor (was: David Mollitor) > Cannot run setBugDatabaseInfo from findbugs during preCommit > > > Key: HIVE-23538 > URL: https://issues.apache.org/jira/browse/HIVE-23538 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: David Mollitor >Priority: Major > > During the preCommit of the patch HIVE-23353 this is seen. > -1findbugs1m 5s patch/common cannot run setBugDatabaseInfo from > findbugs > -1findbugs10m 27s patch/ql cannot run setBugDatabaseInfo > from findbugs > -1findbugs1m 51s patch/itests/hive-unit cannot run > setBugDatabaseInfo from findbugs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23538) Cannot run setBugDatabaseInfo from findbugs during preCommit
[ https://issues.apache.org/jira/browse/HIVE-23538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114414#comment-17114414 ] David Mollitor commented on HIVE-23538: --- Hmmm. I need to dig into this more. I'm not sure if it's directly related to my work in [HIVE-22066] since this was an issue already back in 2019: [HIVE-21474] > Cannot run setBugDatabaseInfo from findbugs during preCommit > > > Key: HIVE-23538 > URL: https://issues.apache.org/jira/browse/HIVE-23538 > Project: Hive > Issue Type: Bug >Reporter: Pravin Sinha >Assignee: David Mollitor >Priority: Major > > During the preCommit of the patch HIVE-23353 this is seen. > -1findbugs1m 5s patch/common cannot run setBugDatabaseInfo from > findbugs > -1findbugs10m 27s patch/ql cannot run setBugDatabaseInfo > from findbugs > -1findbugs1m 51s patch/itests/hive-unit cannot run > setBugDatabaseInfo from findbugs -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23525) TestAcidTxnCleanerService is unstable
[
https://issues.apache.org/jira/browse/HIVE-23525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114418#comment-17114418
]
Hive QA commented on HIVE-23525:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003719/HIVE-23525.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 16394 tests
executed
*Failed tests:*
{noformat}
TestAcidTxnCleanerService - did not produce a TEST-*.xml file (likely timed
out) (batchId=155)
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=155)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestListPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestLockRequestBuilder - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestOpenTxn - did not produce a TEST-*.xml file (likely timed out) (batchId=155)
TestPartitionManagement - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestRawStoreProxy - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestRuntimeStats - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestTablesCreateDropAlterTruncate - did not produce a TEST-*.xml file (likely
timed out) (batchId=155)
TestTxnHandlerNegative - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
TestTxnUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22551/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22551/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22551/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003719 - PreCommit-HIVE-Build
> TestAcidTxnCleanerService is unstable
> -
>
> Key: HIVE-23525
> URL: https://issues.apache.org/jira/browse/HIVE-23525
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Peter Varga
>Priority: Major
> Attachments: HIVE-23525.1.patch
>
>
> from time to time this exception happens
> http://34.66.156.144:8080/job/hive-c/7/console
> {code}
> 15:03:41 [INFO]
> 15:03:41 [INFO] ---
> 15:03:41 [INFO] T E S T S
> 15:03:41 [INFO] ---
> 15:03:42 [INFO] Running
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 25.582 s <<< FAILURE! - in
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService
> 15:04:10 [ERROR]
> cleansAllCommittedTxns(org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService)
> Time elapsed: 9.952 s <<< FAILURE!
> 15:04:10 java.lang.AssertionError: expected:<6> but was:<7>
> 15:04:10 at
> org.apache.hadoop.hive.metastore.txn.TestAcidTxnCleanerService.cleansAllCommittedTxns(TestAcidTxnCleanerService.java:107)
> 15:04:10
> 15:04:10 [INFO]
> 15:04:10 [INFO] Results:
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Failures:
> 15:04:10 [ERROR] TestAcidTxnCleanerService.cleansAllCommittedTxns:107
> expected:<6> but was:<7>
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0
> 15:04:10 [INFO]
> 15:04:10 [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on
> project hive-standalone-metastore-server: There are test failures.
> 15:04:10 [ERROR]
> 15:04:10 [ERROR] Please refer to
> /home/jenkins/agent/worksp
[jira] [Commented] (HIVE-22077) Inserting overwrite partitions clause does not clean directories while partitions' info is not stored in metadata
[
https://issues.apache.org/jira/browse/HIVE-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114419#comment-17114419
]
Hive QA commented on HIVE-22077:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12978761/HIVE-22077.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22552/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22552/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22552/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-05-22 21:47:20.469
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-22552/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-05-22 21:47:20.472
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at e649562 HIVE-22066: Upgrade Apache parent POM to version 21
(David Mollitor, reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at e649562 HIVE-22066: Upgrade Apache parent POM to version 21
(David Mollitor, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-05-22 21:47:21.390
+ rm -rf ../yetus_PreCommit-HIVE-Build-22552
+ mkdir ../yetus_PreCommit-HIVE-Build-22552
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-22552
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-22552/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: a/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java: does not
exist in index
Trying to apply the patch with -p1
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:2181
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java'
with conflicts.
Going to apply patch with: git apply -p1
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:2181
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java'
with conflicts.
U ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-22552
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12978761 - PreCommit-HIVE-Build
> Inserting overwrite partitions clause does not clean directories while
> partitions' info is not stored in metadata
> -
>
> Key: HIVE-22077
> URL: https://issues.apache.org/jira/browse/HIVE-22077
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1, 4.0.0, 2.3.4
>Reporter: Hui An
>Assignee: Hui An
>Priority: Major
> Attachments: HIVE-22077.patch.1
>
>
> Inserting overwrite static partitions may not clean related HDFS location if
> partitions' info is not stored in metadata.
> Steps to reproduce this issue :
>
> 1. Create a managed table :
>
> {code:sql}
> CREATE TABLE `test`(
>`id` string)
> PARTITIONED BY (
[jira] [Updated] (HIVE-23069) Memory efficient iterator should be used during replication.
[ https://issues.apache.org/jira/browse/HIVE-23069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha updated HIVE-23069: Summary: Memory efficient iterator should be used during replication. (was: Memory efficient iterator should be used durind replication.) > Memory efficient iterator should be used during replication. > > > Key: HIVE-23069 > URL: https://issues.apache.org/jira/browse/HIVE-23069 > Project: Hive > Issue Type: Improvement >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > > Currently the iterator used while copying table data is memory based. In case > of a database with very large number of table/partitions, such iterator may > cause HS2 process to go OOM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23069) Memory efficient iterator should be used durind replication.
[ https://issues.apache.org/jira/browse/HIVE-23069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pravin Sinha updated HIVE-23069: Issue Type: Improvement (was: Bug) > Memory efficient iterator should be used durind replication. > > > Key: HIVE-23069 > URL: https://issues.apache.org/jira/browse/HIVE-23069 > Project: Hive > Issue Type: Improvement >Reporter: Pravin Sinha >Assignee: Pravin Sinha >Priority: Major > > Currently the iterator used while copying table data is memory based. In case > of a database with very large number of table/partitions, such iterator may > cause HS2 process to go OOM. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23513) Fix Json output for SHOW TABLES and SHOW MATERIALIZED VIEWS
[
https://issues.apache.org/jira/browse/HIVE-23513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114432#comment-17114432
]
Hive QA commented on HIVE-23513:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
27s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22553/dev-support/hive-personality.sh
|
| git revision | master / e649562 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22553/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix Json output for SHOW TABLES and SHOW MATERIALIZED VIEWS
> ---
>
> Key: HIVE-23513
> URL: https://issues.apache.org/jira/browse/HIVE-23513
> Project: Hive
> Issue Type: Bug
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23513.01.patch, HIVE-23513.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23269) Unsafe comparing bigints and strings
[
https://issues.apache.org/jira/browse/HIVE-23269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated HIVE-23269:
---
Attachment: HIVE-23269.3.patch
> Unsafe comparing bigints and strings
>
>
> Key: HIVE-23269
> URL: https://issues.apache.org/jira/browse/HIVE-23269
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Zhihua Deng
>Priority: Major
> Attachments: HIVE-23269.2.patch, HIVE-23269.3.patch, HIVE-23269.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Comparing bigints and varchars or chars may result to wrong result, for
> example:
> CREATE TABLE test_a (appid1 varchar(256), appid2 char(20));
> INSERT INTO test_a VALUES ('2882303761517473127', '2882303761517473127'),
> ('2882303761517473276','2882303761517473276');
> SET hive.strict.checks.type.safety=false;
> SELECT appid1 FROM test_a WHERE appid1 = 2882303761517473127;
> SELECT appid2 FROM test_a WHERE appid2 = 2882303761517473127;
> Both queries will output the row:
> ('2882303761517473276','2882303761517473276')
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23513) Fix Json output for SHOW TABLES and SHOW MATERIALIZED VIEWS
[
https://issues.apache.org/jira/browse/HIVE-23513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114441#comment-17114441
]
Hive QA commented on HIVE-23513:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003723/HIVE-23513.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17282 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22553/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22553/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22553/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003723 - PreCommit-HIVE-Build
> Fix Json output for SHOW TABLES and SHOW MATERIALIZED VIEWS
> ---
>
> Key: HIVE-23513
> URL: https://issues.apache.org/jira/browse/HIVE-23513
> Project: Hive
> Issue Type: Bug
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23513.01.patch, HIVE-23513.02.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23513) Fix Json output for SHOW TABLES and SHOW MATERIALIZED VIEWS
[ https://issues.apache.org/jira/browse/HIVE-23513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23513: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Fix Json output for SHOW TABLES and SHOW MATERIALIZED VIEWS > --- > > Key: HIVE-23513 > URL: https://issues.apache.org/jira/browse/HIVE-23513 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23513.01.patch, HIVE-23513.02.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23365) Put RS deduplication optimization under cost based decision
[
https://issues.apache.org/jira/browse/HIVE-23365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114448#comment-17114448
]
Hive QA commented on HIVE-23365:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
22s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
0s{color} | {color:red} ql: The patch generated 1 new + 63 unchanged - 0 fixed
= 64 total (was 63) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22555/dev-support/hive-personality.sh
|
| git revision | master / e649562 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22555/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22555/yetus/whitespace-eol.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22555/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Put RS deduplication optimization under cost based decision
> ---
>
> Key: HIVE-23365
> URL: https://issues.apache.org/jira/browse/HIVE-23365
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Stamatis Zampetakis
>Priority: Major
> Attachments: HIVE-23365.01.patch
>
>
> Currently, RS deduplication is always executed whenever it is semantically
> correct. However, it could be beneficial to leave both RS operators in the
> plan, e.g., if the NDV of the second RS is very low. Thus, we would like this
> decision to be cost-based. We could use a simple heuristic that would work
> fine for most of the cases without introducing regressions for existing
> cases, e.g., if NDV for partition column is less than estimated parallelism
> in the second RS, do not execute deduplication.
--
This message was sent by Atlassian Jira
(v8.3.4#
[jira] [Commented] (HIVE-23365) Put RS deduplication optimization under cost based decision
[
https://issues.apache.org/jira/browse/HIVE-23365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114457#comment-17114457
]
Hive QA commented on HIVE-23365:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003729/HIVE-23365.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 160 failed/errored test(s), 17282 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[quotedid_basic_standard]
(batchId=4)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_bulk]
(batchId=17)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_handler_bulk]
(batchId=16)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[reduce_deduplicate_stat_based]
(batchId=23)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_delete_nodeletepriv]
(batchId=8)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_update_noupdatepriv]
(batchId=8)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[merge_constraint_notnull]
(batchId=9)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_scalar_corr_multi_rows]
(batchId=9)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query24]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query24]
(batchId=230)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCrudMajorCompactionSplitGrouper
(batchId=179)
org.apache.hadoop.hive.ql.TestTxnCommands.testDelete (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testDeleteIn (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeDeleteUpdate (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeType2SCD01 (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeType2SCD02 (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeUpdateDelete (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeUpdateDeleteNoCardCheck
(batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testMoreBucketsThanReducers
(batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testMultipleDelete (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testQuotedIdentifier (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testQuotedIdentifier2 (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testTimeOutReaper (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testUpdateDeleteOfInserts
(batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands.testUpdateOfInserts (batchId=301)
org.apache.hadoop.hive.ql.TestTxnCommands2.testACIDwithSchemaEvolutionAndCompaction
(batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testBucketCodec (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testCompactWithDelete (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDeleteIn (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDynamicPartitionsMerge
(batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDynamicPartitionsMerge2
(batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testEmptyInTblproperties
(batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge2 (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge3 (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMergeWithPredicate (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMultiInsert (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion02
(batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion2
(batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion3
(batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOrcNoPPD (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOrcPPD (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOriginalFileReaderWhenNonAcidConvertedToAcid
(batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.testUpdateMixedCase (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.updateDeletePartitioned (batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2.writeBetweenWorkerAndCleaner
(batchId=274)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testACIDwithSchemaEvolutionAndCompaction
(batchId=298)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testBucketCodec
(batchId=298)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testCompactWithDelete
(batchId=298)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testDeleteIn
(batchId=298)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testDynamicPartitionsMerge
(batchId=298)
o
[jira] [Assigned] (HIVE-23540) Fix Findbugs Warnings in EncodedColumnBatch
[ https://issues.apache.org/jira/browse/HIVE-23540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-23540: - > Fix Findbugs Warnings in EncodedColumnBatch > --- > > Key: HIVE-23540 > URL: https://issues.apache.org/jira/browse/HIVE-23540 > Project: Hive > Issue Type: Improvement > Components: storage-api >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-23540.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23540) Fix Findbugs Warnings in EncodedColumnBatch
[ https://issues.apache.org/jira/browse/HIVE-23540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23540: -- Attachment: HIVE-23540.1.patch > Fix Findbugs Warnings in EncodedColumnBatch > --- > > Key: HIVE-23540 > URL: https://issues.apache.org/jira/browse/HIVE-23540 > Project: Hive > Issue Type: Improvement > Components: storage-api >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-23540.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23540) Fix Findbugs Warnings in EncodedColumnBatch
[ https://issues.apache.org/jira/browse/HIVE-23540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23540: -- Status: Patch Available (was: Open) > Fix Findbugs Warnings in EncodedColumnBatch > --- > > Key: HIVE-23540 > URL: https://issues.apache.org/jira/browse/HIVE-23540 > Project: Hive > Issue Type: Improvement > Components: storage-api >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-23540.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23466) ZK registry base should remove only specific instance instead of host
[
https://issues.apache.org/jira/browse/HIVE-23466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114466#comment-17114466
]
Hive QA commented on HIVE-23466:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13003331/HIVE-23466.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22556/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22556/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22556/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-05-23 01:15:57.414
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-22556/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-05-23 01:15:57.419
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
e649562..c641497 master -> origin/master
+ git reset --hard HEAD
HEAD is now at e649562 HIVE-22066: Upgrade Apache parent POM to version 21
(David Mollitor, reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at c641497 HIVE-23513 Fix Json output for SHOW TABLES and SHOW
MATERIALIZED VIEWS (Miklos Gergely, reviewed by Zoltan Haindrich)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-05-23 01:15:58.860
+ rm -rf ../yetus_PreCommit-HIVE-Build-22556
+ mkdir ../yetus_PreCommit-HIVE-Build-22556
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-22556
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-22556/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error:
a/llap-client/src/java/org/apache/hadoop/hive/registry/impl/ZkRegistryBase.java:
does not exist in index
error:
a/llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/ContainerFactory.java:
does not exist in index
error:
a/llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskSchedulerService.java:
does not exist in index
Trying to apply the patch with -p1
error: patch failed:
llap-client/src/java/org/apache/hadoop/hive/registry/impl/ZkRegistryBase.java:448
error: repository lacks the necessary blob to fall back on 3-way merge.
error:
llap-client/src/java/org/apache/hadoop/hive/registry/impl/ZkRegistryBase.java:
patch does not apply
Trying to apply the patch with -p2
error: src/java/org/apache/hadoop/hive/registry/impl/ZkRegistryBase.java: does
not exist in index
error: src/java/org/apache/hadoop/hive/llap/tezplugins/ContainerFactory.java:
does not exist in index
error:
src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskSchedulerService.java:
does not exist in index
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-22556
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13003331 - PreCommit-HIVE-Build
> ZK registry base should remove only specific instance instead of host
> -
>
> Key: HIVE-23466
> URL: https://issues.apache.org/jira/browse/HIVE-23466
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Attila Magyar
>Priority: Major
>
[jira] [Commented] (HIVE-23531) Major CRUD QB compaction failing with ClassCastException when vectorization off
[
https://issues.apache.org/jira/browse/HIVE-23531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17114501#comment-17114501
]
Hive QA commented on HIVE-23531:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
23s{color} | {color:blue} ql in master has 1524 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22557/dev-support/hive-personality.sh
|
| git revision | master / c641497 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22557/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Major CRUD QB compaction failing with ClassCastException when vectorization
> off
> ---
>
> Key: HIVE-23531
> URL: https://issues.apache.org/jira/browse/HIVE-23531
> Project: Hive
> Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23531.01.patch, HIVE-23531.01.patch
>
>
> Exception:
> {code:java}
> 2020-05-22T01:33:09,944 ERROR [TezChild] tez.MapRecordSource:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> at
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:403)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> at
> org.apache.hadoop.hive.ql.e
[jira] [Updated] (HIVE-23526) Out of sequence seen in Beeline may swallow the real problem
[ https://issues.apache.org/jira/browse/HIVE-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated HIVE-23526: --- Attachment: HIVE-23526.2.patch > Out of sequence seen in Beeline may swallow the real problem > - > > Key: HIVE-23526 > URL: https://issues.apache.org/jira/browse/HIVE-23526 > Project: Hive > Issue Type: Improvement > Components: Beeline > Environment: Hive 1.2.2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23526.2.patch, HIVE-23526.patch, outofsequence.log > > Time Spent: 10m > Remaining Estimate: 0h > > Sometimes we can see 'out of sequence response' message in beeline, for > example: > Error: org.apache.thrift.TApplicationException: CloseOperation failed: out of > sequence response (state=08S01,code=0) > java.sql.SQLException: org.apache.thrift.TApplicationException: > CloseOperation failed: out of sequence response > at > org.apache.hive.jdbc.HiveStatement.closeClientOperation(HiveStatement.java:198) > at org.apache.hive.jdbc.HiveStatement.close(HiveStatement.java:217) > at org.apache.hive.beeline.Commands.execute(Commands.java:891) > at org.apache.hive.beeline.Commands.sql(Commands.java:713) > at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:976) > at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:816) > at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:774) > at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:487) > at org.apache.hive.beeline.BeeLine.main(BeeLine.java:470) > and there is no other usage messages to figured it out, even with verbose > flag, this makes problem puzzled as beeline does not have concurrency problem > on underlying thrift transport. -- This message was sent by Atlassian Jira (v8.3.4#803005)
