[jira] [Updated] (HIVE-21050) Upgrade Parquet to 1.11.0 and use LogicalTypes
[ https://issues.apache.org/jira/browse/HIVE-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21050: - Attachment: HIVE-21050.6.patch.txt Status: Patch Available (was: Open) > Upgrade Parquet to 1.11.0 and use LogicalTypes > -- > > Key: HIVE-21050 > URL: https://issues.apache.org/jira/browse/HIVE-21050 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: Parquet, parquet > Attachments: HIVE-21050.1.patch, HIVE-21050.1.patch, > HIVE-21050.1.patch, HIVE-21050.2.patch, HIVE-21050.3.patch, > HIVE-21050.4.patch, HIVE-21050.4.patch, HIVE-21050.4.patch, > HIVE-21050.5.patch, HIVE-21050.5.patch, HIVE-21050.5.patch, > HIVE-21050.6.patch, HIVE-21050.6.patch, HIVE-21050.6.patch.txt > > > [WIP until Parquet community releases version 1.11.0] > The new Parquet version (1.11.0) uses > [LogicalTypes|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md] > instead of OriginalTypes. > These are backwards-compatible with OriginalTypes. > Thanks to [~kuczoram] for her work on this patch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21050) Upgrade Parquet to 1.11.0 and use LogicalTypes
[ https://issues.apache.org/jira/browse/HIVE-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21050: - Status: Open (was: Patch Available) > Upgrade Parquet to 1.11.0 and use LogicalTypes > -- > > Key: HIVE-21050 > URL: https://issues.apache.org/jira/browse/HIVE-21050 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: Parquet, parquet > Attachments: HIVE-21050.1.patch, HIVE-21050.1.patch, > HIVE-21050.1.patch, HIVE-21050.2.patch, HIVE-21050.3.patch, > HIVE-21050.4.patch, HIVE-21050.4.patch, HIVE-21050.4.patch, > HIVE-21050.5.patch, HIVE-21050.5.patch, HIVE-21050.5.patch, > HIVE-21050.6.patch, HIVE-21050.6.patch > > > [WIP until Parquet community releases version 1.11.0] > The new Parquet version (1.11.0) uses > [LogicalTypes|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md] > instead of OriginalTypes. > These are backwards-compatible with OriginalTypes. > Thanks to [~kuczoram] for her work on this patch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21050) Upgrade Parquet to 1.11.0 and use LogicalTypes
[ https://issues.apache.org/jira/browse/HIVE-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21050: - Description: [WIP until Parquet community releases version 1.11.0] The new Parquet version (1.11.0) uses [LogicalTypes|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md] instead of OriginalTypes. These are backwards-compatible with OriginalTypes. Thanks to [~kuczoram] for her work on this patch. was: [WIP until Parquet community releases version 1.11.0] The new Parquet version (1.12.0) uses [LogicalTypes|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md] instead of OriginalTypes. These are backwards-compatible with OriginalTypes. Thanks to [~kuczoram] for her work on this patch. > Upgrade Parquet to 1.11.0 and use LogicalTypes > -- > > Key: HIVE-21050 > URL: https://issues.apache.org/jira/browse/HIVE-21050 > Project: Hive > Issue Type: Improvement > Components: File Formats >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Labels: Parquet, parquet > Attachments: HIVE-21050.1.patch, HIVE-21050.1.patch, > HIVE-21050.1.patch, HIVE-21050.2.patch, HIVE-21050.3.patch, > HIVE-21050.4.patch, HIVE-21050.4.patch, HIVE-21050.4.patch, > HIVE-21050.5.patch, HIVE-21050.5.patch, HIVE-21050.5.patch, > HIVE-21050.6.patch, HIVE-21050.6.patch > > > [WIP until Parquet community releases version 1.11.0] > The new Parquet version (1.11.0) uses > [LogicalTypes|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md] > instead of OriginalTypes. > These are backwards-compatible with OriginalTypes. > Thanks to [~kuczoram] for her work on this patch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22551) BytesColumnVector initBuffer should clean vector and length consistently
[
https://issues.apache.org/jira/browse/HIVE-22551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984781#comment-16984781
]
Hive QA commented on HIVE-22551:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} storage-api in master has 58 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19668/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: storage-api U: storage-api |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19668/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> BytesColumnVector initBuffer should clean vector and length consistently
> -
>
> Key: HIVE-22551
> URL: https://issues.apache.org/jira/browse/HIVE-22551
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22551.01.patch, HIVE-22551.01.patch,
> HIVE-22551.01.patch
>
>
> VectorExtractRow relies on the fact that vector[i] and length[i] are
> consistent within the BytesColumnVector, otherwise it throws exception:
> https://github.com/apache/hive/blob/edc53cc0d95e983c371a224943dd866210f0c65c/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorExtractRow.java#L275
> There is a scenario when only vector[i] has been cleaned while reusing the
> column vector, and then this kind of exception can be thrown:
> the reproduction was made with
> [LlapDump|https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapDump.java]
> with String columns (longer than 16 chars)
> {code}
> 19/10/17 15:55:49 ERROR llap.LlapArrowRowRecordReader: Failed to fetch Arrow
> batch
> java.lang.RuntimeException: STRING entry: batchIndex 45
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.BytesReadError(VectorExtractRow.java:488)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRowColumn(VectorExtractRow.java:294)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRowColumn(VectorExtractRow.java:193)
> at
>
[jira] [Commented] (HIVE-22327) Repl: Ignore read-only transactions in notification log
[
https://issues.apache.org/jira/browse/HIVE-22327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984770#comment-16984770
]
Hive QA commented on HIVE-22327:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987068/HIVE-22327.13.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 17822 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_avro]
(batchId=303)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesWithJsonMessage.testOpenTxnEvent
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testOpenTxnEvent
(batchId=275)
org.apache.hive.hcatalog.api.TestHCatClient.testBasicDDLCommands (batchId=211)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19667/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19667/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19667/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987068 - PreCommit-HIVE-Build
> Repl: Ignore read-only transactions in notification log
> ---
>
> Key: HIVE-22327
> URL: https://issues.apache.org/jira/browse/HIVE-22327
> Project: Hive
> Issue Type: Improvement
> Components: repl
>Reporter: Gopal Vijayaraghavan
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22327.1.patch, HIVE-22327.10.patch,
> HIVE-22327.11.patch, HIVE-22327.12.patch, HIVE-22327.13.patch,
> HIVE-22327.2.patch, HIVE-22327.3.patch, HIVE-22327.4.patch,
> HIVE-22327.5.patch, HIVE-22327.6.patch, HIVE-22327.7.patch,
> HIVE-22327.8.patch, HIVE-22327.9.patch
>
>
> Read txns need not be replicated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22544) Disable null sort order at user level
[ https://issues.apache.org/jira/browse/HIVE-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22544: -- Status: Patch Available (was: Open) > Disable null sort order at user level > - > > Key: HIVE-22544 > URL: https://issues.apache.org/jira/browse/HIVE-22544 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22544.1.patch, HIVE-22544.1.patch, > HIVE-22544.2.patch, HIVE-22544.3.patch, HIVE-22544.4.patch > > > "sort order" and "null sort order" in ReduceSinkDesc and TopNKeyDesc should > not be exposed at user level -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22544) Disable null sort order at user level
[ https://issues.apache.org/jira/browse/HIVE-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22544: -- Status: Open (was: Patch Available) > Disable null sort order at user level > - > > Key: HIVE-22544 > URL: https://issues.apache.org/jira/browse/HIVE-22544 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22544.1.patch, HIVE-22544.1.patch, > HIVE-22544.2.patch, HIVE-22544.3.patch, HIVE-22544.4.patch > > > "sort order" and "null sort order" in ReduceSinkDesc and TopNKeyDesc should > not be exposed at user level -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22544) Disable null sort order at user level
[ https://issues.apache.org/jira/browse/HIVE-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22544: -- Attachment: HIVE-22544.4.patch > Disable null sort order at user level > - > > Key: HIVE-22544 > URL: https://issues.apache.org/jira/browse/HIVE-22544 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22544.1.patch, HIVE-22544.1.patch, > HIVE-22544.2.patch, HIVE-22544.3.patch, HIVE-22544.4.patch > > > "sort order" and "null sort order" in ReduceSinkDesc and TopNKeyDesc should > not be exposed at user level -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22555) Upgrade ORC version to 1.5.8
[ https://issues.apache.org/jira/browse/HIVE-22555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22555: Status: Open (was: Patch Available) > Upgrade ORC version to 1.5.8 > > > Key: HIVE-22555 > URL: https://issues.apache.org/jira/browse/HIVE-22555 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22555.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive currently depends on ORC 1.5.6. We need 1.5.8 upgrade for > https://issues.apache.org/jira/browse/HIVE-22499 > ORC-1.5.7 includes https://issues.apache.org/jira/browse/ORC-361 . It causes > some tests overriding MemoryManager to fail. These need to be addressed while > upgrading. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22555) Upgrade ORC version to 1.5.8
[ https://issues.apache.org/jira/browse/HIVE-22555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22555: Status: Patch Available (was: Open) > Upgrade ORC version to 1.5.8 > > > Key: HIVE-22555 > URL: https://issues.apache.org/jira/browse/HIVE-22555 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22555.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive currently depends on ORC 1.5.6. We need 1.5.8 upgrade for > https://issues.apache.org/jira/browse/HIVE-22499 > ORC-1.5.7 includes https://issues.apache.org/jira/browse/ORC-361 . It causes > some tests overriding MemoryManager to fail. These need to be addressed while > upgrading. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22537) getAcidState() not saving directory snapshot causes multiple calls to S3 api
[ https://issues.apache.org/jira/browse/HIVE-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22537: Status: Open (was: Patch Available) > getAcidState() not saving directory snapshot causes multiple calls to S3 api > > > Key: HIVE-22537 > URL: https://issues.apache.org/jira/browse/HIVE-22537 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22537.2.patch, HIVE-22537.3.patch, > HIVE-22537.4.patch, HIVE-22537.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Fix for HIVE-21225 is not enabled in query coordinator codepath. The last > argument (generateDirSnapshots) for getAcidState() is set to false when > invoked by callInternal(). Also, snapshot is not used for file exists calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22537) getAcidState() not saving directory snapshot causes multiple calls to S3 api
[ https://issues.apache.org/jira/browse/HIVE-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22537: Status: Patch Available (was: Open) > getAcidState() not saving directory snapshot causes multiple calls to S3 api > > > Key: HIVE-22537 > URL: https://issues.apache.org/jira/browse/HIVE-22537 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22537.2.patch, HIVE-22537.3.patch, > HIVE-22537.4.patch, HIVE-22537.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Fix for HIVE-21225 is not enabled in query coordinator codepath. The last > argument (generateDirSnapshots) for getAcidState() is set to false when > invoked by callInternal(). Also, snapshot is not used for file exists calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22327) Repl: Ignore read-only transactions in notification log
[
https://issues.apache.org/jira/browse/HIVE-22327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984755#comment-16984755
]
Hive QA commented on HIVE-22327:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
58s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
23s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
27s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
49s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
27s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 4 new + 563 unchanged - 3 fixed = 567 total (was 566) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 1 new + 33 unchanged - 0 fixed
= 34 total (was 33) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch server-extensions passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} itests/hcatalog-unit: The patch generated 0 new + 17
unchanged - 1 fixed = 17 total (was 18) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
26s{color} | {color:green} standalone-metastore/metastore-server generated 0
new + 178 unchanged - 1 fixed = 178 total (was 179) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
27s{color} | {color:green} ql in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} server-extensions in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
52s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Open (was: Patch Available) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.2.patch, HIVE-20150.4.patch, > HIVE-20150.5.patch, HIVE-20150.6.patch, HIVE-20150.7.patch, > HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Attachment: HIVE-20150.17.patch > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.2.patch, HIVE-20150.4.patch, > HIVE-20150.5.patch, HIVE-20150.6.patch, HIVE-20150.7.patch, > HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Patch Available (was: Open) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.17.patch, HIVE-20150.2.patch, HIVE-20150.4.patch, > HIVE-20150.5.patch, HIVE-20150.6.patch, HIVE-20150.7.patch, > HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22554) ACID: Wait timeout for blocking compaction should be configurable
[
https://issues.apache.org/jira/browse/HIVE-22554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984745#comment-16984745
]
Hive QA commented on HIVE-22554:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987070/HIVE-22554.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 17817 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19666/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19666/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19666/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987070 - PreCommit-HIVE-Build
> ACID: Wait timeout for blocking compaction should be configurable
> -
>
> Key: HIVE-22554
> URL: https://issues.apache.org/jira/browse/HIVE-22554
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22554.01.patch
>
>
> The wait timeout for blocking compaction is hardcoded to 5 minutes.
> {code:java}
> public class AlterTableCompactOperation extends
> DDLOperation {
> private static final int FIVE_MINUTES_IN_MILLIES = 5*60*1000;
> ...
> }{code}
> This should be configurable via a Hive Configuration parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22554) ACID: Wait timeout for blocking compaction should be configurable
[
https://issues.apache.org/jira/browse/HIVE-22554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984736#comment-16984736
]
Hive QA commented on HIVE-22554:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
24s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19666/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19666/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Wait timeout for blocking compaction should be configurable
> -
>
> Key: HIVE-22554
> URL: https://issues.apache.org/jira/browse/HIVE-22554
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22554.01.patch
>
>
> The wait timeout for blocking compaction is hardcoded to 5 minutes.
> {code:java}
> public class AlterTableCompactOperation extends
> DDLOperation {
> private static final int FIVE_MINUTES_IN_MILLIES = 5*60*1000;
> ...
> }{code}
> This should be configurable via a Hive Configuration parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22546) Postgres schema not using quoted identifiers for certain tables
[
https://issues.apache.org/jira/browse/HIVE-22546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984733#comment-16984733
]
Hive QA commented on HIVE-22546:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987064/HIVE-22546.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19665/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19665/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19665/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-11-29 04:09:53.507
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-19665/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-11-29 04:09:53.511
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at ab71e5a HIVE-22280: Q tests for partitioned temporary tables
(Laszlo Pinter via Peter Vary)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at ab71e5a HIVE-22280: Q tests for partitioned temporary tables
(Laszlo Pinter via Peter Vary)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-11-29 04:09:54.385
+ rm -rf ../yetus_PreCommit-HIVE-Build-19665
+ mkdir ../yetus_PreCommit-HIVE-Build-19665
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-19665
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-19665/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
standalone-metastore/metastore-server/src/main/sql/postgres/hive-schema-4.0.0.postgres.sql:1829
Falling back to three-way merge...
error: patch failed:
standalone-metastore/metastore-server/src/main/sql/postgres/hive-schema-4.0.0.postgres.sql:1829
error:
standalone-metastore/metastore-server/src/main/sql/postgres/hive-schema-4.0.0.postgres.sql:
patch does not apply
error: patch failed: pom.xml:104
Falling back to three-way merge...
Applied patch to 'pom.xml' with conflicts.
error: hive-site.xml: does not exist in index
error: conf/llap/hive-site.xml: does not exist in index
error: scripts/upgrade/hive/upgrade-3.1.0-to-4.0.0.hive.sql: does not exist in
index
error: metastore-server/src/main/sql/postgres/hive-schema-4.0.0.postgres.sql:
does not exist in index
error:
metastore-server/src/main/sql/postgres/upgrade-3.2.0-to-4.0.0.postgres.sql:
does not exist in index
fatal: git diff header lacks filename information when removing 2 leading
pathname components (line 5)
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-19665
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987064 - PreCommit-HIVE-Build
> Postgres schema not using quoted identifiers for certain tables
> ---
>
> Key: HIVE-22546
> URL: https://issues.apache.org/jira/browse/HIVE-22546
> Project: Hive
> Issue Type: Bug
> Components: Database/Schema
>Affects Versions: 4.0.0
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22546.2.patch, HIVE-22546.3.patch, HIVE-22546.patch
>
>
> In the latest postgresql schema
> (standalone-metastore/metastore-server/src/main/sql/postgres/hive-schema-4.0.0.postgres.sql)
> the following tables have lowercase table and
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984731#comment-16984731
]
Hive QA commented on HIVE-21954:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987052/HIVE-21954.10.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 17815 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[delete_non_acid_table]
(batchId=106)
org.apache.hive.service.server.TestHS2HttpServerPam.org.apache.hive.service.server.TestHS2HttpServerPam
(batchId=243)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19664/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19664/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19664/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987052 - PreCommit-HIVE-Build
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch, HIVE-21954.08.patch, HIVE-21954.09.patch,
> HIVE-21954.10.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick
[jira] [Commented] (HIVE-22552) Some q tests uses the same name for test tables
[
https://issues.apache.org/jira/browse/HIVE-22552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984715#comment-16984715
]
Hive QA commented on HIVE-22552:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987051/HIVE-22552.2.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 17817 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19663/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19663/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19663/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987051 - PreCommit-HIVE-Build
> Some q tests uses the same name for test tables
> ---
>
> Key: HIVE-22552
> URL: https://issues.apache.org/jira/browse/HIVE-22552
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Minor
> Attachments: HIVE-22552.1.patch, HIVE-22552.2.patch
>
>
> Some q tests uses the name "t_test" when creating a test table. This can
> conflict when running the tests parallelly using the same metastore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22563) Required field 'client_protocol' is unset (hive server backward compatibility)
[ https://issues.apache.org/jira/browse/HIVE-22563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhangZhiCheng updated HIVE-22563: - Description: I try to connect to hive server 1.2.1 by using hive client (hive-jdbc-2.3.6.jar) , then I got this issue "Required field 'client_protocol' is unset". Is that means hive server 1.2.1 has no backward compatibility for newer hive client version? !image-2019-11-29-10-20-46-982.png! was: I try to connect to hive server 1.2.1 by using hive client (hive-jdbc-2.3.6.jar) , then I got this issue "Required field 'client_protocol' is unset". is that means hive server 1.2.1 has no backward compatibility for newer hive client version? !image-2019-11-29-10-20-46-982.png! > Required field 'client_protocol' is unset (hive server backward compatibility) > -- > > Key: HIVE-22563 > URL: https://issues.apache.org/jira/browse/HIVE-22563 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 1.2.1, 2.3.6 >Reporter: ZhangZhiCheng >Priority: Blocker > Attachments: image-2019-11-29-10-20-46-982.png > > > I try to connect to hive server 1.2.1 by using hive client > (hive-jdbc-2.3.6.jar) , then I got this issue "Required field > 'client_protocol' is unset". Is that means hive server 1.2.1 has no backward > compatibility for newer hive client version? > !image-2019-11-29-10-20-46-982.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22552) Some q tests uses the same name for test tables
[
https://issues.apache.org/jira/browse/HIVE-22552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984700#comment-16984700
]
Hive QA commented on HIVE-22552:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 2m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19663/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19663/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Some q tests uses the same name for test tables
> ---
>
> Key: HIVE-22552
> URL: https://issues.apache.org/jira/browse/HIVE-22552
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Minor
> Attachments: HIVE-22552.1.patch, HIVE-22552.2.patch
>
>
> Some q tests uses the name "t_test" when creating a test table. This can
> conflict when running the tests parallelly using the same metastore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22553) Expose locks and transactions in sys db
[
https://issues.apache.org/jira/browse/HIVE-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984697#comment-16984697
]
Hive QA commented on HIVE-22553:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986960/HIVE-22553.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 17817 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[strict_managed_tables_sysdb]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19662/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19662/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19662/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986960 - PreCommit-HIVE-Build
> Expose locks and transactions in sys db
> ---
>
> Key: HIVE-22553
> URL: https://issues.apache.org/jira/browse/HIVE-22553
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22553.patch
>
>
> Create new sysdb tables/views to access lock and transaction data.
> This allows to provide admins with live data about ongoing locks and
> transacions. Due to this being in the sys db access to this information can
> be restricted to select privileged users.
> Information about locks and compactions can be joined and accessed at the
> same time.
> Compaction related transactions would also be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22553) Expose locks and transactions in sys db
[
https://issues.apache.org/jira/browse/HIVE-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984681#comment-16984681
]
Hive QA commented on HIVE-22553:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
3s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19662/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| modules | C: metastore U: metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19662/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Expose locks and transactions in sys db
> ---
>
> Key: HIVE-22553
> URL: https://issues.apache.org/jira/browse/HIVE-22553
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22553.patch
>
>
> Create new sysdb tables/views to access lock and transaction data.
> This allows to provide admins with live data about ongoing locks and
> transacions. Due to this being in the sys db access to this information can
> be restricted to select privileged users.
> Information about locks and compactions can be joined and accessed at the
> same time.
> Compaction related transactions would also be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22544) Disable null sort order at user level
[
https://issues.apache.org/jira/browse/HIVE-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984679#comment-16984679
]
Hive QA commented on HIVE-22544:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987044/HIVE-22544.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 17817 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
org.apache.hive.service.server.TestHS2HttpServerPamConfiguration.testPamCorrectConfiguration
(batchId=243)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19661/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19661/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19661/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987044 - PreCommit-HIVE-Build
> Disable null sort order at user level
> -
>
> Key: HIVE-22544
> URL: https://issues.apache.org/jira/browse/HIVE-22544
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Minor
> Attachments: HIVE-22544.1.patch, HIVE-22544.1.patch,
> HIVE-22544.2.patch, HIVE-22544.3.patch
>
>
> "sort order" and "null sort order" in ReduceSinkDesc and TopNKeyDesc should
> not be exposed at user level
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22544) Disable null sort order at user level
[
https://issues.apache.org/jira/browse/HIVE-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984672#comment-16984672
]
Hive QA commented on HIVE-22544:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
18s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19661/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19661/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Disable null sort order at user level
> -
>
> Key: HIVE-22544
> URL: https://issues.apache.org/jira/browse/HIVE-22544
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 4.0.0
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Minor
> Attachments: HIVE-22544.1.patch, HIVE-22544.1.patch,
> HIVE-22544.2.patch, HIVE-22544.3.patch
>
>
> "sort order" and "null sort order" in ReduceSinkDesc and TopNKeyDesc should
> not be exposed at user level
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20150) TopNKey pushdown
[
https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984661#comment-16984661
]
Hive QA commented on HIVE-20150:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987040/HIVE-20150.17.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19660/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19660/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19660/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-11-28 22:23:57.170
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-19660/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-11-28 22:23:57.174
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at ab71e5a HIVE-22280: Q tests for partitioned temporary tables
(Laszlo Pinter via Peter Vary)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at ab71e5a HIVE-22280: Q tests for partitioned temporary tables
(Laszlo Pinter via Peter Vary)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-11-28 22:23:58.038
+ rm -rf ../yetus_PreCommit-HIVE-Build-19660
+ mkdir ../yetus_PreCommit-HIVE-Build-19660
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-19660
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-19660/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java:59
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java' with
conflicts.
error: patch failed:
ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out:100
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out' cleanly.
error: patch failed:
ql/src/test/results/clientpositive/perf/tez/query70.q.out:100
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/perf/tez/query70.q.out'
cleanly.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:3535: trailing whitespace.
Map 5
/data/hiveptest/working/scratch/build.patch:3591: trailing whitespace.
Reducer 3
/data/hiveptest/working/scratch/build.patch:3611: trailing whitespace.
Reducer 4
/data/hiveptest/working/scratch/build.patch:3722: trailing whitespace.
Map 5
/data/hiveptest/working/scratch/build.patch:3781: trailing whitespace.
Reducer 4
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java:59
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java' with
conflicts.
error: patch failed:
ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out:100
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/perf/tez/constraints/query70.q.out' cleanly.
error: patch failed:
ql/src/test/results/clientpositive/perf/tez/query70.q.out:100
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/perf/tez/query70.q.out'
cleanly.
U ql/src/java/org/apache/hadoop/hive/ql/optimizer/TopNKeyProcessor.java
warning: squelched 5 whitespace errors
warning: 10 lines add whitespace
[jira] [Commented] (HIVE-22476) Hive datediff function provided inconsistent results when hive.fetch.task.conversion is set to none
[
https://issues.apache.org/jira/browse/HIVE-22476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984660#comment-16984660
]
Hive QA commented on HIVE-22476:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987034/HIVE-22476.8.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 17817 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19659/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19659/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19659/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987034 - PreCommit-HIVE-Build
> Hive datediff function provided inconsistent results when
> hive.fetch.task.conversion is set to none
> ---
>
> Key: HIVE-22476
> URL: https://issues.apache.org/jira/browse/HIVE-22476
> Project: Hive
> Issue Type: Bug
>Reporter: Slim Bouguerra
>Assignee: Slim Bouguerra
>Priority: Major
> Attachments: HIVE-22476.2.patch, HIVE-22476.3.patch,
> HIVE-22476.5.patch, HIVE-22476.6.patch, HIVE-22476.7.patch,
> HIVE-22476.7.patch, HIVE-22476.8.patch, HIVE-22476.8.patch, HIVE-22476.8.patch
>
>
> The actual issue stems to the different date parser used by various part of
> the engine.
> Fetch task uses udfdatediff via {code}
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDate{code} while the
> vectorized llap execution uses {code}VectorUDFDateDiffScalarCol{code}.
> This fix is meant to be not very intrusive and will add more support to the
> GenericUDFToDate by enhancing the parser.
> For the longer term will be better to use one parser for all the operators.
> Thanks [~Rajkumar Singh] for the repro example
> {code}
> create external table testdatediff(datetimecol string) stored as orc;
> insert into testdatediff values ('2019-09-09T10:45:49+02:00'),('2019-07-24');
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183;
> set hive.ferch.task.conversion=none;
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22476) Hive datediff function provided inconsistent results when hive.fetch.task.conversion is set to none
[
https://issues.apache.org/jira/browse/HIVE-22476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984650#comment-16984650
]
Hive QA commented on HIVE-22476:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
8s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
33s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
50s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19659/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19659/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive datediff function provided inconsistent results when
> hive.fetch.task.conversion is set to none
> ---
>
> Key: HIVE-22476
> URL: https://issues.apache.org/jira/browse/HIVE-22476
> Project: Hive
> Issue Type: Bug
>Reporter: Slim Bouguerra
>Assignee: Slim Bouguerra
>Priority: Major
> Attachments: HIVE-22476.2.patch, HIVE-22476.3.patch,
> HIVE-22476.5.patch, HIVE-22476.6.patch, HIVE-22476.7.patch,
> HIVE-22476.7.patch, HIVE-22476.8.patch, HIVE-22476.8.patch, HIVE-22476.8.patch
>
>
> The actual issue stems to the different date parser used by various part of
> the engine.
> Fetch task uses udfdatediff via {code}
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDate{code} while the
> vectorized llap execution uses {code}VectorUDFDateDiffScalarCol{code}.
> This fix is meant to be not very intrusive and will add more support to the
> GenericUDFToDate by
[jira] [Commented] (HIVE-22555) Upgrade ORC version to 1.5.8
[
https://issues.apache.org/jira/browse/HIVE-22555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984640#comment-16984640
]
Hive QA commented on HIVE-22555:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986999/HIVE-22555.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 17817 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19658/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19658/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19658/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986999 - PreCommit-HIVE-Build
> Upgrade ORC version to 1.5.8
>
>
> Key: HIVE-22555
> URL: https://issues.apache.org/jira/browse/HIVE-22555
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Mustafa Iman
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22555.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive currently depends on ORC 1.5.6. We need 1.5.8 upgrade for
> https://issues.apache.org/jira/browse/HIVE-22499
> ORC-1.5.7 includes https://issues.apache.org/jira/browse/ORC-361 . It causes
> some tests overriding MemoryManager to fail. These need to be addressed while
> upgrading.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22555) Upgrade ORC version to 1.5.8
[
https://issues.apache.org/jira/browse/HIVE-22555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984637#comment-16984637
]
Hive QA commented on HIVE-22555:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
24s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 5 new + 187 unchanged - 2
fixed = 192 total (was 189) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
17s{color} | {color:red} root: The patch generated 5 new + 187 unchanged - 2
fixed = 192 total (was 189) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 70m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19658/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19658/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19658/yetus/diff-checkstyle-root.txt
|
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19658/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade ORC version to 1.5.8
>
>
> Key: HIVE-22555
> URL: https://issues.apache.org/jira/browse/HIVE-22555
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Mustafa Iman
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22555.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive currently depends on ORC 1.5.6. We need 1.5.8 upgrade for
> https://issues.apache.org/jira/browse/HIVE-22499
> ORC-1.5.7 includes https://issues.apache.org/jira/browse/ORC-361 . It causes
> some tests overriding MemoryManager to fail. These need to be addressed while
> upgrading.
>
--
This message was sent by
[jira] [Updated] (HIVE-22551) BytesColumnVector initBuffer should clean vector and length consistently
[
https://issues.apache.org/jira/browse/HIVE-22551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-22551:
Attachment: HIVE-22551.01.patch
> BytesColumnVector initBuffer should clean vector and length consistently
> -
>
> Key: HIVE-22551
> URL: https://issues.apache.org/jira/browse/HIVE-22551
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22551.01.patch, HIVE-22551.01.patch,
> HIVE-22551.01.patch
>
>
> VectorExtractRow relies on the fact that vector[i] and length[i] are
> consistent within the BytesColumnVector, otherwise it throws exception:
> https://github.com/apache/hive/blob/edc53cc0d95e983c371a224943dd866210f0c65c/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorExtractRow.java#L275
> There is a scenario when only vector[i] has been cleaned while reusing the
> column vector, and then this kind of exception can be thrown:
> the reproduction was made with
> [LlapDump|https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapDump.java]
> with String columns (longer than 16 chars)
> {code}
> 19/10/17 15:55:49 ERROR llap.LlapArrowRowRecordReader: Failed to fetch Arrow
> batch
> java.lang.RuntimeException: STRING entry: batchIndex 45
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.BytesReadError(VectorExtractRow.java:488)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRowColumn(VectorExtractRow.java:294)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRowColumn(VectorExtractRow.java:193)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRow(VectorExtractRow.java:483)
> at
> org.apache.hadoop.hive.ql.io.arrow.Deserializer.deserialize(Deserializer.java:125)
> at
> org.apache.hadoop.hive.ql.io.arrow.ArrowColumnarBatchSerDe.deserialize(ArrowColumnarBatchSerDe.java:284)
> at
> org.apache.hadoop.hive.llap.LlapArrowRowRecordReader.next(LlapArrowRowRecordReader.java:75)
> at
> org.apache.hadoop.hive.llap.LlapArrowRowRecordReader.next(LlapArrowRowRecordReader.java:41)
> at datareader.LlapDump.main(LlapDump.java:124)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22551) BytesColumnVector initBuffer should clean vector and length consistently
[
https://issues.apache.org/jira/browse/HIVE-22551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984619#comment-16984619
]
Hive QA commented on HIVE-22551:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987002/HIVE-22551.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 17817 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19657/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19657/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19657/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987002 - PreCommit-HIVE-Build
> BytesColumnVector initBuffer should clean vector and length consistently
> -
>
> Key: HIVE-22551
> URL: https://issues.apache.org/jira/browse/HIVE-22551
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22551.01.patch, HIVE-22551.01.patch
>
>
> VectorExtractRow relies on the fact that vector[i] and length[i] are
> consistent within the BytesColumnVector, otherwise it throws exception:
> https://github.com/apache/hive/blob/edc53cc0d95e983c371a224943dd866210f0c65c/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorExtractRow.java#L275
> There is a scenario when only vector[i] has been cleaned while reusing the
> column vector, and then this kind of exception can be thrown:
> the reproduction was made with
> [LlapDump|https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapDump.java]
> with String columns (longer than 16 chars)
> {code}
> 19/10/17 15:55:49 ERROR llap.LlapArrowRowRecordReader: Failed to fetch Arrow
> batch
> java.lang.RuntimeException: STRING entry: batchIndex 45
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.BytesReadError(VectorExtractRow.java:488)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRowColumn(VectorExtractRow.java:294)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRowColumn(VectorExtractRow.java:193)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRow(VectorExtractRow.java:483)
> at
> org.apache.hadoop.hive.ql.io.arrow.Deserializer.deserialize(Deserializer.java:125)
> at
> org.apache.hadoop.hive.ql.io.arrow.ArrowColumnarBatchSerDe.deserialize(ArrowColumnarBatchSerDe.java:284)
> at
> org.apache.hadoop.hive.llap.LlapArrowRowRecordReader.next(LlapArrowRowRecordReader.java:75)
> at
> org.apache.hadoop.hive.llap.LlapArrowRowRecordReader.next(LlapArrowRowRecordReader.java:41)
> at datareader.LlapDump.main(LlapDump.java:124)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22551) BytesColumnVector initBuffer should clean vector and length consistently
[
https://issues.apache.org/jira/browse/HIVE-22551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984587#comment-16984587
]
Hive QA commented on HIVE-22551:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} storage-api in master has 58 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19657/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: storage-api U: storage-api |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19657/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> BytesColumnVector initBuffer should clean vector and length consistently
> -
>
> Key: HIVE-22551
> URL: https://issues.apache.org/jira/browse/HIVE-22551
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Attachments: HIVE-22551.01.patch, HIVE-22551.01.patch
>
>
> VectorExtractRow relies on the fact that vector[i] and length[i] are
> consistent within the BytesColumnVector, otherwise it throws exception:
> https://github.com/apache/hive/blob/edc53cc0d95e983c371a224943dd866210f0c65c/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorExtractRow.java#L275
> There is a scenario when only vector[i] has been cleaned while reusing the
> column vector, and then this kind of exception can be thrown:
> the reproduction was made with
> [LlapDump|https://github.com/apache/hive/blob/master/llap-ext-client/src/java/org/apache/hadoop/hive/llap/LlapDump.java]
> with String columns (longer than 16 chars)
> {code}
> 19/10/17 15:55:49 ERROR llap.LlapArrowRowRecordReader: Failed to fetch Arrow
> batch
> java.lang.RuntimeException: STRING entry: batchIndex 45
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.BytesReadError(VectorExtractRow.java:488)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRowColumn(VectorExtractRow.java:294)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorExtractRow.extractRowColumn(VectorExtractRow.java:193)
> at
>
[jira] [Commented] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers
[
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984575#comment-16984575
]
Hive QA commented on HIVE-22557:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986995/HIVE-22557.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 17813 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
org.apache.hive.service.cli.thrift.TestThriftHttpCLIServiceFeatures.org.apache.hive.service.cli.thrift.TestThriftHttpCLIServiceFeatures
(batchId=284)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19656/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19656/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19656/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986995 - PreCommit-HIVE-Build
> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Attachments: HIVE-22557.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from
> DDLSemanticAnalyzer, and move them under the new package.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22537) getAcidState() not saving directory snapshot causes multiple calls to S3 api
[ https://issues.apache.org/jira/browse/HIVE-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22537: Status: Patch Available (was: Open) > getAcidState() not saving directory snapshot causes multiple calls to S3 api > > > Key: HIVE-22537 > URL: https://issues.apache.org/jira/browse/HIVE-22537 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22537.2.patch, HIVE-22537.3.patch, > HIVE-22537.4.patch, HIVE-22537.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Fix for HIVE-21225 is not enabled in query coordinator codepath. The last > argument (generateDirSnapshots) for getAcidState() is set to false when > invoked by callInternal(). Also, snapshot is not used for file exists calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22537) getAcidState() not saving directory snapshot causes multiple calls to S3 api
[ https://issues.apache.org/jira/browse/HIVE-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22537: Status: Open (was: Patch Available) > getAcidState() not saving directory snapshot causes multiple calls to S3 api > > > Key: HIVE-22537 > URL: https://issues.apache.org/jira/browse/HIVE-22537 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22537.2.patch, HIVE-22537.3.patch, > HIVE-22537.4.patch, HIVE-22537.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Fix for HIVE-21225 is not enabled in query coordinator codepath. The last > argument (generateDirSnapshots) for getAcidState() is set to false when > invoked by callInternal(). Also, snapshot is not used for file exists calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22557) Break up DDLSemanticAnalyzer - extract Table constraints analyzers
[
https://issues.apache.org/jira/browse/HIVE-22557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984550#comment-16984550
]
Hive QA commented on HIVE-22557:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
20s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 6 new + 177 unchanged - 11
fixed = 183 total (was 188) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19656/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19656/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19656/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLSemanticAnalyzer - extract Table constraints analyzers
> --
>
> Key: HIVE-22557
> URL: https://issues.apache.org/jira/browse/HIVE-22557
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Attachments: HIVE-22557.01.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #10: extract the table constraints related analyzers from
> DDLSemanticAnalyzer, and move them under the new package.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22562) Harmonize SessionState.getUserName
[
https://issues.apache.org/jira/browse/HIVE-22562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984528#comment-16984528
]
Zoltan Haindrich commented on HIVE-22562:
-
my proposal is to drop the {{SS.userName}} field; and make the
{{SS.getUserName()}} a fallback method to access username of the authenticator;
I don't think we should support any situations in which:
* sessionstate is not present
** I think ignoring it might be already problematic
* authenticator is not configured
> Harmonize SessionState.getUserName
> --
>
> Key: HIVE-22562
> URL: https://issues.apache.org/jira/browse/HIVE-22562
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>
> we might have 2 different user names at the same time:
> *
> [getUserName()|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L1912]
> ** a method which relies on the userName field of the SessionState
> *
> [getUserFromAuthenticator()|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L1291]
> ** a method which uses the authenticator to do the heavy lifting
> * there all kind of interesting call sites like:
> ** there are some which are [prefering the authenticator over
> getUserName()|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L254]
> ** there are some which [use getUserName() regardless authenticator, but have
> fixme|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/Driver.java#L1669]
> ** and there are some which are just using the authenticator with or without
> notes/etc
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21851) FireEventResponse should include event id when available
[
https://issues.apache.org/jira/browse/HIVE-21851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984527#comment-16984527
]
Hive QA commented on HIVE-21851:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986991/HIVE-21851.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 17817 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=368)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19655/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19655/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19655/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986991 - PreCommit-HIVE-Build
> FireEventResponse should include event id when available
>
>
> Key: HIVE-21851
> URL: https://issues.apache.org/jira/browse/HIVE-21851
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-21851.01.patch, HIVE-21851.02.patch,
> HIVE-21851.03.patch
>
>
> The metastore API {{fire_listener_event}} gives clients the ability to fire a
> INSERT event on DML operations. However, the returned response is empty
> struct. It would be useful to sent back the event id information in the
> response so that clients can take actions based of the event id.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22562) Harmonize SessionState.getUserName
[ https://issues.apache.org/jira/browse/HIVE-22562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-22562: --- > Harmonize SessionState.getUserName > -- > > Key: HIVE-22562 > URL: https://issues.apache.org/jira/browse/HIVE-22562 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > > we might have 2 different user names at the same time: > * > [getUserName()|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L1912] > ** a method which relies on the userName field of the SessionState > * > [getUserFromAuthenticator()|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L1291] > ** a method which uses the authenticator to do the heavy lifting > * there all kind of interesting call sites like: > ** there are some which are [prefering the authenticator over > getUserName()|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L254] > ** there are some which [use getUserName() regardless authenticator, but have > fixme|https://github.com/apache/hive/blob/ab71e5a22834b5fdd17d6e4ddb54bcd324ae97d7/ql/src/java/org/apache/hadoop/hive/ql/Driver.java#L1669] > ** and there are some which are just using the authenticator with or without > notes/etc -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21851) FireEventResponse should include event id when available
[
https://issues.apache.org/jira/browse/HIVE-21851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984503#comment-16984503
]
Hive QA commented on HIVE-21851:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
43s{color} | {color:blue} standalone-metastore/metastore-common in master has
37 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
16s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
29s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 179 unchanged - 0 fixed = 180 total (was 179) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 4s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | Boxing/unboxing to parse a primitive
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.fire_listener_event(FireEventRequest)
At
HiveMetaStore.java:org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.fire_listener_event(FireEventRequest)
At HiveMetaStore.java:[line 8544] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19655/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19655/yetus/new-findbugs-standalone-metastore_metastore-server.html
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server itests/hcatalog-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19655/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> FireEventResponse should include event id when available
>
>
> Key: HIVE-21851
> URL: https://issues.apache.org/jira/browse/HIVE-21851
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-21851.01.patch, HIVE-21851.02.patch,
> HIVE-21851.03.patch
>
>
> The metastore
[jira] [Commented] (HIVE-22561) Data loss on map join for bucketed, partitioned table
[
https://issues.apache.org/jira/browse/HIVE-22561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984499#comment-16984499
]
Aditya Shah commented on HIVE-22561:
[~djaiswal] [~prasanth_j] [~jcamachorodriguez] Can you please take a look at
this. I tried debugging a bit. Some of the observations I made where:
# The mapjoin operator does not populate the hashtable (hybrid as well as
normal) completely for each task.
# The results vary with the number of buckets.
Is the hashtable distributed in someway according to buckets?
> Data loss on map join for bucketed, partitioned table
> -
>
> Key: HIVE-22561
> URL: https://issues.apache.org/jira/browse/HIVE-22561
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.2
>Reporter: Aditya Shah
>Priority: Blocker
> Attachments: Screenshot 2019-11-28 at 8.45.17 PM.png,
> image-2019-11-28-20-46-25-432.png
>
>
> A map join on a column (which is neither involved in bucketing and partition)
> causes data loss.
> Steps to reproduce:
> Env: [hive-dev-box|[https://github.com/kgyrtkirk/hive-dev-box]] hive 3.1.2.
> Create tables:
>
> {code:java}
> CREATE TABLE `testj2`(
> `id` int,
> `bn` string,
> `cn` string,
> `ad` map,
> `mi` array)
> PARTITIONED BY (
> `br` string)
> CLUSTERED BY (
> bn)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> CREATE TABLE `testj1`(
> `id` int,
> `can` string,
> `cn` string,
> `ad` map,
> `av` boolean,
> `mi` array)
> PARTITIONED BY (
> `brand` string)
> CLUSTERED BY (
> can)
> INTO 2 BUCKETS
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY ','
> STORED AS TEXTFILE
> TBLPROPERTIES (
> 'bucketing_version'='2');
> {code}
> insert some data in both:
> {code:java}
> insert into testj1 values (100, 'mes_1', 'customer_1', map('city1', 560077),
> false, array(5, 10), 'brand_1'),
> (101, 'mes_2', 'customer_2', map('city2', 560078), true, array(10, 20),
> 'brand_2'),
> (102, 'mes_3', 'customer_3', map('city3', 560079), false, array(15, 30),
> 'brand_3'),
> (103, 'mes_4', 'customer_4', map('city4', 560080), true, array(20, 40),
> 'brand_4'),
> (104, 'mes_5', 'customer_5', map('city5', 560081), false, array(25, 50),
> 'brand_5');
> insert into table testj2 values (100, 'tv_0', 'customer_0', map('city0',
> 560076),array(0, 0, 0), 'tv'),
> (101, 'tv_1', 'customer_1', map('city1', 560077),array(20, 25, 30), 'tv'),
> (102, 'tv_2', 'customer_2', map('city2', 560078),array(40, 50, 60), 'tv'),
> (103, 'tv_3', 'customer_3', map('city3', 560079),array(60, 75, 90), 'tv'),
> (104, 'tv_4', 'customer_4', map('city4', 560080),array(80, 100, 120), 'tv');
> {code}
> Do a join between them:
> {code:java}
> select t1.id, t1.can, t1.cn, t2.bn,t2.ad, t2.br FROM testj1 t1 JOIN testj2 t2
> on (t1.id = t2.id) order by t1.id;
> {code}
> Observed results:
> !image-2019-11-28-20-46-25-432.png|width=524,height=100!
> In the plan, I can see a map join. Disabling it gives the correct result.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22521) Both Driver and SessionState has a userName
[ https://issues.apache.org/jira/browse/HIVE-22521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984482#comment-16984482 ] Zoltan Haindrich commented on HIVE-22521: - [~mgergely] could you please take a look? > Both Driver and SessionState has a userName > --- > > Key: HIVE-22521 > URL: https://issues.apache.org/jira/browse/HIVE-22521 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22521.01.patch, HIVE-22521.01.patch, > HIVE-22521.01.patch, HIVE-22521.01.patch, HIVE-22521.01.patch, > HIVE-22521.01.patch > > > This caused some confusing behaviour to me...especially when the 2 values > were different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22556) Unset the credential provider config in the tez jobs configurations
[
https://issues.apache.org/jira/browse/HIVE-22556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984481#comment-16984481
]
Hive QA commented on HIVE-22556:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986993/HIVE-22556.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 17817 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition2]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition3]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition4]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition5]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_boolexpr]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_ctas]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
org.apache.hive.spark.client.rpc.TestRpc.testClientTimeout (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19654/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19654/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19654/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986993 - PreCommit-HIVE-Build
> Unset the credential provider config in the tez jobs configurations
> ---
>
> Key: HIVE-22556
> URL: https://issues.apache.org/jira/browse/HIVE-22556
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22556.1.patch
>
>
> Unset the credential provider config in the tez jobs configurations
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22556) Unset the credential provider config in the tez jobs configurations
[
https://issues.apache.org/jira/browse/HIVE-22556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984450#comment-16984450
]
Hive QA commented on HIVE-22556:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
30s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19654/dev-support/hive-personality.sh
|
| git revision | master / ab71e5a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19654/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Unset the credential provider config in the tez jobs configurations
> ---
>
> Key: HIVE-22556
> URL: https://issues.apache.org/jira/browse/HIVE-22556
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22556.1.patch
>
>
> Unset the credential provider config in the tez jobs configurations
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-22519) TestMiniLlapLocalCliDriver#testCliDriver[sysdb_schq] fails intermittently
[ https://issues.apache.org/jira/browse/HIVE-22519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich resolved HIVE-22519. - Fix Version/s: 4.0.0 Resolution: Fixed I think this is much better now; HIVE-22516 have made this better by fixing the sessionid > TestMiniLlapLocalCliDriver#testCliDriver[sysdb_schq] fails intermittently > - > > Key: HIVE-22519 > URL: https://issues.apache.org/jira/browse/HIVE-22519 > Project: Hive > Issue Type: Bug > Components: Tests >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > > Sometimes there is some "desc formatted" alike output in the q.out ; which > shows up as a difference -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Summary: Reduce Sink operator should order nulls by parameter (was:
Reduce Sink operator orders nulls first)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.2.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string value}
> serialization.format 1
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 5812
> A masked pattern was here
>
[jira] [Updated] (HIVE-22554) ACID: Wait timeout for blocking compaction should be configurable
[
https://issues.apache.org/jira/browse/HIVE-22554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-22554:
-
Attachment: HIVE-22554.01.patch
> ACID: Wait timeout for blocking compaction should be configurable
> -
>
> Key: HIVE-22554
> URL: https://issues.apache.org/jira/browse/HIVE-22554
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22554.01.patch
>
>
> The wait timeout for blocking compaction is hardcoded to 5 minutes.
> {code:java}
> public class AlterTableCompactOperation extends
> DDLOperation {
> private static final int FIVE_MINUTES_IN_MILLIES = 5*60*1000;
> ...
> }{code}
> This should be configurable via a Hive Configuration parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22554) ACID: Wait timeout for blocking compaction should be configurable
[
https://issues.apache.org/jira/browse/HIVE-22554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-22554:
-
Attachment: (was: HIVE-22554.01.patch)
> ACID: Wait timeout for blocking compaction should be configurable
> -
>
> Key: HIVE-22554
> URL: https://issues.apache.org/jira/browse/HIVE-22554
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22554.01.patch
>
>
> The wait timeout for blocking compaction is hardcoded to 5 minutes.
> {code:java}
> public class AlterTableCompactOperation extends
> DDLOperation {
> private static final int FIVE_MINUTES_IN_MILLIES = 5*60*1000;
> ...
> }{code}
> This should be configurable via a Hive Configuration parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22554) ACID: Wait timeout for blocking compaction should be configurable
[
https://issues.apache.org/jira/browse/HIVE-22554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-22554:
-
Attachment: HIVE-22554.01.patch
> ACID: Wait timeout for blocking compaction should be configurable
> -
>
> Key: HIVE-22554
> URL: https://issues.apache.org/jira/browse/HIVE-22554
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22554.01.patch
>
>
> The wait timeout for blocking compaction is hardcoded to 5 minutes.
> {code:java}
> public class AlterTableCompactOperation extends
> DDLOperation {
> private static final int FIVE_MINUTES_IN_MILLIES = 5*60*1000;
> ...
> }{code}
> This should be configurable via a Hive Configuration parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22537) getAcidState() not saving directory snapshot causes multiple calls to S3 api
[
https://issues.apache.org/jira/browse/HIVE-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984417#comment-16984417
]
Hive QA commented on HIVE-22537:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986985/HIVE-22537.4.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17753 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_partitioned]
(batchId=4)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19653/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19653/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19653/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986985 - PreCommit-HIVE-Build
> getAcidState() not saving directory snapshot causes multiple calls to S3 api
>
>
> Key: HIVE-22537
> URL: https://issues.apache.org/jira/browse/HIVE-22537
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Mustafa Iman
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22537.2.patch, HIVE-22537.3.patch,
> HIVE-22537.4.patch, HIVE-22537.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Fix for HIVE-21225 is not enabled in query coordinator codepath. The last
> argument (generateDirSnapshots) for getAcidState() is set to false when
> invoked by callInternal(). Also, snapshot is not used for file exists calls.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22327) Repl: Ignore read-only transactions in notification log
[ https://issues.apache.org/jira/browse/HIVE-22327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22327: -- Attachment: HIVE-22327.13.patch > Repl: Ignore read-only transactions in notification log > --- > > Key: HIVE-22327 > URL: https://issues.apache.org/jira/browse/HIVE-22327 > Project: Hive > Issue Type: Improvement > Components: repl >Reporter: Gopal Vijayaraghavan >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22327.1.patch, HIVE-22327.10.patch, > HIVE-22327.11.patch, HIVE-22327.12.patch, HIVE-22327.13.patch, > HIVE-22327.2.patch, HIVE-22327.3.patch, HIVE-22327.4.patch, > HIVE-22327.5.patch, HIVE-22327.6.patch, HIVE-22327.7.patch, > HIVE-22327.8.patch, HIVE-22327.9.patch > > > Read txns need not be replicated. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22554) ACID: Wait timeout for blocking compaction should be configurable
[
https://issues.apache.org/jira/browse/HIVE-22554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-22554:
-
Status: Patch Available (was: Open)
> ACID: Wait timeout for blocking compaction should be configurable
> -
>
> Key: HIVE-22554
> URL: https://issues.apache.org/jira/browse/HIVE-22554
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22554.01.patch
>
>
> The wait timeout for blocking compaction is hardcoded to 5 minutes.
> {code:java}
> public class AlterTableCompactOperation extends
> DDLOperation {
> private static final int FIVE_MINUTES_IN_MILLIES = 5*60*1000;
> ...
> }{code}
> This should be configurable via a Hive Configuration parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22538) RS deduplication does not always enforce hive.optimize.reducededuplication.min.reducer
[
https://issues.apache.org/jira/browse/HIVE-22538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984404#comment-16984404
]
Peter Vary commented on HIVE-22538:
---
[~jcamachorodriguez]: Maybe because the fix is very old (HIVE-8367), and nobody
realized that this could be done differently. Do you know how can these
traits/parallelism be set? It might worth a try.
Thanks,
Peter
> RS deduplication does not always enforce
> hive.optimize.reducededuplication.min.reducer
> --
>
> Key: HIVE-22538
> URL: https://issues.apache.org/jira/browse/HIVE-22538
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22538.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> For transactional tables, that property might be overriden to 1, which can
> lead to merging final aggregation into a single stage (hence leading to
> performance degradation). For instance, when autogather column stats is
> enabled, this can happen for the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-21765) Support partitions on temporary tables
[ https://issues.apache.org/jira/browse/HIVE-21765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter resolved HIVE-21765. -- Resolution: Fixed > Support partitions on temporary tables > -- > > Key: HIVE-21765 > URL: https://issues.apache.org/jira/browse/HIVE-21765 > Project: Hive > Issue Type: New Feature >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > > HIVE-18739 and HIVE-20661 added partial support for partition columns on > temporary tables, but it was not complete and it was available only for > internal usage. The scope of this jira, is to implement the missing parts and > expose this functionality. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22280) Q tests for partitioned temporary tables
[ https://issues.apache.org/jira/browse/HIVE-22280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984403#comment-16984403 ] Laszlo Pinter commented on HIVE-22280: -- [~pvary] Thanks for the review and commit. > Q tests for partitioned temporary tables > > > Key: HIVE-22280 > URL: https://issues.apache.org/jira/browse/HIVE-22280 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22280.01.patch, HIVE-22280.02.patch, > HIVE-22280.03.patch, HIVE-22280.04.patch, HIVE-22280.05.patch, > HIVE-22280.06.patch, HIVE-22280.07.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22280) Q tests for partitioned temporary tables
[ https://issues.apache.org/jira/browse/HIVE-22280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-22280: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks for the patch [~lpinter]! Please check the next few runs of ptest for failures and check how much extra test time is added by this patch. Thanks, Peter > Q tests for partitioned temporary tables > > > Key: HIVE-22280 > URL: https://issues.apache.org/jira/browse/HIVE-22280 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22280.01.patch, HIVE-22280.02.patch, > HIVE-22280.03.patch, HIVE-22280.04.patch, HIVE-22280.05.patch, > HIVE-22280.06.patch, HIVE-22280.07.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22537) getAcidState() not saving directory snapshot causes multiple calls to S3 api
[
https://issues.apache.org/jira/browse/HIVE-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984389#comment-16984389
]
Hive QA commented on HIVE-22537:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
17s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} ql: The patch generated 0 new + 584 unchanged - 3
fixed = 584 total (was 587) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19653/dev-support/hive-personality.sh
|
| git revision | master / 7c796af |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19653/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> getAcidState() not saving directory snapshot causes multiple calls to S3 api
>
>
> Key: HIVE-22537
> URL: https://issues.apache.org/jira/browse/HIVE-22537
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Mustafa Iman
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22537.2.patch, HIVE-22537.3.patch,
> HIVE-22537.4.patch, HIVE-22537.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Fix for HIVE-21225 is not enabled in query coordinator codepath. The last
> argument (generateDirSnapshots) for getAcidState() is set to false when
> invoked by callInternal(). Also, snapshot is not used for file exists calls.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22553) Expose locks and transactions in sys db
[ https://issues.apache.org/jira/browse/HIVE-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984381#comment-16984381 ] Peter Vary commented on HIVE-22553: --- This might change a few q.out files as well. There were some query tests where the sysdb schema is explicitly included. > Expose locks and transactions in sys db > --- > > Key: HIVE-22553 > URL: https://issues.apache.org/jira/browse/HIVE-22553 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Major > Attachments: HIVE-22553.patch > > > Create new sysdb tables/views to access lock and transaction data. > This allows to provide admins with live data about ongoing locks and > transacions. Due to this being in the sys db access to this information can > be restricted to select privileged users. > Information about locks and compactions can be joined and accessed at the > same time. > Compaction related transactions would also be visible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22546) Postgres schema not using quoted identifiers for certain tables
[
https://issues.apache.org/jira/browse/HIVE-22546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Chovan updated HIVE-22546:
-
Attachment: HIVE-22546.3.patch
> Postgres schema not using quoted identifiers for certain tables
> ---
>
> Key: HIVE-22546
> URL: https://issues.apache.org/jira/browse/HIVE-22546
> Project: Hive
> Issue Type: Bug
> Components: Database/Schema
>Affects Versions: 4.0.0
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22546.2.patch, HIVE-22546.3.patch, HIVE-22546.patch
>
>
> In the latest postgresql schema
> (standalone-metastore/metastore-server/src/main/sql/postgres/hive-schema-4.0.0.postgres.sql)
> the following tables have lowercase table and column names:
> {code:java}
> aux_table
> compaction_queue
> completed_compactions
> completed_txn_components
> hive_locks
> materialization_rebuild_locks
> min_history_level
> next_compaction_queue_id
> next_lock_id
> next_txn_id
> next_write_id
> repl_txn_map
> runtime_stats
> txn_components
> txn_to_write_id
> txns
> write_set{code}
> As these tables are referenced from the Hive sys database, the queries to
> these tables will fail with a "Table not found" error.
> The problem is that the table and column names are not enclosed in quotes, so
> postgres will turn these identifiers into lowercase.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22395) Add ability to read Druid metastore password from jceks
[
https://issues.apache.org/jira/browse/HIVE-22395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984375#comment-16984375
]
Hive QA commented on HIVE-22395:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986979/HIVE-22395.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17751 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcWithMiniHS2ErasureCoding.testDescribeErasureCoding
(batchId=289)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19652/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19652/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19652/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986979 - PreCommit-HIVE-Build
> Add ability to read Druid metastore password from jceks
> ---
>
> Key: HIVE-22395
> URL: https://issues.apache.org/jira/browse/HIVE-22395
> Project: Hive
> Issue Type: Bug
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22395.1.patch, HIVE-22395.2.patch, HIVE-22395.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22395) Add ability to read Druid metastore password from jceks
[
https://issues.apache.org/jira/browse/HIVE-22395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984331#comment-16984331
]
Hive QA commented on HIVE-22395:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} druid-handler in master has 3 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19652/dev-support/hive-personality.sh
|
| git revision | master / 7c796af |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: druid-handler U: druid-handler |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19652/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add ability to read Druid metastore password from jceks
> ---
>
> Key: HIVE-22395
> URL: https://issues.apache.org/jira/browse/HIVE-22395
> Project: Hive
> Issue Type: Bug
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22395.1.patch, HIVE-22395.2.patch, HIVE-22395.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22529) Make Debugging Stacktrace More Explicit
[
https://issues.apache.org/jira/browse/HIVE-22529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984315#comment-16984315
]
Hive QA commented on HIVE-22529:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986978/HIVE-22529.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17751 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19651/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19651/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19651/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986978 - PreCommit-HIVE-Build
> Make Debugging Stacktrace More Explicit
> ---
>
> Key: HIVE-22529
> URL: https://issues.apache.org/jira/browse/HIVE-22529
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22529.1.patch, HIVE-22529.1.patch
>
>
> In some places, the following DEBUG logging was introduced:
> {code:java}
> LOG.debug("Message", new Exception());
> {code}
> The purpose of this is to log the stack trace of the Thread calling this
> debug logging method. However, the resulting log message includes the
> following:
> {code:none}
> 2019-11-19T08:13:31,392 DEBUG [Thread] Logger: Message
> java.lang.Exception: null
> at
> {code}
> To the observer, it appears that there was perhaps some sort of NPE. Add a
> message to the Exception being generated to make it more clear that this
> "Exception" is for debugging purposes and not an actual error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-22542) Update orc version to 1.5.8
[ https://issues.apache.org/jira/browse/HIVE-22542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter resolved HIVE-22542. -- Resolution: Duplicate > Update orc version to 1.5.8 > --- > > Key: HIVE-22542 > URL: https://issues.apache.org/jira/browse/HIVE-22542 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > Attachments: HIVE-22542.01.patch > > > Update orc to 1.5.8 in root pom -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22542) Update orc version to 1.5.8
[ https://issues.apache.org/jira/browse/HIVE-22542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter updated HIVE-22542: - Status: Open (was: Patch Available) > Update orc version to 1.5.8 > --- > > Key: HIVE-22542 > URL: https://issues.apache.org/jira/browse/HIVE-22542 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > Attachments: HIVE-22542.01.patch > > > Update orc to 1.5.8 in root pom -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-21954:
Attachment: HIVE-21954.10.patch
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch, HIVE-21954.08.patch, HIVE-21954.09.patch,
> HIVE-21954.10.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984298#comment-16984298
]
Hive QA commented on HIVE-22066:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
4s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 23m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 20m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
33s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
10s{color} | {color:red} contrib in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
12s{color} | {color:red} hplsql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
14s{color} | {color:red} llap-ext-client in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 37m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 37m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
46s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 21m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}141m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19650/dev-support/hive-personality.sh
|
| git revision | master / 520aa19 |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19650/yetus/patch-mvninstall-ql.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19650/yetus/patch-mvninstall-llap-server.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19650/yetus/patch-mvninstall-contrib.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19650/yetus/patch-mvninstall-hplsql.txt
|
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19650/yetus/patch-mvninstall-llap-ext-client.txt
|
| modules | C: storage-api shims/common shims/0.23 shims/scheduler
standalone-metastore standalone-metastore/metastore-common common service-rpc
serde metastore vector-code-gen llap-common llap-client llap-tez spark-client
ql llap-server service accumulo-handler jdbc beeline cli contrib druid-handler
hbase-handler jdbc-handler hcatalog hcatalog/core hcatalog/hcatalog-pig-adapter
hcatalog/server-extensions hcatalog/webhcat/java-client hcatalog/webhcat/svr
hcatalog/streaming hplsql streaming llap-ext-client kudu-handler kafka-handler
standalone-metastore/metastore-tools
standalone-metastore/metastore-tools/metastore-benchmarks upgrade-acid
upgrade-acid/pre-upgrade . itests/hive-jmh itests/qtest-druid U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19650/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade Apache parent POM to version 21
>
[jira] [Commented] (HIVE-21954) QTest: support for running qtests on various metastore DBs
[
https://issues.apache.org/jira/browse/HIVE-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984293#comment-16984293
]
Zoltan Haindrich commented on HIVE-21954:
-
+1
> QTest: support for running qtests on various metastore DBs
> --
>
> Key: HIVE-21954
> URL: https://issues.apache.org/jira/browse/HIVE-21954
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore, Testing Infrastructure
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch,
> HIVE-21954.03.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
> HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch,
> HIVE-21954.07.patch, HIVE-21954.08.patch, HIVE-21954.09.patch
>
>
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local
> reproduction has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> -Dmetastore.itest.no.stop.container=true
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest
> -Dtest=TestCliDriver -Dqfile=partition_params_postgres.q
> -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be
> edited manually. My proposal is to introduce a property
> (-Dmetastore.db=postgres), which can take care of the parameters on the fly.
> 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password,
> db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro
> patch|https://issues.apache.org/jira/secure/attachment/12973534/HIVE-21940.repro.patch]
>
> In the first implementation, I would not start metastore db automatically
> (which is done be 'mvn verify ...'), but it's still under planning.
> In the long term, we should consider running this kind of tests in precommit
> phase, so maybe -Dmetastore.db=postgres could start metastore db
> automatically. Also we should consider running some qtests on various
> metastores. I would not pick randomly, but choose some "metastore-heavy" ones
> instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22552) Some q tests uses the same name for test tables
[ https://issues.apache.org/jira/browse/HIVE-22552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22552: -- Status: Patch Available (was: Open) > Some q tests uses the same name for test tables > --- > > Key: HIVE-22552 > URL: https://issues.apache.org/jira/browse/HIVE-22552 > Project: Hive > Issue Type: Bug > Components: Tests >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22552.1.patch, HIVE-22552.2.patch > > > Some q tests uses the name "t_test" when creating a test table. This can > conflict when running the tests parallelly using the same metastore. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22552) Some q tests uses the same name for test tables
[ https://issues.apache.org/jira/browse/HIVE-22552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22552: -- Attachment: HIVE-22552.2.patch > Some q tests uses the same name for test tables > --- > > Key: HIVE-22552 > URL: https://issues.apache.org/jira/browse/HIVE-22552 > Project: Hive > Issue Type: Bug > Components: Tests >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22552.1.patch, HIVE-22552.2.patch > > > Some q tests uses the name "t_test" when creating a test table. This can > conflict when running the tests parallelly using the same metastore. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22552) Some q tests uses the same name for test tables
[ https://issues.apache.org/jira/browse/HIVE-22552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22552: -- Status: Open (was: Patch Available) > Some q tests uses the same name for test tables > --- > > Key: HIVE-22552 > URL: https://issues.apache.org/jira/browse/HIVE-22552 > Project: Hive > Issue Type: Bug > Components: Tests >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22552.1.patch, HIVE-22552.2.patch > > > Some q tests uses the name "t_test" when creating a test table. This can > conflict when running the tests parallelly using the same metastore. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22529) Make Debugging Stacktrace More Explicit
[
https://issues.apache.org/jira/browse/HIVE-22529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984280#comment-16984280
]
Hive QA commented on HIVE-22529:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
50s{color} | {color:blue} standalone-metastore/metastore-server in master has
179 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19651/dev-support/hive-personality.sh
|
| git revision | master / 7c796af |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19651/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Make Debugging Stacktrace More Explicit
> ---
>
> Key: HIVE-22529
> URL: https://issues.apache.org/jira/browse/HIVE-22529
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22529.1.patch, HIVE-22529.1.patch
>
>
> In some places, the following DEBUG logging was introduced:
> {code:java}
> LOG.debug("Message", new Exception());
> {code}
> The purpose of this is to log the stack trace of the Thread calling this
> debug logging method. However, the resulting log message includes the
> following:
> {code:none}
> 2019-11-19T08:13:31,392 DEBUG [Thread] Logger: Message
> java.lang.Exception: null
> at
> {code}
> To the observer, it appears that there was perhaps some sort of NPE. Add a
> message to the Exception being generated to make it more clear that this
> "Exception" is for debugging purposes and not an actual error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22553) Expose locks and transactions in sys db
[ https://issues.apache.org/jira/browse/HIVE-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Chovan updated HIVE-22553: - Status: Patch Available (was: Open) > Expose locks and transactions in sys db > --- > > Key: HIVE-22553 > URL: https://issues.apache.org/jira/browse/HIVE-22553 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Major > Attachments: HIVE-22553.patch > > > Create new sysdb tables/views to access lock and transaction data. > This allows to provide admins with live data about ongoing locks and > transacions. Due to this being in the sys db access to this information can > be restricted to select privileged users. > Information about locks and compactions can be joined and accessed at the > same time. > Compaction related transactions would also be visible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984254#comment-16984254
]
Hive QA commented on HIVE-22066:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12986976/HIVE-22066.8.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17751 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=285)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19650/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19650/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19650/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12986976 - PreCommit-HIVE-Build
> Upgrade Apache parent POM to version 21
> ---
>
> Key: HIVE-22066
> URL: https://issues.apache.org/jira/browse/HIVE-22066
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch,
> HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch,
> HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch,
> HIVE-22066.8.patch, HIVE-22066.patch, HIVE-22066.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22560) ACID MERGE fails if NOT NULL constraint is used for column and PDD is enabled
[
https://issues.apache.org/jira/browse/HIVE-22560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aron Hamvas updated HIVE-22560:
---
Description:
If hive.optimize.ppd=true is set, ACID MERGE can fail if the column used in
matching has NOT NULL constraint.
To reproduce the issue:
{code:sql}
CREATE TABLE SIMPLECDCMASTER (CLI_NUM VARCHAR(13) NOT NULL , FIELD1
VARCHAR(100));
CREATE TABLE SIMPLECDCDELTA (CLI_NUM VARCHAR(13), FIELD1 VARCHAR(100));
insert into simplecdcdelta values (123, 'abc');
merge into simplecdcmaster as m using simplecdcdelta as d on m.cli_num =
d.cli_num
when matched then update set field1=d.field1
when not matched then insert values (d.cli_num, d.field1);
{code}
The NOT NULL constraint is then enforced in the Filter operator processing the
output of the right outer join, where, of course, the not matched records have
nulls in all columns.
was:
If hive.optimize.ppd=true is set, ACID MERGE can fail if the column used in
matching has NOT NULL constraint.
To reproduce the issue:
{code:sql}
CREATE TABLE SIMPLECDCMASTER (CLI_NUM VARCHAR(13) NOT NULL , FIELD1
VARCHAR(100));
CREATE TABLE SIMPLECDCDELTA (CLI_NUM VARCHAR(13), FIELD1 VARCHAR(100));
insert into simplecdcdelta values (123, 'abc');
merge into simplecdcmaster as m using simplecdcdelta as d on m.cli_num =
d.cli_num
when matched then update set field1=d.field1
when not matched then insert values (d.cli_num, d.field1);
{code}
The NOT NULL constrained is then enforced in the Filter operator processing the
output of the right outer join, where, of course, the not matched records have
nulls in all columns.
> ACID MERGE fails if NOT NULL constraint is used for column and PDD is enabled
> -
>
> Key: HIVE-22560
> URL: https://issues.apache.org/jira/browse/HIVE-22560
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 3.1.2
>Reporter: Aron Hamvas
>Priority: Major
>
> If hive.optimize.ppd=true is set, ACID MERGE can fail if the column used in
> matching has NOT NULL constraint.
> To reproduce the issue:
> {code:sql}
> CREATE TABLE SIMPLECDCMASTER (CLI_NUM VARCHAR(13) NOT NULL , FIELD1
> VARCHAR(100));
> CREATE TABLE SIMPLECDCDELTA (CLI_NUM VARCHAR(13), FIELD1 VARCHAR(100));
> insert into simplecdcdelta values (123, 'abc');
> merge into simplecdcmaster as m using simplecdcdelta as d on m.cli_num =
> d.cli_num
> when matched then update set field1=d.field1
> when not matched then insert values (d.cli_num, d.field1);
> {code}
> The NOT NULL constraint is then enforced in the Filter operator processing
> the output of the right outer join, where, of course, the not matched records
> have nulls in all columns.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22544) Disable null sort order at user level
[ https://issues.apache.org/jira/browse/HIVE-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22544: -- Status: Patch Available (was: Open) > Disable null sort order at user level > - > > Key: HIVE-22544 > URL: https://issues.apache.org/jira/browse/HIVE-22544 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22544.1.patch, HIVE-22544.1.patch, > HIVE-22544.2.patch, HIVE-22544.3.patch > > > "sort order" and "null sort order" in ReduceSinkDesc and TopNKeyDesc should > not be exposed at user level -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22544) Disable null sort order at user level
[ https://issues.apache.org/jira/browse/HIVE-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22544: -- Attachment: HIVE-22544.3.patch > Disable null sort order at user level > - > > Key: HIVE-22544 > URL: https://issues.apache.org/jira/browse/HIVE-22544 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22544.1.patch, HIVE-22544.1.patch, > HIVE-22544.2.patch, HIVE-22544.3.patch > > > "sort order" and "null sort order" in ReduceSinkDesc and TopNKeyDesc should > not be exposed at user level -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22544) Disable null sort order at user level
[ https://issues.apache.org/jira/browse/HIVE-22544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22544: -- Status: Open (was: Patch Available) > Disable null sort order at user level > - > > Key: HIVE-22544 > URL: https://issues.apache.org/jira/browse/HIVE-22544 > Project: Hive > Issue Type: Bug > Components: Query Planning >Affects Versions: 4.0.0 >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Minor > Attachments: HIVE-22544.1.patch, HIVE-22544.1.patch, > HIVE-22544.2.patch, HIVE-22544.3.patch > > > "sort order" and "null sort order" in ReduceSinkDesc and TopNKeyDesc should > not be exposed at user level -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22379) Reduce db lookups during dynamic partition loading
[
https://issues.apache.org/jira/browse/HIVE-22379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-22379:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to master, thanks [~rajesh.balamohan] for review.
> Reduce db lookups during dynamic partition loading
> --
>
> Key: HIVE-22379
> URL: https://issues.apache.org/jira/browse/HIVE-22379
> Project: Hive
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: performance
> Attachments: HIVE-22379.1.patch, HIVE-22379.2.patch,
> HIVE-22379.3.patch, HIVE-22379.4.patch, HIVE-22379.5.patch,
> HIVE-22379.6.patch, HIVE-22379.7.patch
>
>
> {{HiveAlterHandler::alterPartitions}} could lookup all partition details via
> single call instead of multiple lookups.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22559) Maintain ownership of parent directories of an external table directory after replication
[ https://issues.apache.org/jira/browse/HIVE-22559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat reassigned HIVE-22559: - > Maintain ownership of parent directories of an external table directory after > replication > - > > Key: HIVE-22559 > URL: https://issues.apache.org/jira/browse/HIVE-22559 > Project: Hive > Issue Type: Improvement >Reporter: Ashutosh Bapat >Assignee: Anishek Agarwal >Priority: Major > > For replicating an external table we specify a base directory on the target > (say /base_ext for example). The path of an external table directory on the > source (say /xyz/abc/ext_t1) is prefixed with the base directory on the > target (/base_ext in our example) when replicating the external table data. > Thus the path of the external table on the target becomes > /base_ext/xyz/abc/ext_t1. In this path only the ownership permissions of > ext_t1 directory is preserved but the owenship of xyz and abc directories is > set to the user executing REPL LOAD. Instead we should preserve the ownership > of xyz and abc as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Attachment: HIVE-20150.17.patch > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.2.patch, HIVE-20150.4.patch, HIVE-20150.5.patch, > HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Patch Available (was: Open) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.2.patch, HIVE-20150.4.patch, HIVE-20150.5.patch, > HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20150) TopNKey pushdown
[ https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-20150: -- Status: Open (was: Patch Available) > TopNKey pushdown > > > Key: HIVE-20150 > URL: https://issues.apache.org/jira/browse/HIVE-20150 > Project: Hive > Issue Type: Improvement > Components: Physical Optimizer >Affects Versions: 4.0.0 >Reporter: Teddy Choi >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch, > HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch, > HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch, > HIVE-20150.2.patch, HIVE-20150.4.patch, HIVE-20150.5.patch, > HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch > > > TopNKey operator is implemented in HIVE-17896, but it needs more work in > pushdown implementation. So this issue covers TopNKey pushdown implementation > with proper tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
