[jira] [Commented] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'
[
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657741#comment-16657741
]
Hive QA commented on HIVE-18767:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 20s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-14583/patches/PreCommit-HIVE-Build-14583.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14583/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Some alterPartitions invocations throw 'NumberFormatException: null'
>
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.3, 3.1.0, 4.0.0, 3.2.0
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 3.1.1, 2.3.4
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767-branch-2.patch,
> HIVE-18767-branch-3.1.patch, HIVE-18767-branch-3.patch, HIVE-18767.1.patch,
> HIVE-18767.2-branch-2.3.patch, HIVE-18767.2-branch-2.patch,
> HIVE-18767.2-branch-3.1.patch, HIVE-18767.2.branch-3.patch,
> HIVE-18767.2.patch, HIVE-18767.3-branch-3.1.patch,
> HIVE-18767.3.branch-3.patch, HIVE-18767.3.patch,
> HIVE-18767.4-branch-3.1.patch, HIVE-18767.4.branch-3.1.patch,
> HIVE-18767.4.patch, HIVE-18767.5.patch, HIVE-18767.6.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown
> Source)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[
https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657738#comment-16657738
]
Hive QA commented on HIVE-20703:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944777/HIVE-20703.11.patch
{color:green}SUCCESS:{color} +1 due to 10 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15109 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=168)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=258)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=258)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14582/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14582/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14582/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944777 - PreCommit-HIVE-Build
> Put dynamic sort partition optimization under cost based decision
> -
>
> Key: HIVE-20703
> URL: https://issues.apache.org/jira/browse/HIVE-20703
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch,
> HIVE-20703.11.patch, HIVE-20703.2.patch, HIVE-20703.3.patch,
> HIVE-20703.4.patch, HIVE-20703.5.patch, HIVE-20703.6.patch,
> HIVE-20703.7.patch, HIVE-20703.8.patch, HIVE-20703.9.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[
https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657731#comment-16657731
]
Hive QA commented on HIVE-20703:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
44s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
19s{color} | {color:red} common: The patch generated 1 new + 425 unchanged - 0
fixed = 426 total (was 425) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 3 new + 57 unchanged - 9 fixed
= 60 total (was 66) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14582/dev-support/hive-personality.sh
|
| git revision | master / 7ed8bf4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14582/yetus/diff-checkstyle-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14582/yetus/diff-checkstyle-ql.txt
|
| modules | C: common itests/hive-blobstore ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14582/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Put dynamic sort partition optimization under cost based decision
> -
>
> Key: HIVE-20703
> URL: https://issues.apache.org/jira/browse/HIVE-20703
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch,
> HIVE-20703.11.patch, HIVE-20703.2.patch, HIVE-20703.3.patch,
> HIVE-20703.4.patch, HIVE-20703.5.patch, HIVE-20703.6.patch,
> HIVE-20703.7.patch, HIVE-20703.8.patch, HIVE-20703.9.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20701) Allow HiveStreaming to receive a key value to commit atomically together with the transaction
[
https://issues.apache.org/jira/browse/HIVE-20701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657708#comment-16657708
]
Hive QA commented on HIVE-20701:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944773/HIVE-20701.7.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15112 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14581/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14581/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14581/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944773 - PreCommit-HIVE-Build
> Allow HiveStreaming to receive a key value to commit atomically together with
> the transaction
> -
>
> Key: HIVE-20701
> URL: https://issues.apache.org/jira/browse/HIVE-20701
> Project: Hive
> Issue Type: Improvement
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Major
> Attachments: HIVE-20701.1.patch, HIVE-20701.2.patch,
> HIVE-20701.3.patch, HIVE-20701.4.patch, HIVE-20701.5.patch,
> HIVE-20701.6.patch, HIVE-20701.7.patch
>
>
> Following up with HIVE-20538 it'd be nice to be able to use this feature with
> hive streaming
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20762) NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
[ https://issues.apache.org/jira/browse/HIVE-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-20762: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) 02.patch committed to master. Thanks [~thejas] for the review! > NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. > --- > > Key: HIVE-20762 > URL: https://issues.apache.org/jira/browse/HIVE-20762 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Metastore >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Minor > Labels: Metastore > Fix For: 4.0.0 > > Attachments: HIVE-20762.01.patch, HIVE-20762.02.patch > > > NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. It > should be set to several hours or else the number of metastore calls would be > too high and impact other operations. > Make it configurable item and set it as 2Hrs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20701) Allow HiveStreaming to receive a key value to commit atomically together with the transaction
[
https://issues.apache.org/jira/browse/HIVE-20701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657696#comment-16657696
]
Hive QA commented on HIVE-20701:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14581/dev-support/hive-personality.sh
|
| git revision | master / 4a7de47 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: streaming U: streaming |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14581/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Allow HiveStreaming to receive a key value to commit atomically together with
> the transaction
> -
>
> Key: HIVE-20701
> URL: https://issues.apache.org/jira/browse/HIVE-20701
> Project: Hive
> Issue Type: Improvement
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Major
> Attachments: HIVE-20701.1.patch, HIVE-20701.2.patch,
> HIVE-20701.3.patch, HIVE-20701.4.patch, HIVE-20701.5.patch,
> HIVE-20701.6.patch, HIVE-20701.7.patch
>
>
> Following up with HIVE-20538 it'd be nice to be able to use this feature with
> hive streaming
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20708) Load (dumped) an external table as an external table on target with the same location as on the source
[
https://issues.apache.org/jira/browse/HIVE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657692#comment-16657692
]
Hive QA commented on HIVE-20708:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944771/HIVE-20708.02
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15111 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14580/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14580/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14580/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944771 - PreCommit-HIVE-Build
> Load (dumped) an external table as an external table on target with the same
> location as on the source
> --
>
> Key: HIVE-20708
> URL: https://issues.apache.org/jira/browse/HIVE-20708
> Project: Hive
> Issue Type: Improvement
> Components: repl
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20708.01, HIVE-20708.02
>
>
> External tables are currently mapped to managed tables on target. A lot of
> jobs in user environment are dependent upon locations specified in external
> table definitions to run, hence, the path for external tables on the target
> and on the source are expected to be the same. An external table being loaded
> as a managed table makes it difficult for failover (Controlled Failover) /
> failback since there is no option of moving data from managed to external
> table. So the external table replicated to target cluster needs to be kept as
> external table with same location as on the source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20708) Load (dumped) an external table as an external table on target with the same location as on the source
[
https://issues.apache.org/jira/browse/HIVE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657682#comment-16657682
]
Hive QA commented on HIVE-20708:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
49s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
44s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14580/dev-support/hive-personality.sh
|
| git revision | master / 4a7de47 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14580/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Load (dumped) an external table as an external table on target with the same
> location as on the source
> --
>
> Key: HIVE-20708
> URL: https://issues.apache.org/jira/browse/HIVE-20708
> Project: Hive
> Issue Type: Improvement
> Components: repl
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20708.01, HIVE-20708.02
>
>
> External tables are currently mapped to managed tables on target. A lot of
> jobs in user environment are dependent upon locations specified in external
> table definitions to run, hence, the path for external tables on the target
> and on the source are expected to be the same. An external table being loaded
> as a managed table makes it difficult for failover (Controlled Failover) /
> failback since there is no option of moving data from managed to external
> table. So the external table replicated to target cluster needs to be kept as
> external table with same location as on the source.
--
This message
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657680#comment-16657680 ] Vihang Karajgaonkar commented on HIVE-20765: BTW, since I already took a look at the patch :) Can you please remove the commented out line for getPartitionsByNames call. Rest looks good. +1 > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.02.patch, HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20542) Incremental REPL DUMP progress information log message is incorrect.
[
https://issues.apache.org/jira/browse/HIVE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657642#comment-16657642
]
Hive QA commented on HIVE-20542:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944753/HIVE-20542.03
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15106 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=195)
[druidmini_dynamic_partition.q,druidmini_test_ts.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test_insert.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14579/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14579/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14579/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944753 - PreCommit-HIVE-Build
> Incremental REPL DUMP progress information log message is incorrect.
>
>
> Key: HIVE-20542
> URL: https://issues.apache.org/jira/browse/HIVE-20542
> Project: Hive
> Issue Type: Bug
> Components: repl
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Sankar Hariappan
>Assignee: Ashutosh Bapat
>Priority: Minor
> Labels: DR, Replication, pull-request-available
> Attachments: HIVE-20542.01.patch, HIVE-20542.02, HIVE-20542.03
>
>
> Incremental REPL DUMP have the progress information logged as
> "eventsDumpProgress":"49/0".
> It should actually log the estimated number of events are denominator but it
> is coming as 0 always.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20542) Incremental REPL DUMP progress information log message is incorrect.
[
https://issues.apache.org/jira/browse/HIVE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657641#comment-16657641
]
Hive QA commented on HIVE-20542:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
7s{color} | {color:blue} standalone-metastore/metastore-common in master has 28
extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} hcatalog/server-extensions in master has 1 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
50s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
44s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
17s{color} | {color:blue} standalone-metastore/metastore-server in master has
182 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
48s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
5s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/hcatalog-unit: The patch generated 6 new + 16
unchanged - 0 fixed = 22 total (was 16) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} itests/hive-unit: The patch generated 2 new + 74
unchanged - 0 fixed = 76 total (was 74) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
28s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 182 unchanged - 0 fixed = 183 total (was 182) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| |
org.apache.hadoop.hive.metastore.ObjectStore.getNotificationEventsCount(NotificationEventsCountRequest)
invokes inefficient new Long(long) constructor; use Long.valueOf(long) instead
At ObjectStore.java:constructor; use Long.valueOf(long) instead At
ObjectStore.java:[line 10270] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14579/dev-support/hive-personality.sh
|
| git revision | master / 4a7de47 |
|
[jira] [Updated] (HIVE-20718) Add perf cli driver with constraints
[ https://issues.apache.org/jira/browse/HIVE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20718: --- Attachment: HIVE-20718.10.patch > Add perf cli driver with constraints > > > Key: HIVE-20718 > URL: https://issues.apache.org/jira/browse/HIVE-20718 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20718.01.patch, HIVE-20718.02.patch, > HIVE-20718.03.patch, HIVE-20718.04.patch, HIVE-20718.05.patch, > HIVE-20718.06.patch, HIVE-20718.07.patch, HIVE-20718.08.patch, > HIVE-20718.09.patch, HIVE-20718.10.patch, HIVE-20718.patch > > > Now that subtasks in HIVE-17039 will be completed, it will be good to have a > perf cli driver with constraints declaration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20784) Migrate hbase.util.Base64 to java.util.Base64
[ https://issues.apache.org/jira/browse/HIVE-20784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657630#comment-16657630 ] ASF GitHub Bot commented on HIVE-20784: --- GitHub user functicons opened a pull request: https://github.com/apache/hive/pull/451 HIVE-20784 Migrate hbase.util.Base64 to java.util.Base64 You can merge this pull request into a Git repository by running: $ git pull https://github.com/functicons/hive branch-3.1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/451.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #451 commit bee498d0920f7e42c38e9b33f8c427a5b6118859 Author: Dagang Wei Date: 2018-10-20T01:01:00Z HIVE-20784 Migrate hbase.util.Base64 to java.util.Base64 > Migrate hbase.util.Base64 to java.util.Base64 > - > > Key: HIVE-20784 > URL: https://issues.apache.org/jira/browse/HIVE-20784 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Affects Versions: 3.1.0 > Environment: HBase 2.0.2 > Hive 3.1.0 >Reporter: Dagang Wei >Assignee: Dagang Wei >Priority: Critical > Labels: pull-request-available > > By default Hive 3.1.0 depends on HBase 2.0.0-alpha4. HBase 2.0.2 migrated > from hbase.util.Base64 to java.util.Base64 (HBASE-20884), which causes Hive > 3.1.0 fails to build with HBase 2.0.2. > > $ cd hbase-handler > $ mvn package -DskipTests -Dhbase.version=2.0.2 > [ERROR] > .../hive/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableSnapshotInputFormat.java:[29,36] > cannot find symbol > [ERROR] symbol: class Base64 > [ERROR] location: package org.apache.hadoop.hbase.util > > To make Hive works with 2.0.2+ (and also older versions), we should consider > migrating Hive to java.util.Base64. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20784) Migrate hbase.util.Base64 to java.util.Base64
[ https://issues.apache.org/jira/browse/HIVE-20784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-20784: -- Labels: pull-request-available (was: ) > Migrate hbase.util.Base64 to java.util.Base64 > - > > Key: HIVE-20784 > URL: https://issues.apache.org/jira/browse/HIVE-20784 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Affects Versions: 3.1.0 > Environment: HBase 2.0.2 > Hive 3.1.0 >Reporter: Dagang Wei >Assignee: Dagang Wei >Priority: Critical > Labels: pull-request-available > > By default Hive 3.1.0 depends on HBase 2.0.0-alpha4. HBase 2.0.2 migrated > from hbase.util.Base64 to java.util.Base64 (HBASE-20884), which causes Hive > 3.1.0 fails to build with HBase 2.0.2. > > $ cd hbase-handler > $ mvn package -DskipTests -Dhbase.version=2.0.2 > [ERROR] > .../hive/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableSnapshotInputFormat.java:[29,36] > cannot find symbol > [ERROR] symbol: class Base64 > [ERROR] location: package org.apache.hadoop.hbase.util > > To make Hive works with 2.0.2+ (and also older versions), we should consider > migrating Hive to java.util.Base64. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20784) Migrate hbase.util.Base64 to java.util.Base64
[ https://issues.apache.org/jira/browse/HIVE-20784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dagang Wei reassigned HIVE-20784: - > Migrate hbase.util.Base64 to java.util.Base64 > - > > Key: HIVE-20784 > URL: https://issues.apache.org/jira/browse/HIVE-20784 > Project: Hive > Issue Type: Improvement > Components: HBase Handler >Affects Versions: 3.1.0 > Environment: HBase 2.0.2 > Hive 3.1.0 >Reporter: Dagang Wei >Assignee: Dagang Wei >Priority: Critical > > By default Hive 3.1.0 depends on HBase 2.0.0-alpha4. HBase 2.0.2 migrated > from hbase.util.Base64 to java.util.Base64 (HBASE-20884), which causes Hive > 3.1.0 fails to build with HBase 2.0.2. > > $ cd hbase-handler > $ mvn package -DskipTests -Dhbase.version=2.0.2 > [ERROR] > .../hive/hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableSnapshotInputFormat.java:[29,36] > cannot find symbol > [ERROR] symbol: class Base64 > [ERROR] location: package org.apache.hadoop.hbase.util > > To make Hive works with 2.0.2+ (and also older versions), we should consider > migrating Hive to java.util.Base64. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[ https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20767: --- Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Pushed to master. Cc [~vgarg] > Multiple project between join operators may affect join reordering using > constraints > > > Key: HIVE-20767 > URL: https://issues.apache.org/jira/browse/HIVE-20767 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20767.01.patch, HIVE-20767.02.patch, > HIVE-20767.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[
https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657599#comment-16657599
]
Hive QA commented on HIVE-20767:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944747/HIVE-20767.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15111 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14578/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14578/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14578/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944747 - PreCommit-HIVE-Build
> Multiple project between join operators may affect join reordering using
> constraints
>
>
> Key: HIVE-20767
> URL: https://issues.apache.org/jira/browse/HIVE-20767
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20767.01.patch, HIVE-20767.02.patch,
> HIVE-20767.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20617) Fix type of constants in IN expressions to have correct type
[
https://issues.apache.org/jira/browse/HIVE-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657593#comment-16657593
]
Ashutosh Chauhan commented on HIVE-20617:
-
[~kgyrtkirk] Can you please update RB ?
> Fix type of constants in IN expressions to have correct type
>
>
> Key: HIVE-20617
> URL: https://issues.apache.org/jira/browse/HIVE-20617
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20617.01.patch, HIVE-20617.02.patch,
> HIVE-20617.03.patch, HIVE-20617.05.patch, HIVE-20617.06.patch,
> HIVE-20617.07.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.09.patch
>
>
> In statements like {{struct(a,b) IN (const struct('x','y'), ... )}} the
> comparision in UDFIn may fail because if a or b is of char/varchar type the
> constants will retain string type - especially after PointlookupOptimizer
> compaction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[
https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657589#comment-16657589
]
Vihang Karajgaonkar commented on HIVE-20765:
Yeah, the other tests which exercise this code path are all in
{{standalone-metastore}} which doesn't have the javax.jdo dependency I guess.
Its good that this was exposed and we fixed it. I grepped for {{try (Query}}
and couldn't find other instances. So I think we are good now. Thanks for
fixing this!
> fetch partitions for txn stats validation in get_aggr_stats with one call
> -
>
> Key: HIVE-20765
> URL: https://issues.apache.org/jira/browse/HIVE-20765
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch,
> HIVE-20765.02.patch, HIVE-20765.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657586#comment-16657586 ] Sergey Shelukhin commented on HIVE-20765: - Well we use DN, so presumably the DN's version comes as a package deal. However I'm not sure how it manages to use one version of the interface here and another there (since other tests hitting this code path pass)... might be something messed up with the classpath for this test. Or it might be that the rest of Hive works due to luck because jars are always in the correct order/setup, and so we never run into this problem, and this test exposes it :) > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.02.patch, HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657583#comment-16657583 ] Vihang Karajgaonkar edited comment on HIVE-20765 at 10/20/18 12:02 AM: --- Do you know what is difference between jdo-api and datanucleus-jdo? Why do we use both? was (Author: vihangk1): Do you know what is difference between jdo-api and datanucleus-jdo.version? Why do we use both? > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.02.patch, HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657583#comment-16657583 ] Vihang Karajgaonkar commented on HIVE-20765: Do you know what is difference between jdo-api and datanucleus-jdo.version? Why do we use both? > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.02.patch, HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657580#comment-16657580 ] Sergey Shelukhin edited comment on HIVE-20765 at 10/20/18 12:00 AM: According to Oracle docs JDO2 Query does not implement AutoCloseable: https://docs.oracle.com/cd/E24329_01/apirefs.1211/e24397/index.html?javax/jdo/Query.html This is another example for JDO3... https://db.apache.org/jdo/api30/apidocs/javax/jdo/Query.html However, the one we use during compile appears to come from DN javax.jdo 3.2 (?!!), that does (org/datanucleus/javax.jdo/3.2.0-m3/javax.jdo-3.2.0-m3.jar I've no idea why DN uses the same package name or what 3.2 is... seems like that's what we are running into. was (Author: sershe): According to Oracle docs JDO2 Query does not implement AutoCloseable: https://docs.oracle.com/cd/E24329_01/apirefs.1211/e24397/index.html?javax/jdo/Query.html This is another example for JDO3... https://db.apache.org/jdo/api30/apidocs/javax/jdo/Query.html However, the one we use during compile appears to come from DN javax.jdo, that does. I've no idea why DN uses the same package name... seems like that's what we are running into. > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.02.patch, HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657580#comment-16657580 ] Sergey Shelukhin commented on HIVE-20765: - According to Oracle docs JDO2 Query does not implement AutoCloseable: https://docs.oracle.com/cd/E24329_01/apirefs.1211/e24397/index.html?javax/jdo/Query.html This is another example for JDO3... https://db.apache.org/jdo/api30/apidocs/javax/jdo/Query.html However, the one we use during compile appears to come from DN javax.jdo, that does. I've no idea why DN uses the same package name... seems like that's what we are running into. > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.02.patch, HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[
https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657578#comment-16657578
]
Vihang Karajgaonkar commented on HIVE-20765:
I could repro it too. It looks like it loads \{{Query}} from
{{jdo-api-3.0.1.jar}} which is one of the dependency from the root pom.xml
3.0.1
{noformat}
javax.jdo
jdo-api
${jdo-api.version}
{noformat}
> fetch partitions for txn stats validation in get_aggr_stats with one call
> -
>
> Key: HIVE-20765
> URL: https://issues.apache.org/jira/browse/HIVE-20765
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch,
> HIVE-20765.02.patch, HIVE-20765.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[
https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657573#comment-16657573
]
Hive QA commented on HIVE-20767:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
41s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14578/dev-support/hive-personality.sh
|
| git revision | master / 14c72c6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14578/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Multiple project between join operators may affect join reordering using
> constraints
>
>
> Key: HIVE-20767
> URL: https://issues.apache.org/jira/browse/HIVE-20767
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20767.01.patch, HIVE-20767.02.patch,
> HIVE-20767.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657557#comment-16657557 ] Sergey Shelukhin edited comment on HIVE-20765 at 10/19/18 11:42 PM: Replacing with closeAll fixes this issue. It should still work with close... this can be investigated later ;) Seems like other codepaths using this call closeAll. was (Author: sershe): Replacing with closeAll fixes this issue > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.02.patch, HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-20765: Attachment: HIVE-20765.02.patch > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.02.patch, HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20783) Factor reduction in cost by DPP when costing SJ reduction
[ https://issues.apache.org/jira/browse/HIVE-20783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-20783: -- > Factor reduction in cost by DPP when costing SJ reduction > - > > Key: HIVE-20783 > URL: https://issues.apache.org/jira/browse/HIVE-20783 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > > Currently, while costing the SJ in a plan, the stats of the a TS that is > reduced by a SJ are not adjusted after we have decided to keep a SJ in the > tree. Ideally, we could adjust the stats to take into account decisions that > have already been made. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657557#comment-16657557 ] Sergey Shelukhin commented on HIVE-20765: - Replacing with closeAll fixes this issue > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657553#comment-16657553 ] Sergey Shelukhin commented on HIVE-20765: - Apply this patch and run the test (TestTxnCommands2.testMultiInsertStatement). Yeah it's really bizarre, I tried replacing auto-close with an explicit call and get the same result. Not sure how that is possible... picking up a wrong jar somewhere in this test? I'd assume other tests call this method fine. > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20748) Disable materialized view rewriting when plan pattern is not allowed
[ https://issues.apache.org/jira/browse/HIVE-20748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20748: --- Attachment: HIVE-20748.02.patch > Disable materialized view rewriting when plan pattern is not allowed > > > Key: HIVE-20748 > URL: https://issues.apache.org/jira/browse/HIVE-20748 > Project: Hive > Issue Type: Bug > Components: Materialized views >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20748.01.patch, HIVE-20748.01.patch, > HIVE-20748.02.patch, HIVE-20748.patch > > > For instance, currently rewriting algorithm does not support some operators. > Or we cannot have non-deterministic function in the MV definition. In those > cases, we should fail either when we try to create the MV with rewriting > enabled, or when when we enable the rewriting for a MV already created. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20768) Adding Tumbling Window UDF
[
https://issues.apache.org/jira/browse/HIVE-20768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-20768:
--
Attachment: HIVE-20768.3.patch
> Adding Tumbling Window UDF
> --
>
> Key: HIVE-20768
> URL: https://issues.apache.org/jira/browse/HIVE-20768
> Project: Hive
> Issue Type: New Feature
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20768.2.patch, HIVE-20768.3.patch,
> HIVE-20768.patch, HIVE-20768.patch
>
>
> Goal is to provide a UDF that truncates a timestamp to a beginning of a
> tumbling window interval.
> {code}
> /**
> * Tumbling windows are a series of fixed-sized, non-overlapping and
> contiguous time intervals.
> * Tumbling windows are inclusive start exclusive end.
> * By default the beginning instant of fist window is Epoch 0 Thu Jan 01
> 00:00:00 1970 UTC.
> * Optionally users may provide a different origin as a timestamp arg3.
> *
> * This an example of series of window with an interval of 5 seconds and
> origin Epoch 0 Thu Jan 01 00:00:00 1970 UTC:
> *
> *
> * interval 1 interval 2interval 3
> * Jan 01 00:00:00 Jan 01 00:00:05 Jan 01 00:00:10
> * 0 -- 4 : 5 --- 9: 10 --- 14
> *
> * This UDF rounds timestamp agr1 to the beginning of window interval where
> it belongs to.
> *
> */
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20768) Adding Tumbling Window UDF
[
https://issues.apache.org/jira/browse/HIVE-20768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657546#comment-16657546
]
Hive QA commented on HIVE-20768:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944739/HIVE-20768.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14577/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14577/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14577/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-10-19 23:08:50.334
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14577/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-10-19 23:08:50.339
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 14c72c6 HIVE-20477: OptimizedSql is not shown if the expression
contains INs (Zoltan Haindrich reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-client/
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 14c72c6 HIVE-20477: OptimizedSql is not shown if the expression
contains INs (Zoltan Haindrich reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-10-19 23:08:51.586
+ rm -rf ../yetus_PreCommit-HIVE-Build-14577
+ mkdir ../yetus_PreCommit-HIVE-Build-14577
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14577
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14577/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: cannot apply binary patch to 'data/files/test.jceks' without full index
line
error: data/files/test.jceks: patch does not apply
fatal: git diff header lacks filename information when removing 1 leading
pathname component (line 727)
fatal: git diff header lacks filename information when removing 2 leading
pathname components (line 80)
The patch does not appear to apply with p0, p1, or p2
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-14577
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944739 - PreCommit-HIVE-Build
> Adding Tumbling Window UDF
> --
>
> Key: HIVE-20768
> URL: https://issues.apache.org/jira/browse/HIVE-20768
> Project: Hive
> Issue Type: New Feature
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20768.2.patch, HIVE-20768.patch, HIVE-20768.patch
>
>
> Goal is to provide a UDF that truncates a timestamp to a beginning of a
> tumbling window interval.
> {code}
> /**
> * Tumbling windows are a series of fixed-sized, non-overlapping and
> contiguous time intervals.
> * Tumbling windows are inclusive start exclusive end.
> * By default the beginning instant of fist window is Epoch 0 Thu Jan 01
> 00:00:00 1970 UTC.
> * Optionally users may provide a different origin as a timestamp arg3.
> *
> * This an example of series of window with an interval of 5 seconds and
> origin Epoch 0 Thu Jan 01 00:00:00 1970 UTC:
> *
> *
> * interval 1 interval 2interval 3
> * Jan 01 00:00:00 Jan 01 00:00:05 Jan 01 00:00:10
> * 0 -- 4 : 5 --- 9: 10 --- 14
> *
> * This UDF rounds timestamp agr1 to the beginning of window interval where
[jira] [Commented] (HIVE-20189) Separate metastore client code into its own module
[
https://issues.apache.org/jira/browse/HIVE-20189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657544#comment-16657544
]
Hive QA commented on HIVE-20189:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944709/HIVE-20189.06.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15111 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14575/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14575/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14575/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944709 - PreCommit-HIVE-Build
> Separate metastore client code into its own module
> --
>
> Key: HIVE-20189
> URL: https://issues.apache.org/jira/browse/HIVE-20189
> Project: Hive
> Issue Type: Sub-task
> Components: Standalone Metastore
>Affects Versions: 4.0.0, 3.2.0
>Reporter: Alexander Kolbasov
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-20189.01.patch, HIVE-20189.03.patch,
> HIVE-20189.04.patch, HIVE-20189.05.patch, HIVE-20189.06.patch
>
>
> The goal of this JIRA is to split HiveMetastoreClient code out of
> metastore-common. This is a pom-only change that does not require any changes
> in the code.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657534#comment-16657534 ] Vihang Karajgaonkar commented on HIVE-20765: hmm .. NoSuchMethodError is very weird. I haven't seen this. How can I reproduce? > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657534#comment-16657534 ] Vihang Karajgaonkar edited comment on HIVE-20765 at 10/19/18 11:00 PM: --- hmm .. NoSuchMethodError is very weird. I haven't seen this. How can I reproduce this error? was (Author: vihangk1): hmm .. NoSuchMethodError is very weird. I haven't seen this. How can I reproduce? > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20782) Cleaning some unused code
[
https://issues.apache.org/jira/browse/HIVE-20782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-20782:
--
Description:
Am making my way into the vectorize code and trying understand the APIs. Ran
into this unused one, i guess it is not used anymore.
[~ashutoshc] maybe can explain as you are the main contributor to this file
{code}
a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java{code} ?
was:
Am making my way into the vectorize code and trying understand the APIs. Ran
into this unused one, i guess it is not used anymore.
[~ashutoshc] maybe can explain as you are the main contributor to this file
\{code}
a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java\{code}?
> Cleaning some unused code
> -
>
> Key: HIVE-20782
> URL: https://issues.apache.org/jira/browse/HIVE-20782
> Project: Hive
> Issue Type: Improvement
>Reporter: slim bouguerra
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-20782.patch
>
>
> Am making my way into the vectorize code and trying understand the APIs. Ran
> into this unused one, i guess it is not used anymore.
> [~ashutoshc] maybe can explain as you are the main contributor to this file
> {code}
> a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java{code}
> ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[
https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657533#comment-16657533
]
Sergey Shelukhin commented on HIVE-20765:
-
Failure is related, but weird... {noformat}
2018-10-19T15:52:41,324 ERROR [main] metastore.RetryingHMSHandler:
java.lang.NoSuchMethodError: javax.jdo.Query.close()V
at
org.apache.hadoop.hive.metastore.PartitionProjectionEvaluator.setSingleValuedFields(PartitionProjectionEvaluator.java:456)
at
org.apache.hadoop.hive.metastore.PartitionProjectionEvaluator.getPartitionsUsingProjectionList(PartitionProjectionEvaluator.java:358)
at
org.apache.hadoop.hive.metastore.MetaStoreDirectSql$3.run(MetaStoreDirectSql.java:635)
at
org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:73)
at
org.apache.hadoop.hive.metastore.MetaStoreDirectSql.getPartitionsUsingProjectionAndFilterSpec(MetaStoreDirectSql.java:632)
at
org.apache.hadoop.hive.metastore.ObjectStore$13.getSqlResult(ObjectStore.java:4115)
at
org.apache.hadoop.hive.metastore.ObjectStore$13.getSqlResult(ObjectStore.java:4080)
at
org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3759)
at
org.apache.hadoop.hive.metastore.ObjectStore.getPartitionSpecsByFilterAndProjection(ObjectStore.java:4165)
at
org.apache.hadoop.hive.metastore.ObjectStore.get_aggr_stats_for(ObjectStore.java:9066)
{noformat}
[~vihangk1] have you ever seen this trying to use the new API?
I will look into this a little bit and might just switch to fetching partitions
by names the old way for expediency.
> fetch partitions for txn stats validation in get_aggr_stats with one call
> -
>
> Key: HIVE-20765
> URL: https://issues.apache.org/jira/browse/HIVE-20765
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch,
> HIVE-20765.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20782) Cleaning some unused code
[
https://issues.apache.org/jira/browse/HIVE-20782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-20782:
--
Description:
Am making my way into the vectorize code and trying understand the APIs. Ran
into this unused one, i guess it is not used anymore.
[~ashutoshc] maybe can explain as you are the main contributor to this file
\{code}
a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java\{code}?
was:
Am making my way into the vectorize code and trying understand the APIs. Ran
into this unused one, i guess it is not used anymore.
[~ashutoshc] maybe can explain as you are the main contributor to this file ?
> Cleaning some unused code
> -
>
> Key: HIVE-20782
> URL: https://issues.apache.org/jira/browse/HIVE-20782
> Project: Hive
> Issue Type: Improvement
>Reporter: slim bouguerra
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-20782.patch
>
>
> Am making my way into the vectorize code and trying understand the APIs. Ran
> into this unused one, i guess it is not used anymore.
> [~ashutoshc] maybe can explain as you are the main contributor to this file
> \{code}
> a/ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java\{code}?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20782) Cleaning some unused code
[ https://issues.apache.org/jira/browse/HIVE-20782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20782: -- Attachment: HIVE-20782.patch > Cleaning some unused code > - > > Key: HIVE-20782 > URL: https://issues.apache.org/jira/browse/HIVE-20782 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-20782.patch > > > Am making my way into the vectorize code and trying understand the APIs. Ran > into this unused one, i guess it is not used anymore. > [~ashutoshc] maybe can explain as you are the main contributor to this file ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20782) Cleaning some unused code
[ https://issues.apache.org/jira/browse/HIVE-20782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra reassigned HIVE-20782: - > Cleaning some unused code > - > > Key: HIVE-20782 > URL: https://issues.apache.org/jira/browse/HIVE-20782 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: Teddy Choi >Priority: Major > > Am making my way into the vectorize code and trying understand the APIs. Ran > into this unused one, i guess it is not used anymore. > [~ashutoshc] maybe can explain as you are the main contributor to this file ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20782) Cleaning some unused code
[ https://issues.apache.org/jira/browse/HIVE-20782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20782: -- Status: Patch Available (was: Open) > Cleaning some unused code > - > > Key: HIVE-20782 > URL: https://issues.apache.org/jira/browse/HIVE-20782 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: Teddy Choi >Priority: Major > > Am making my way into the vectorize code and trying understand the APIs. Ran > into this unused one, i guess it is not used anymore. > [~ashutoshc] maybe can explain as you are the main contributor to this file ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20189) Separate metastore client code into its own module
[
https://issues.apache.org/jira/browse/HIVE-20189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657531#comment-16657531
]
Hive QA commented on HIVE-20189:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
55s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
4s{color} | {color:blue} standalone-metastore/metastore-common in master has 28
extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} llap-server in master has 83 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
16s{color} | {color:blue} standalone-metastore/metastore-server in master has
182 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch standalone-metastore passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} standalone-metastore/metastore-client: The patch
generated 2180 new + 0 unchanged - 0 fixed = 2180 total (was 0) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch metastore passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} llap-server: The patch generated 0 new + 8 unchanged
- 1 fixed = 8 total (was 9) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
23s{color} | {color:red} standalone-metastore/metastore-common generated 4 new
+ 24 unchanged - 4 fixed = 28 total (was 28) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
23s{color} | {color:red} standalone-metastore generated 1 new + 66 unchanged -
1 fixed = 67 total (was 67) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} standalone-metastore_metastore-common generated 0
new + 16 unchanged - 1 fixed = 16 total (was 17) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
12s{color} | {color:red} standalone-metastore_metastore-client generated 1 new
+ 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} metastore in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} llap-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green}
[jira] [Commented] (HIVE-20661) Dynamic partitions loading calls add partition for every partition 1-by-1
[
https://issues.apache.org/jira/browse/HIVE-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657491#comment-16657491
]
Hive QA commented on HIVE-20661:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944677/HIVE-20661.11.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15106 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=195)
[druidmini_dynamic_partition.q,druidmini_test_ts.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test_insert.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14574/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14574/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14574/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944677 - PreCommit-HIVE-Build
> Dynamic partitions loading calls add partition for every partition 1-by-1
> -
>
> Key: HIVE-20661
> URL: https://issues.apache.org/jira/browse/HIVE-20661
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20661.01.patch, HIVE-20661.02.patch,
> HIVE-20661.03.patch, HIVE-20661.04.patch, HIVE-20661.05.patch,
> HIVE-20661.06.patch, HIVE-20661.07.patch, HIVE-20661.08.patch,
> HIVE-20661.09.patch, HIVE-20661.10.patch, HIVE-20661.11.patch
>
>
> Hive.loadDynamicPartitions creates partitions using a threadpool, but the
> update of the Metastore via the MetastoreClient is done 1-by-1. This adds
> unnecessary extra calls. The partitions should be created in one batch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20661) Dynamic partitions loading calls add partition for every partition 1-by-1
[
https://issues.apache.org/jira/browse/HIVE-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657473#comment-16657473
]
Hive QA commented on HIVE-20661:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
10s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
51s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
48s{color} | {color:red} ql: The patch generated 4 new + 242 unchanged - 3
fixed = 246 total (was 245) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 38m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14574/dev-support/hive-personality.sh
|
| git revision | master / 14c72c6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14574/yetus/diff-checkstyle-ql.txt
|
| modules | C: standalone-metastore/metastore-common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14574/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Dynamic partitions loading calls add partition for every partition 1-by-1
> -
>
> Key: HIVE-20661
> URL: https://issues.apache.org/jira/browse/HIVE-20661
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20661.01.patch, HIVE-20661.02.patch,
> HIVE-20661.03.patch, HIVE-20661.04.patch, HIVE-20661.05.patch,
> HIVE-20661.06.patch, HIVE-20661.07.patch, HIVE-20661.08.patch,
> HIVE-20661.09.patch, HIVE-20661.10.patch, HIVE-20661.11.patch
>
>
> Hive.loadDynamicPartitions creates partitions using a threadpool, but the
> update of the Metastore via the MetastoreClient is done 1-by-1. This adds
> unnecessary extra calls. The partitions should be created in one batch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20772) record per-task CPU counters in LLAP
[ https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657446#comment-16657446 ] Sergey Shelukhin commented on HIVE-20772: - Fixed an issue where the wrapper callable from the IO is not being handled correctly by Stats thread pool. > record per-task CPU counters in LLAP > > > Key: HIVE-20772 > URL: https://issues.apache.org/jira/browse/HIVE-20772 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20772.01.patch, HIVE-20772.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20772) record per-task CPU counters in LLAP
[ https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-20772: Attachment: HIVE-20772.01.patch > record per-task CPU counters in LLAP > > > Key: HIVE-20772 > URL: https://issues.apache.org/jira/browse/HIVE-20772 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20772.01.patch, HIVE-20772.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20740) Remove global lock in ObjectStore.setConf method
[
https://issues.apache.org/jira/browse/HIVE-20740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-20740:
---
Attachment: HIVE-20740.04.patch
> Remove global lock in ObjectStore.setConf method
>
>
> Key: HIVE-20740
> URL: https://issues.apache.org/jira/browse/HIVE-20740
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-20740.01.patch, HIVE-20740.02.patch,
> HIVE-20740.04.patch
>
>
> The ObjectStore#setConf method has a global lock which can block other
> clients in concurrent workloads.
> {code}
> @Override
> @SuppressWarnings("nls")
> public void setConf(Configuration conf) {
> // Although an instance of ObjectStore is accessed by one thread, there
> may
> // be many threads with ObjectStore instances. So the static variables
> // pmf and prop need to be protected with locks.
> pmfPropLock.lock();
> try {
> isInitialized = false;
> this.conf = conf;
> this.areTxnStatsSupported = MetastoreConf.getBoolVar(conf,
> ConfVars.HIVE_TXN_STATS_ENABLED);
> configureSSL(conf);
> Properties propsFromConf = getDataSourceProps(conf);
> boolean propsChanged = !propsFromConf.equals(prop);
> if (propsChanged) {
> if (pmf != null){
> clearOutPmfClassLoaderCache(pmf);
> if (!forTwoMetastoreTesting) {
> // close the underlying connection pool to avoid leaks
> pmf.close();
> }
> }
> pmf = null;
> prop = null;
> }
> assert(!isActiveTransaction());
> shutdown();
> // Always want to re-create pm as we don't know if it were created by
> the
> // most recent instance of the pmf
> pm = null;
> directSql = null;
> expressionProxy = null;
> openTrasactionCalls = 0;
> currentTransaction = null;
> transactionStatus = TXN_STATUS.NO_STATE;
> initialize(propsFromConf);
> String partitionValidationRegex =
> MetastoreConf.getVar(this.conf,
> ConfVars.PARTITION_NAME_WHITELIST_PATTERN);
> if (partitionValidationRegex != null &&
> !partitionValidationRegex.isEmpty()) {
> partitionValidationPattern =
> Pattern.compile(partitionValidationRegex);
> } else {
> partitionValidationPattern = null;
> }
> // Note, if metrics have not been initialized this will return null,
> which means we aren't
> // using metrics. Thus we should always check whether this is non-null
> before using.
> MetricRegistry registry = Metrics.getRegistry();
> if (registry != null) {
> directSqlErrors =
> Metrics.getOrCreateCounter(MetricsConstants.DIRECTSQL_ERRORS);
> }
> this.batchSize = MetastoreConf.getIntVar(conf,
> ConfVars.RAWSTORE_PARTITION_BATCH_SIZE);
> if (!isInitialized) {
> throw new RuntimeException(
> "Unable to create persistence manager. Check dss.log for details");
> } else {
> LOG.debug("Initialized ObjectStore");
> }
> } finally {
> pmfPropLock.unlock();
> }
> }
> {code}
> The {{pmfPropLock}} is a static object and it disallows any other new
> connection to HMS which is trying to instantiate ObjectStore. We should
> either remove the lock or reduce the scope of the lock so that it is held for
> a very small amount of time.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20762) NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
[
https://issues.apache.org/jira/browse/HIVE-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657409#comment-16657409
]
Hive QA commented on HIVE-20762:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944666/HIVE-20762.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15111 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14573/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14573/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14573/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944666 - PreCommit-HIVE-Build
> NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
> ---
>
> Key: HIVE-20762
> URL: https://issues.apache.org/jira/browse/HIVE-20762
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2, Metastore
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Minor
> Labels: Metastore
> Attachments: HIVE-20762.01.patch, HIVE-20762.02.patch
>
>
> NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small. It
> should be set to several hours or else the number of metastore calls would be
> too high and impact other operations.
> Make it configurable item and set it as 2Hrs.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20778) Join reordering may not be triggered if all joins in plan are created by decorrelation logic
[ https://issues.apache.org/jira/browse/HIVE-20778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20778: --- Status: Patch Available (was: Open) > Join reordering may not be triggered if all joins in plan are created by > decorrelation logic > > > Key: HIVE-20778 > URL: https://issues.apache.org/jira/browse/HIVE-20778 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20778.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20778) Join reordering may not be triggered if all joins in plan are created by decorrelation logic
[ https://issues.apache.org/jira/browse/HIVE-20778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20778: --- Attachment: HIVE-20778.1.patch > Join reordering may not be triggered if all joins in plan are created by > decorrelation logic > > > Key: HIVE-20778 > URL: https://issues.apache.org/jira/browse/HIVE-20778 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20778.1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18772) Make Acid Cleaner use MIN_HISTORY_LEVEL
[ https://issues.apache.org/jira/browse/HIVE-18772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18772: -- Description: Instead of using Lock Manager state as it currently does. This will eliminate possible race conditions See this [comment|https://issues.apache.org/jira/browse/HIVE-18192?focusedCommentId=16338208=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16338208] Suppose A is the set of all ValidTxnList across all active readers. Each ValidTxnList has minOpenTxnId. MIN_HISTORY_LEVEL allows us to determine X = min(minOpenTxnId) across all currently active readers This means that no active transaction in the system sees any txn with txnid < X as open. This means if construct ValidTxnIdList with HWM=X-1 and use that in getAcidState(), any files determined by this call as 'obsolete', will be seen as obsolete by any existing/future reader, i.e. can be physically deleted. This is also necessary for multi-statement transactions where relying on the state of Lock Manager is not sufficient. For example Suppose txn 17 starts at t1 and sees txnid 13 with writeID 13 open. 13 commits (via it's parent txn) at t2 > t1. (17 is still running). Compaction runs at t3 >t2 to produce base_14 (or delta_10_14 for example) on Table1/Part1 (17 is still running) Now delta_13 may be cleaned since it can be seen as obsolete and there may be no locks on it, i.e. no one is reading it. Now at t4 > t3 17 may (multi stmt txn) needs to read Table1/Part1. It cannot use base_14 is that may have absorbed delete events from delete_delta_14. Another Use Case There is delta_1_1 and delta_2_2 on disk both created by committed txns. T5 starts reading these. At the same time compactor creates delta_1_2. Now Cleaner sees delta_1_1 and delta_1_2 as obsolete and may remove them while the read is still in progress. This is because Compactor itself is not running in a txn and the files that it produces are visible immediately. If it ran in a txn, the new files would only be visible once this txn is visible to others (including the Cleaner). Using MIN_HISTORY_LEVEL solves this. See description of HIVE-18747 for more details on MIN_HISTORY_LEVEL was: Instead of using Lock Manager state as it currently does. This will eliminate possible race conditions See this [comment|https://issues.apache.org/jira/browse/HIVE-18192?focusedCommentId=16338208=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16338208] Suppose A is the set of all ValidTxnList across all active readers. Each ValidTxnList has minOpenTxnId. MIN_HISTORY_LEVEL allows us to determine X = min(minOpenTxnId) across all currently active readers This means that no active transaction in the system sees any txn with txnid < X as open. This means if construct ValidTxnIdList with HWM=X-1 and use that in getAcidState(), any files determined by this call as 'obsolete', will be seen as obsolete by any existing/future reader, i.e. can be physically deleted. This is also necessary for multi-statement transactions where relying on the state of Lock Manager is not sufficient. For example Suppose txn 17 starts at t1 and sees txnid 13 with writeID 13 open. 13 commits (via it's parent txn) at t2 > t1. (17 is still running). Compaction runs at t3 >t2 to produce base_14 (or delta_10_14 for example) on Table1/Part1 (17 is still running) Now delta_13 may be cleaned since it can be seen as obsolete and there may be no locks on it, i.e. no one is reading it. Now at t4 > t3 17 may (multi stmt txn) needs to read Table1/Part1. It cannot use base_14 is that may have absorbed delete events from delete_delta_14. Using MIN_HISTORY_LEVEL solves this. See description of HIVE-18747 for more details on MIN_HISTORY_LEVEL > Make Acid Cleaner use MIN_HISTORY_LEVEL > --- > > Key: HIVE-18772 > URL: https://issues.apache.org/jira/browse/HIVE-18772 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Major > Attachments: HIVE-18772.01.patch, HIVE-18772.02.patch, > HIVE-18772.02.patch, HIVE-18772.03.patch > > > Instead of using Lock Manager state as it currently does. > This will eliminate possible race conditions > See this > [comment|https://issues.apache.org/jira/browse/HIVE-18192?focusedCommentId=16338208=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16338208] > Suppose A is the set of all ValidTxnList across all active readers. Each > ValidTxnList has minOpenTxnId. > MIN_HISTORY_LEVEL allows us to determine X = min(minOpenTxnId) across all > currently active readers > This means that no active transaction in the system sees any txn with txnid <
[jira] [Commented] (HIVE-20781) Don't set type for CASE statements during Rex translation
[ https://issues.apache.org/jira/browse/HIVE-20781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657371#comment-16657371 ] Jesus Camacho Rodriguez commented on HIVE-20781: [~kgyrtkirk], can you provide more context? Is retType incorrect? Calcite and Hive may have different ways of inferring return type for expressions, hence I believe it would be safer to fix retType rather than not specifying it. > Don't set type for CASE statements during Rex translation > - > > Key: HIVE-20781 > URL: https://issues.apache.org/jira/browse/HIVE-20781 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20781.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20781) Don't set type for CASE statements during Rex translation
[ https://issues.apache.org/jira/browse/HIVE-20781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20781: Attachment: HIVE-20781.01.patch > Don't set type for CASE statements during Rex translation > - > > Key: HIVE-20781 > URL: https://issues.apache.org/jira/browse/HIVE-20781 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20781.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20781) Don't set type for CASE statements during Rex translation
[ https://issues.apache.org/jira/browse/HIVE-20781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20781: Status: Patch Available (was: Open) > Don't set type for CASE statements during Rex translation > - > > Key: HIVE-20781 > URL: https://issues.apache.org/jira/browse/HIVE-20781 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-20781.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20781) Don't set type for CASE statements during Rex translation
[ https://issues.apache.org/jira/browse/HIVE-20781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich reassigned HIVE-20781: --- > Don't set type for CASE statements during Rex translation > - > > Key: HIVE-20781 > URL: https://issues.apache.org/jira/browse/HIVE-20781 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20762) NOTIFICATION_LOG cleanup interval is hardcoded as 60s and is too small.
[
https://issues.apache.org/jira/browse/HIVE-20762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657356#comment-16657356
]
Hive QA commented on HIVE-20762:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
29s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
32s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} hcatalog/server-extensions in master has 1 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
19s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
18s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 18s{color}
| {color:red} server-extensions in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} hcatalog/server-extensions: The patch generated 0
new + 4 unchanged - 1 fixed = 4 total (was 5) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} itests/hcatalog-unit: The patch generated 2 new + 16
unchanged - 0 fixed = 18 total (was 16) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} server-extensions in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14573/dev-support/hive-personality.sh
|
| git revision | master / bd3c05d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14573/yetus/patch-mvninstall-hcatalog_server-extensions.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14573/yetus/patch-compile-hcatalog_server-extensions.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14573/yetus/patch-compile-hcatalog_server-extensions.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14573/yetus/diff-checkstyle-itests_hcatalog-unit.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14573/yetus/patch-findbugs-hcatalog_server-extensions.txt
|
| modules | C: standalone-metastore/metastore-common hcatalog/server-extensions
itests/hcatalog-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14573/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
>
[jira] [Updated] (HIVE-20477) OptimizedSql is not shown if the expression contains INs
[ https://issues.apache.org/jira/browse/HIVE-20477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20477: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) pushed to master. Thank you Ashutosh for reviewing the changes! > OptimizedSql is not shown if the expression contains INs > > > Key: HIVE-20477 > URL: https://issues.apache.org/jira/browse/HIVE-20477 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20477.01.patch, HIVE-20477.01.patch, > HIVE-20477.01wip01.patch > > > This ticket should fix HiveIn to be able to unparse; currently if an IN is > unparsed - there are some exceptions because HiveIn is a special operator; > but doesn't have unparse implemented. > CALCITE-2444 is also needed to fix rel2sql to be able to process INs which > are not there represent a subquery. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20617) Fix type of constants in IN expressions to have correct type
[
https://issues.apache.org/jira/browse/HIVE-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-20617:
Attachment: HIVE-20617.09.patch
> Fix type of constants in IN expressions to have correct type
>
>
> Key: HIVE-20617
> URL: https://issues.apache.org/jira/browse/HIVE-20617
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20617.01.patch, HIVE-20617.02.patch,
> HIVE-20617.03.patch, HIVE-20617.05.patch, HIVE-20617.06.patch,
> HIVE-20617.07.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.08.patch, HIVE-20617.08.patch, HIVE-20617.08.patch,
> HIVE-20617.09.patch
>
>
> In statements like {{struct(a,b) IN (const struct('x','y'), ... )}} the
> comparision in UDFIn may fail because if a or b is of char/varchar type the
> constants will retain string type - especially after PointlookupOptimizer
> compaction.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[
https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657312#comment-16657312
]
Hive QA commented on HIVE-20765:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944648/HIVE-20765.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15109 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_wm]
(batchId=72)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=172)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMultiInsertStatement
(batchId=305)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testMultiInsertStatement
(batchId=317)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14572/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14572/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14572/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944648 - PreCommit-HIVE-Build
> fetch partitions for txn stats validation in get_aggr_stats with one call
> -
>
> Key: HIVE-20765
> URL: https://issues.apache.org/jira/browse/HIVE-20765
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch,
> HIVE-20765.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-20765: Attachment: HIVE-20765.01.patch > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[ https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657300#comment-16657300 ] Sergey Shelukhin commented on HIVE-20765: - Some weird typo made it into the patch.. > fetch partitions for txn stats validation in get_aggr_stats with one call > - > > Key: HIVE-20765 > URL: https://issues.apache.org/jira/browse/HIVE-20765 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-20765.01.patch, HIVE-20765.01.patch, > HIVE-20765.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20321) Vectorization: Cut down memory size of 1 col VectorHashKeyWrapper to <1 CacheLine
[
https://issues.apache.org/jira/browse/HIVE-20321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-20321:
Fix Version/s: 4.0.0
> Vectorization: Cut down memory size of 1 col VectorHashKeyWrapper to <1
> CacheLine
> -
>
> Key: HIVE-20321
> URL: https://issues.apache.org/jira/browse/HIVE-20321
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Matt McCline
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20321.01.patch, HIVE-20321.02.patch,
> HIVE-20321.03.patch, HIVE-20321.04.patch, HIVE-20321.05.patch,
> HIVE-20321.06.patch
>
>
> With a full sized LLAP instance, the memory size of the VectorHashKeyWrapper
> is bigger than the low Xmx JVMs.
> {code}
> * 64-bit VM: **
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapper object internals:
> OFFSET SIZE
> TYPE DESCRIPTION VALUE
> 016
> (object header) N/A
> 16 4
> int VectorHashKeyWrapper.hashcodeN/A
> 20 4
> (alignment/padding gap)
> 24 8
> long[] VectorHashKeyWrapper.longValues N/A
> 32 8
> double[] VectorHashKeyWrapper.doubleValuesN/A
> 40 8
> byte[][] VectorHashKeyWrapper.byteValues N/A
> 48 8
>int[] VectorHashKeyWrapper.byteStarts N/A
> 56 8
>int[] VectorHashKeyWrapper.byteLengths N/A
> 64 8
> org.apache.hadoop.hive.serde2.io.HiveDecimalWritable[]
> VectorHashKeyWrapper.decimalValues N/A
> 72 8
> java.sql.Timestamp[] VectorHashKeyWrapper.timestampValues N/A
> 80 8
> org.apache.hadoop.hive.common.type.HiveIntervalDayTime[]
> VectorHashKeyWrapper.intervalDayTimeValues N/A
> 88 8
> boolean[] VectorHashKeyWrapper.isNull N/A
> 96 8
> org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapper.HashContext
> VectorHashKeyWrapper.hashCtx N/A
> Instance size: 104 bytes
> Space losses: 4 bytes internal + 0 bytes external = 4 bytes total
> {code}
> Pulling this up to a parent class allows for this to be cut down to 32 bytes
> for the single column case.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'
[
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mass Dosage updated HIVE-18767:
---
Target Version/s: 3.1.0, 2.3.3, 4.0.0, 3.2.0 (was: 2.3.3, 3.1.0, 4.0.0,
3.2.0)
Status: Patch Available (was: In Progress)
OK, I've gone with the new patch name format and have added
HIVE-18767.3.branch-3.patch.
I tested that this applies cleanly on "branch-3" a few minutes ago and also
successfully run all the tests in the "standalone-metastore" module.
> Some alterPartitions invocations throw 'NumberFormatException: null'
>
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.0, 2.3.3, 4.0.0, 3.2.0
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 3.1.1, 2.3.4
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767-branch-2.patch,
> HIVE-18767-branch-3.1.patch, HIVE-18767-branch-3.patch, HIVE-18767.1.patch,
> HIVE-18767.2-branch-2.3.patch, HIVE-18767.2-branch-2.patch,
> HIVE-18767.2-branch-3.1.patch, HIVE-18767.2.branch-3.patch,
> HIVE-18767.2.patch, HIVE-18767.3-branch-3.1.patch,
> HIVE-18767.3.branch-3.patch, HIVE-18767.3.patch,
> HIVE-18767.4-branch-3.1.patch, HIVE-18767.4.branch-3.1.patch,
> HIVE-18767.4.patch, HIVE-18767.5.patch, HIVE-18767.6.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown
> Source)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'
[
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mass Dosage updated HIVE-18767:
---
Attachment: HIVE-18767.3.branch-3.patch
> Some alterPartitions invocations throw 'NumberFormatException: null'
>
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.3, 3.1.0, 4.0.0, 3.2.0
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 3.1.1, 2.3.4
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767-branch-2.patch,
> HIVE-18767-branch-3.1.patch, HIVE-18767-branch-3.patch, HIVE-18767.1.patch,
> HIVE-18767.2-branch-2.3.patch, HIVE-18767.2-branch-2.patch,
> HIVE-18767.2-branch-3.1.patch, HIVE-18767.2.branch-3.patch,
> HIVE-18767.2.patch, HIVE-18767.3-branch-3.1.patch,
> HIVE-18767.3.branch-3.patch, HIVE-18767.3.patch,
> HIVE-18767.4-branch-3.1.patch, HIVE-18767.4.branch-3.1.patch,
> HIVE-18767.4.patch, HIVE-18767.5.patch, HIVE-18767.6.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown
> Source)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'
[
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mass Dosage updated HIVE-18767:
---
Target Version/s: 3.1.0, 2.3.3, 4.0.0, 3.2.0 (was: 2.3.3, 3.1.0, 4.0.0,
3.2.0)
Status: In Progress (was: Patch Available)
> Some alterPartitions invocations throw 'NumberFormatException: null'
>
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.1.0, 2.3.3, 4.0.0, 3.2.0
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 3.1.1, 2.3.4
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767-branch-2.patch,
> HIVE-18767-branch-3.1.patch, HIVE-18767-branch-3.patch, HIVE-18767.1.patch,
> HIVE-18767.2-branch-2.3.patch, HIVE-18767.2-branch-2.patch,
> HIVE-18767.2-branch-3.1.patch, HIVE-18767.2.branch-3.patch,
> HIVE-18767.2.patch, HIVE-18767.3-branch-3.1.patch, HIVE-18767.3.patch,
> HIVE-18767.4-branch-3.1.patch, HIVE-18767.4.branch-3.1.patch,
> HIVE-18767.4.patch, HIVE-18767.5.patch, HIVE-18767.6.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown
> Source)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'
[
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mass Dosage updated HIVE-18767:
---
Attachment: HIVE-18767.2.branch-3.patch
> Some alterPartitions invocations throw 'NumberFormatException: null'
>
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.3, 3.1.0, 4.0.0, 3.2.0
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 3.1.1, 2.3.4
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767-branch-2.patch,
> HIVE-18767-branch-3.1.patch, HIVE-18767-branch-3.patch, HIVE-18767.1.patch,
> HIVE-18767.2-branch-2.3.patch, HIVE-18767.2-branch-2.patch,
> HIVE-18767.2-branch-3.1.patch, HIVE-18767.2.branch-3.patch,
> HIVE-18767.2.patch, HIVE-18767.3-branch-3.1.patch, HIVE-18767.3.patch,
> HIVE-18767.4-branch-3.1.patch, HIVE-18767.4.branch-3.1.patch,
> HIVE-18767.4.patch, HIVE-18767.5.patch, HIVE-18767.6.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown
> Source)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20718) Add perf cli driver with constraints
[ https://issues.apache.org/jira/browse/HIVE-20718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20718: --- Attachment: HIVE-20718.09.patch > Add perf cli driver with constraints > > > Key: HIVE-20718 > URL: https://issues.apache.org/jira/browse/HIVE-20718 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20718.01.patch, HIVE-20718.02.patch, > HIVE-20718.03.patch, HIVE-20718.04.patch, HIVE-20718.05.patch, > HIVE-20718.06.patch, HIVE-20718.07.patch, HIVE-20718.08.patch, > HIVE-20718.09.patch, HIVE-20718.patch > > > Now that subtasks in HIVE-17039 will be completed, it will be good to have a > perf cli driver with constraints declaration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-20780) Add additional object owner information to ExtendJSON message classes
[ https://issues.apache.org/jira/browse/HIVE-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña resolved HIVE-20780. Resolution: Won't Fix > Add additional object owner information to ExtendJSON message classes > - > > Key: HIVE-20780 > URL: https://issues.apache.org/jira/browse/HIVE-20780 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.1.0 >Reporter: Sergio Peña >Assignee: Sergio Peña >Priority: Major > > Add HMS database and table ownership information on the create and alter > messages so that other components, such as Sentry, can get the ownership > information from new HMS events. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20765) fetch partitions for txn stats validation in get_aggr_stats with one call
[
https://issues.apache.org/jira/browse/HIVE-20765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657259#comment-16657259
]
Hive QA commented on HIVE-20765:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
3s{color} | {color:blue} standalone-metastore/metastore-server in master has
182 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14572/dev-support/hive-personality.sh
|
| git revision | master / bd3c05d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14572/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> fetch partitions for txn stats validation in get_aggr_stats with one call
> -
>
> Key: HIVE-20765
> URL: https://issues.apache.org/jira/browse/HIVE-20765
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20765.01.patch, HIVE-20765.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657255#comment-16657255 ] Vineet Garg commented on HIVE-20703: Latest patch (11) rebased with master. > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch, > HIVE-20703.11.patch, HIVE-20703.2.patch, HIVE-20703.3.patch, > HIVE-20703.4.patch, HIVE-20703.5.patch, HIVE-20703.6.patch, > HIVE-20703.7.patch, HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20703: --- Status: Open (was: Patch Available) > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch, > HIVE-20703.11.patch, HIVE-20703.2.patch, HIVE-20703.3.patch, > HIVE-20703.4.patch, HIVE-20703.5.patch, HIVE-20703.6.patch, > HIVE-20703.7.patch, HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20703: --- Status: Patch Available (was: Open) > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch, > HIVE-20703.11.patch, HIVE-20703.2.patch, HIVE-20703.3.patch, > HIVE-20703.4.patch, HIVE-20703.5.patch, HIVE-20703.6.patch, > HIVE-20703.7.patch, HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20703) Put dynamic sort partition optimization under cost based decision
[ https://issues.apache.org/jira/browse/HIVE-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20703: --- Attachment: HIVE-20703.11.patch > Put dynamic sort partition optimization under cost based decision > - > > Key: HIVE-20703 > URL: https://issues.apache.org/jira/browse/HIVE-20703 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-20703.1.patch, HIVE-20703.10.patch, > HIVE-20703.11.patch, HIVE-20703.2.patch, HIVE-20703.3.patch, > HIVE-20703.4.patch, HIVE-20703.5.patch, HIVE-20703.6.patch, > HIVE-20703.7.patch, HIVE-20703.8.patch, HIVE-20703.9.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20780) Add additional object owner information to ExtendJSON message classes
[ https://issues.apache.org/jira/browse/HIVE-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña reassigned HIVE-20780: -- Assignee: Sergio Peña > Add additional object owner information to ExtendJSON message classes > - > > Key: HIVE-20780 > URL: https://issues.apache.org/jira/browse/HIVE-20780 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.1.0 >Reporter: Sergio Peña >Assignee: Sergio Peña >Priority: Major > > Add HMS database and table ownership information on the create and alter > messages so that other components, such as Sentry, can get the ownership > information from new HMS events. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20775) Factor cost of each SJ reduction when costing a follow-up reduction
[
https://issues.apache.org/jira/browse/HIVE-20775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657236#comment-16657236
]
Hive QA commented on HIVE-20775:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944637/HIVE-20775.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 129 failed/errored test(s), 14754 tests
executed
*Failed tests:*
{noformat}
TestAMReporter - did not produce a TEST-*.xml file (likely timed out)
(batchId=330)
TestAddResource - did not produce a TEST-*.xml file (likely timed out)
(batchId=318)
TestBeeLineWithArgs - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestBeelineConnectionUsingHiveSite - did not produce a TEST-*.xml file (likely
timed out) (batchId=252)
TestBeelinePasswordOption - did not produce a TEST-*.xml file (likely timed
out) (batchId=252)
TestBeelineWithUserHs2ConnectionFile - did not produce a TEST-*.xml file
(likely timed out) (batchId=252)
TestBuddyAllocator - did not produce a TEST-*.xml file (likely timed out)
(batchId=330)
TestBuddyAllocatorForceEvict - did not produce a TEST-*.xml file (likely timed
out) (batchId=330)
TestCleanerWithReplication - did not produce a TEST-*.xml file (likely timed
out) (batchId=244)
TestCreateUdfEntities - did not produce a TEST-*.xml file (likely timed out)
(batchId=244)
TestCustomAuthentication - did not produce a TEST-*.xml file (likely timed out)
(batchId=252)
TestDDLWithRemoteMetastoreSecondNamenode - did not produce a TEST-*.xml file
(likely timed out) (batchId=244)
TestDropMacroDesc - did not produce a TEST-*.xml file (likely timed out)
(batchId=318)
TestEvictingPriorityBlockingQueue - did not produce a TEST-*.xml file (likely
timed out) (batchId=330)
TestExportImport - did not produce a TEST-*.xml file (likely timed out)
(batchId=244)
TestFirstInFirstOutComparator - did not produce a TEST-*.xml file (likely timed
out) (batchId=330)
TestHiveOperationType - did not produce a TEST-*.xml file (likely timed out)
(batchId=318)
TestIncrementalObjectSizeEstimator - did not produce a TEST-*.xml file (likely
timed out) (batchId=330)
TestInformationSchemaWithPrivilege - did not produce a TEST-*.xml file (likely
timed out) (batchId=254)
TestLlapDaemonProtocolServerImpl - did not produce a TEST-*.xml file (likely
timed out) (batchId=330)
TestLlapSignerImpl - did not produce a TEST-*.xml file (likely timed out)
(batchId=330)
TestLlapStatusServiceDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=330)
TestLlapTokenChecker - did not produce a TEST-*.xml file (likely timed out)
(batchId=330)
TestLlapWebServices - did not produce a TEST-*.xml file (likely timed out)
(batchId=330)
TestLowLevelCacheImpl - did not produce a TEST-*.xml file (likely timed out)
(batchId=330)
TestLowLevelLrfuCachePolicy - did not produce a TEST-*.xml file (likely timed
out) (batchId=330)
TestMetaStoreLimitPartitionRequest - did not produce a TEST-*.xml file (likely
timed out) (batchId=244)
TestMiniHS2StateWithNoZookeeper - did not produce a TEST-*.xml file (likely
timed out) (batchId=252)
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=178)
[smb_mapjoin_15.q,insert_values_partitioned.q,vector_orc_merge_incompat_schema.q,selectDistinctStar.q,strict_managed_tables1.q,bucket4.q,vectorized_distinct_gby.q,vector_groupby_mapjoin.q,insert_values_dynamic_partitioned.q,autoColumnStats_10.q,tez_vector_dynpart_hashjoin_1.q,filter_join_breaktask2.q,union_rowcounts.q,mm_conversions.q,multiMapJoin2.q,filter_join_breaktask.q,column_table_stats.q,update_access_time_non_current_db.q,schema_evol_orc_nonvec_part.q,dynamic_semijoin_reduction_2.q,alter_merge_2_orc.q,bucket_num_reducers2.q,union4.q,auto_sortmerge_join_8.q,llap_text.q,dec_str.q,dynamic_semijoin_reduction_sw2.q,change_allowincompatible_vectorization_false_date.q,dynamic_semijoin_reduction_3.q,vector_decimal_udf2.q]
TestOperationLoggingAPIWithMr - did not produce a TEST-*.xml file (likely timed
out) (batchId=254)
TestOperationLoggingLayout - did not produce a TEST-*.xml file (likely timed
out) (batchId=254)
TestOperatorCmp - did not produce a TEST-*.xml file (likely timed out)
(batchId=318)
TestOrcMetadataCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=330)
TestQueryDisplay - did not produce a TEST-*.xml file (likely timed out)
(batchId=254)
TestQueryIdentifier - did not produce a TEST-*.xml file (likely timed out)
(batchId=330)
TestReplicationOnHDFSEncryptedZones - did not produce a TEST-*.xml file (likely
timed out) (batchId=244)
TestSQLStdHiveAccessControllerCLI - did not produce a TEST-*.xml file (likely
timed out) (batchId=318)
TestSQLStdHiveAccessControllerHS2 - did not produce a TEST-*.xml file (likely
timed out) (batchId=318)
TestSchemaTool - did not produce a TEST-*.xml file
[jira] [Updated] (HIVE-20701) Allow HiveStreaming to receive a key value to commit atomically together with the transaction
[ https://issues.apache.org/jira/browse/HIVE-20701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20701: --- Status: Open (was: Patch Available) > Allow HiveStreaming to receive a key value to commit atomically together with > the transaction > - > > Key: HIVE-20701 > URL: https://issues.apache.org/jira/browse/HIVE-20701 > Project: Hive > Issue Type: Improvement >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20701.1.patch, HIVE-20701.2.patch, > HIVE-20701.3.patch, HIVE-20701.4.patch, HIVE-20701.5.patch, > HIVE-20701.6.patch, HIVE-20701.7.patch > > > Following up with HIVE-20538 it'd be nice to be able to use this feature with > hive streaming -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20701) Allow HiveStreaming to receive a key value to commit atomically together with the transaction
[ https://issues.apache.org/jira/browse/HIVE-20701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20701: --- Attachment: HIVE-20701.7.patch Status: Patch Available (was: Open) > Allow HiveStreaming to receive a key value to commit atomically together with > the transaction > - > > Key: HIVE-20701 > URL: https://issues.apache.org/jira/browse/HIVE-20701 > Project: Hive > Issue Type: Improvement >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20701.1.patch, HIVE-20701.2.patch, > HIVE-20701.3.patch, HIVE-20701.4.patch, HIVE-20701.5.patch, > HIVE-20701.6.patch, HIVE-20701.7.patch > > > Following up with HIVE-20538 it'd be nice to be able to use this feature with > hive streaming -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20708) Load (dumped) an external table as an external table on target with the same location as on the source
[ https://issues.apache.org/jira/browse/HIVE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20708: -- Attachment: HIVE-20708.02 Status: Patch Available (was: In Progress) Attached patch with fixes for Hive-QA reported errors and test failures. > Load (dumped) an external table as an external table on target with the same > location as on the source > -- > > Key: HIVE-20708 > URL: https://issues.apache.org/jira/browse/HIVE-20708 > Project: Hive > Issue Type: Improvement > Components: repl >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20708.01, HIVE-20708.02 > > > External tables are currently mapped to managed tables on target. A lot of > jobs in user environment are dependent upon locations specified in external > table definitions to run, hence, the path for external tables on the target > and on the source are expected to be the same. An external table being loaded > as a managed table makes it difficult for failover (Controlled Failover) / > failback since there is no option of moving data from managed to external > table. So the external table replicated to target cluster needs to be kept as > external table with same location as on the source. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20708) Load (dumped) an external table as an external table on target with the same location as on the source
[ https://issues.apache.org/jira/browse/HIVE-20708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20708: -- Status: In Progress (was: Patch Available) > Load (dumped) an external table as an external table on target with the same > location as on the source > -- > > Key: HIVE-20708 > URL: https://issues.apache.org/jira/browse/HIVE-20708 > Project: Hive > Issue Type: Improvement > Components: repl >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20708.01 > > > External tables are currently mapped to managed tables on target. A lot of > jobs in user environment are dependent upon locations specified in external > table definitions to run, hence, the path for external tables on the target > and on the source are expected to be the same. An external table being loaded > as a managed table makes it difficult for failover (Controlled Failover) / > failback since there is no option of moving data from managed to external > table. So the external table replicated to target cluster needs to be kept as > external table with same location as on the source. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20775) Factor cost of each SJ reduction when costing a follow-up reduction
[
https://issues.apache.org/jira/browse/HIVE-20775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657181#comment-16657181
]
Hive QA commented on HIVE-20775:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 3 new + 123 unchanged - 3
fixed = 126 total (was 126) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
11s{color} | {color:red} ql generated 2 new + 2316 unchanged - 2 fixed = 2318
total (was 2318) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to tsRowSize in
org.apache.hadoop.hive.ql.parse.TezCompiler.getBloomFilterBenefit(SelectOperator,
ExprNodeDesc, Statistics, ExprNodeDesc) At
TezCompiler.java:org.apache.hadoop.hive.ql.parse.TezCompiler.getBloomFilterBenefit(SelectOperator,
ExprNodeDesc, Statistics, ExprNodeDesc) At TezCompiler.java:[line 1427] |
| | Should org.apache.hadoop.hive.ql.parse.TezCompiler$SemijoinOperatorInfo
be a _static_ inner class? At TezCompiler.java:inner class? At
TezCompiler.java:[lines 1612-1618] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14571/dev-support/hive-personality.sh
|
| git revision | master / bd3c05d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14571/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14571/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14571/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Factor cost of each SJ reduction when costing a follow-up reduction
> ---
>
> Key: HIVE-20775
> URL: https://issues.apache.org/jira/browse/HIVE-20775
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20775.patch
>
>
> Currently, while costing the SJ in a plan, the stats of the a TS that is
> reduced by a SJ are not adjusted after we have decided to keep a SJ in the
> tree. Ideally, we could adjust the stats to take into account decisions that
> have already been made.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Move HMS filterHooks from client-side to server-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657168#comment-16657168
]
Vihang Karajgaonkar commented on HIVE-20776:
Are you talking about {{MetaStoreFilterHook}} implementations? Looks like the
original motivation of these hooks were authorization plugins on the HS2
(client side). If you read the javadoc on the interface, it makes more sense.
{quote}/**
* Metadata filter hook for metastore client. This will be useful for
authorization
* plugins on hiveserver2 to filter metadata results, especially in case of
* non-impersonation mode where the metastore doesn't know the end user's
identity.
*/{quote}
When impersonation is turned OFF (default on secure setups on some
distributions), HMS doesn't really know who the end-user is. And hence it is
hard to implement a authorization policy based on UGI of the end-user. The only
way to enforce such a authorization policy would be on the client side where we
know who is the end-user executing the request.
> Move HMS filterHooks from client-side to server-side
>
>
> Key: HIVE-20776
> URL: https://issues.apache.org/jira/browse/HIVE-20776
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Karthik Manamcheri
>Priority: Major
>
> In HMS, I noticed that all the filter hooks are applied on the client side
> (in HiveMetaStoreClient.java). Is there any reason why we can't apply the
> filters on the server-side?
> Motivation: Some newer apache projects such as Kudu use HMS for metadata
> storage. Kudu is not completely Java-based and there are interaction points
> where they have C++ clients. In such cases, it would be ideal to have
> consistent behavior from HMS side as far as filters, etc are concerned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20772) record per-task CPU counters in LLAP
[
https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657108#comment-16657108
]
Hive QA commented on HIVE-20772:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944615/HIVE-20772.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 15111 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters1]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
(batchId=155)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14569/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14569/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14569/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944615 - PreCommit-HIVE-Build
> record per-task CPU counters in LLAP
>
>
> Key: HIVE-20772
> URL: https://issues.apache.org/jira/browse/HIVE-20772
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20772.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'
[
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657088#comment-16657088
]
Peter Vary commented on HIVE-18767:
---
[~massdos...@gmail.com]: We waited enough. Just show me the last patch! :)
> Some alterPartitions invocations throw 'NumberFormatException: null'
>
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.3.3, 3.1.0, 4.0.0, 3.2.0
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.4.0, 4.0.0, 3.1.1, 2.3.4
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767-branch-2.patch,
> HIVE-18767-branch-3.1.patch, HIVE-18767-branch-3.patch, HIVE-18767.1.patch,
> HIVE-18767.2-branch-2.3.patch, HIVE-18767.2-branch-2.patch,
> HIVE-18767.2-branch-3.1.patch, HIVE-18767.2.patch,
> HIVE-18767.3-branch-3.1.patch, HIVE-18767.3.patch,
> HIVE-18767.4-branch-3.1.patch, HIVE-18767.4.branch-3.1.patch,
> HIVE-18767.4.patch, HIVE-18767.5.patch, HIVE-18767.6.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown
> Source)
> [info] at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20542) Incremental REPL DUMP progress information log message is incorrect.
[ https://issues.apache.org/jira/browse/HIVE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657069#comment-16657069 ] Ashutosh Bapat commented on HIVE-20542: --- [~sankarh], I have updated the pull request addressing your comments as well as the comments by Hive-QA. > Incremental REPL DUMP progress information log message is incorrect. > > > Key: HIVE-20542 > URL: https://issues.apache.org/jira/browse/HIVE-20542 > Project: Hive > Issue Type: Bug > Components: repl >Affects Versions: 4.0.0, 3.2.0 >Reporter: Sankar Hariappan >Assignee: Ashutosh Bapat >Priority: Minor > Labels: DR, Replication, pull-request-available > Attachments: HIVE-20542.01.patch, HIVE-20542.02, HIVE-20542.03 > > > Incremental REPL DUMP have the progress information logged as > "eventsDumpProgress":"49/0". > It should actually log the estimated number of events are denominator but it > is coming as 0 always. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20542) Incremental REPL DUMP progress information log message is incorrect.
[ https://issues.apache.org/jira/browse/HIVE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20542: -- Attachment: HIVE-20542.03 Status: Patch Available (was: In Progress) > Incremental REPL DUMP progress information log message is incorrect. > > > Key: HIVE-20542 > URL: https://issues.apache.org/jira/browse/HIVE-20542 > Project: Hive > Issue Type: Bug > Components: repl >Affects Versions: 4.0.0, 3.2.0 >Reporter: Sankar Hariappan >Assignee: Ashutosh Bapat >Priority: Minor > Labels: DR, Replication, pull-request-available > Attachments: HIVE-20542.01.patch, HIVE-20542.02, HIVE-20542.03 > > > Incremental REPL DUMP have the progress information logged as > "eventsDumpProgress":"49/0". > It should actually log the estimated number of events are denominator but it > is coming as 0 always. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20542) Incremental REPL DUMP progress information log message is incorrect.
[ https://issues.apache.org/jira/browse/HIVE-20542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20542: -- Status: In Progress (was: Patch Available) > Incremental REPL DUMP progress information log message is incorrect. > > > Key: HIVE-20542 > URL: https://issues.apache.org/jira/browse/HIVE-20542 > Project: Hive > Issue Type: Bug > Components: repl >Affects Versions: 4.0.0, 3.2.0 >Reporter: Sankar Hariappan >Assignee: Ashutosh Bapat >Priority: Minor > Labels: DR, Replication, pull-request-available > Attachments: HIVE-20542.01.patch, HIVE-20542.02 > > > Incremental REPL DUMP have the progress information logged as > "eventsDumpProgress":"49/0". > It should actually log the estimated number of events are denominator but it > is coming as 0 always. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[ https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657031#comment-16657031 ] Ashutosh Chauhan commented on HIVE-20767: - +1 pending tests > Multiple project between join operators may affect join reordering using > constraints > > > Key: HIVE-20767 > URL: https://issues.apache.org/jira/browse/HIVE-20767 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20767.01.patch, HIVE-20767.02.patch, > HIVE-20767.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20772) record per-task CPU counters in LLAP
[
https://issues.apache.org/jira/browse/HIVE-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657023#comment-16657023
]
Hive QA commented on HIVE-20772:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} llap-common in master has 71 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} llap-server in master has 83 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
18s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 18s{color}
| {color:red} llap-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} llap-server: The patch generated 3 new + 223 unchanged
- 1 fixed = 226 total (was 224) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14569/dev-support/hive-personality.sh
|
| git revision | master / bd3c05d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14569/yetus/patch-mvninstall-llap-server.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14569/yetus/patch-compile-llap-server.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14569/yetus/patch-compile-llap-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14569/yetus/diff-checkstyle-llap-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14569/yetus/patch-findbugs-llap-server.txt
|
| modules | C: llap-common llap-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14569/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> record per-task CPU counters in LLAP
>
>
> Key: HIVE-20772
> URL: https://issues.apache.org/jira/browse/HIVE-20772
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-20772.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[ https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16657009#comment-16657009 ] Jesus Camacho Rodriguez commented on HIVE-20767: [~ashutoshc], [~vgarg], fixed all those issues in the latest patch by moving the block back outside of the join reordering if clause. Created HIVE-20778 to fix the real underlying problem. Thanks > Multiple project between join operators may affect join reordering using > constraints > > > Key: HIVE-20767 > URL: https://issues.apache.org/jira/browse/HIVE-20767 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20767.01.patch, HIVE-20767.02.patch, > HIVE-20767.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[ https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20767: --- Attachment: HIVE-20767.02.patch > Multiple project between join operators may affect join reordering using > constraints > > > Key: HIVE-20767 > URL: https://issues.apache.org/jira/browse/HIVE-20767 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20767.01.patch, HIVE-20767.02.patch, > HIVE-20767.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[
https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656993#comment-16656993
]
Hive QA commented on HIVE-20767:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944609/HIVE-20767.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15111 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14568/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14568/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14568/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944609 - PreCommit-HIVE-Build
> Multiple project between join operators may affect join reordering using
> constraints
>
>
> Key: HIVE-20767
> URL: https://issues.apache.org/jira/browse/HIVE-20767
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20767.01.patch, HIVE-20767.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20768) Adding Tumbling Window UDF
[
https://issues.apache.org/jira/browse/HIVE-20768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-20768:
--
Attachment: HIVE-20768.2.patch
> Adding Tumbling Window UDF
> --
>
> Key: HIVE-20768
> URL: https://issues.apache.org/jira/browse/HIVE-20768
> Project: Hive
> Issue Type: New Feature
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20768.2.patch, HIVE-20768.patch, HIVE-20768.patch
>
>
> Goal is to provide a UDF that truncates a timestamp to a beginning of a
> tumbling window interval.
> {code}
> /**
> * Tumbling windows are a series of fixed-sized, non-overlapping and
> contiguous time intervals.
> * Tumbling windows are inclusive start exclusive end.
> * By default the beginning instant of fist window is Epoch 0 Thu Jan 01
> 00:00:00 1970 UTC.
> * Optionally users may provide a different origin as a timestamp arg3.
> *
> * This an example of series of window with an interval of 5 seconds and
> origin Epoch 0 Thu Jan 01 00:00:00 1970 UTC:
> *
> *
> * interval 1 interval 2interval 3
> * Jan 01 00:00:00 Jan 01 00:00:05 Jan 01 00:00:10
> * 0 -- 4 : 5 --- 9: 10 --- 14
> *
> * This UDF rounds timestamp agr1 to the beginning of window interval where
> it belongs to.
> *
> */
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20778) Join reordering may not be triggered if all joins in plan are created by decorrelation logic
[ https://issues.apache.org/jira/browse/HIVE-20778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-20778: -- > Join reordering may not be triggered if all joins in plan are created by > decorrelation logic > > > Key: HIVE-20778 > URL: https://issues.apache.org/jira/browse/HIVE-20778 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20767) Multiple project between join operators may affect join reordering using constraints
[
https://issues.apache.org/jira/browse/HIVE-20767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656934#comment-16656934
]
Hive QA commented on HIVE-20767:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14568/dev-support/hive-personality.sh
|
| git revision | master / bd3c05d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14568/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Multiple project between join operators may affect join reordering using
> constraints
>
>
> Key: HIVE-20767
> URL: https://issues.apache.org/jira/browse/HIVE-20767
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20767.01.patch, HIVE-20767.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656891#comment-16656891
]
Hive QA commented on HIVE-20707:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944600/HIVE-20707.6.patch
{color:green}SUCCESS:{color} +1 due to 14 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15121 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=168)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14566/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14566/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14566/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944600 - PreCommit-HIVE-Build
> Automatic partition management
> --
>
> Key: HIVE-20707
> URL: https://issues.apache.org/jira/browse/HIVE-20707
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch,
> HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch, HIVE-20707.6.patch
>
>
> In current scenario, to add partitions for external tables to metastore, MSCK
> REPAIR command has to be executed manually. To avoid this manual step,
> external tables can be specified a table property based on which a background
> metastore thread can sync partitions periodically. Tables can also be
> specified with partition retention period. Any partition whose age exceeds
> the retention period will be dropped automatically.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20395) Parallelize files move in the ql.metadata.Hive#replaceFiles
[ https://issues.apache.org/jira/browse/HIVE-20395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-20395: - Status: Patch Available (was: Open) > Parallelize files move in the ql.metadata.Hive#replaceFiles > --- > > Key: HIVE-20395 > URL: https://issues.apache.org/jira/browse/HIVE-20395 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0 >Reporter: Igor Dvorzhak >Assignee: Igor Dvorzhak >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20395.10-branch-3.patch, HIVE-20395.10.patch, > HIVE-20395.11-branch-2.patch > > > Files move is not parallelized in `replaceFiles` method and could be very > slow on Cloud Object Stores. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20395) Parallelize files move in the ql.metadata.Hive#replaceFiles
[ https://issues.apache.org/jira/browse/HIVE-20395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Dvorzhak updated HIVE-20395: - Status: Open (was: Patch Available) > Parallelize files move in the ql.metadata.Hive#replaceFiles > --- > > Key: HIVE-20395 > URL: https://issues.apache.org/jira/browse/HIVE-20395 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0 >Reporter: Igor Dvorzhak >Assignee: Igor Dvorzhak >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20395.10-branch-3.patch, HIVE-20395.10.patch, > HIVE-20395.11-branch-2.patch > > > Files move is not parallelized in `replaceFiles` method and could be very > slow on Cloud Object Stores. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656835#comment-16656835
]
Hive QA commented on HIVE-20707:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 13s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-14566/patches/PreCommit-HIVE-Build-14566.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14566/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Automatic partition management
> --
>
> Key: HIVE-20707
> URL: https://issues.apache.org/jira/browse/HIVE-20707
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch,
> HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch, HIVE-20707.6.patch
>
>
> In current scenario, to add partitions for external tables to metastore, MSCK
> REPAIR command has to be executed manually. To avoid this manual step,
> external tables can be specified a table property based on which a background
> metastore thread can sync partitions periodically. Tables can also be
> specified with partition retention period. Any partition whose age exceeds
> the retention period will be dropped automatically.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
