[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745552#comment-16745552 ] Jesus Camacho Rodriguez commented on HIVE-20707: Thanks for letting me know [~prasanth_j]. HIVE-21072 already has the (small) part of the patch that would fix that issue described over there, hence I think we will just backport that. > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20702.3.patch, HIVE-20707-branch-3.patch, > HIVE-20707.1.patch, HIVE-20707.2.patch, HIVE-20707.4.patch, > HIVE-20707.5.patch, HIVE-20707.6.patch, HIVE-20707.6.patch, HIVE-20707.7.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744562#comment-16744562 ] Prasanth Jayachandran commented on HIVE-20707: -- [~jcamachorodriguez] This patch brings in breaking changes mostly for external tables (auto partition discovery is enabled by default). That's why did not backport the patch to branch-3. I am guessing the fixes are primarily related to catalog name missing in metastore APIs?. If that's the case we could pick only the changes to ObjectStore.java, NonCatCallsWithCatalog.java which are pretty small. What do you think? > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20702.3.patch, HIVE-20707-branch-3.patch, > HIVE-20707.1.patch, HIVE-20707.2.patch, HIVE-20707.4.patch, > HIVE-20707.5.patch, HIVE-20707.6.patch, HIVE-20707.6.patch, HIVE-20707.7.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744314#comment-16744314 ] Jesus Camacho Rodriguez commented on HIVE-20707: [~prasanth_j], I see there is a branch-3 patch that was never pushed. Should this be backported? It seems to fix issues such as HIVE-21072. > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20702.3.patch, HIVE-20707-branch-3.patch, > HIVE-20707.1.patch, HIVE-20707.2.patch, HIVE-20707.4.patch, > HIVE-20707.5.patch, HIVE-20707.6.patch, HIVE-20707.6.patch, HIVE-20707.7.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667796#comment-16667796 ] Jason Dere commented on HIVE-20707: --- +1 > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch, > HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch, > HIVE-20707.6.patch, HIVE-20707.6.patch, HIVE-20707.7.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667242#comment-16667242
]
Hive QA commented on HIVE-20707:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946012/HIVE-20707.7.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15518 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14680/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14680/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14680/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12946012 - PreCommit-HIVE-Build
> Automatic partition management
> --
>
> Key: HIVE-20707
> URL: https://issues.apache.org/jira/browse/HIVE-20707
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch,
> HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch,
> HIVE-20707.6.patch, HIVE-20707.6.patch, HIVE-20707.7.patch
>
>
> In current scenario, to add partitions for external tables to metastore, MSCK
> REPAIR command has to be executed manually. To avoid this manual step,
> external tables can be specified a table property based on which a background
> metastore thread can sync partitions periodically. Tables can also be
> specified with partition retention period. Any partition whose age exceeds
> the retention period will be dropped automatically.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667209#comment-16667209
]
Hive QA commented on HIVE-20707:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
19s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2317 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
59s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
29s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 40 new + 515 unchanged - 6
fixed = 555 total (was 521) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
27s{color} | {color:red} standalone-metastore/metastore-common generated 1 new
+ 28 unchanged - 0 fixed = 29 total (was 28) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} hbase-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} ql generated 0 new + 2315 unchanged - 2 fixed = 2315
total (was 2317) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
14s{color} | {color:red} standalone-metastore/metastore-server generated 4 new
+ 181 unchanged - 0 fixed = 185 total (was 181) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
19s{color} | {color:red} standalone-metastore_metastore-server generated 1 new
+ 47 unchanged - 0 fixed = 48 total (was 47) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-common |
| | org.apache.hadoop.hive.metastore.CheckResult$PartitionResult defines
[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1889#comment-1889 ] Prasanth Jayachandran commented on HIVE-20707: -- Uploaded wrong patch before. Moved partition management task to remote metastore only task. This fixed druid test failures. > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch, > HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch, > HIVE-20707.6.patch, HIVE-20707.6.patch, HIVE-20707.7.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1891#comment-1891 ] Prasanth Jayachandran commented on HIVE-20707: -- [~jdere] can you please take another look? > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch, > HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch, > HIVE-20707.6.patch, HIVE-20707.6.patch, HIVE-20707.7.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656891#comment-16656891
]
Hive QA commented on HIVE-20707:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944600/HIVE-20707.6.patch
{color:green}SUCCESS:{color} +1 due to 14 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15121 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=168)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14566/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14566/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14566/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944600 - PreCommit-HIVE-Build
> Automatic partition management
> --
>
> Key: HIVE-20707
> URL: https://issues.apache.org/jira/browse/HIVE-20707
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch,
> HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch, HIVE-20707.6.patch
>
>
> In current scenario, to add partitions for external tables to metastore, MSCK
> REPAIR command has to be executed manually. To avoid this manual step,
> external tables can be specified a table property based on which a background
> metastore thread can sync partitions periodically. Tables can also be
> specified with partition retention period. Any partition whose age exceeds
> the retention period will be dropped automatically.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16656835#comment-16656835
]
Hive QA commented on HIVE-20707:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 13s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-14566/patches/PreCommit-HIVE-Build-14566.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14566/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Automatic partition management
> --
>
> Key: HIVE-20707
> URL: https://issues.apache.org/jira/browse/HIVE-20707
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch,
> HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch, HIVE-20707.6.patch
>
>
> In current scenario, to add partitions for external tables to metastore, MSCK
> REPAIR command has to be executed manually. To avoid this manual step,
> external tables can be specified a table property based on which a background
> metastore thread can sync partitions periodically. Tables can also be
> specified with partition retention period. Any partition whose age exceeds
> the retention period will be dropped automatically.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655714#comment-16655714 ] Prasanth Jayachandran commented on HIVE-20707: -- druid test failure looks strange. Running it locally is passing. > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch, > HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch, HIVE-20707.6.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654900#comment-16654900
]
Hive QA commented on HIVE-20707:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944396/HIVE-20707.5.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15106 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_discovery]
(batchId=24)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=168)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14537/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14537/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14537/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944396 - PreCommit-HIVE-Build
> Automatic partition management
> --
>
> Key: HIVE-20707
> URL: https://issues.apache.org/jira/browse/HIVE-20707
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch,
> HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch
>
>
> In current scenario, to add partitions for external tables to metastore, MSCK
> REPAIR command has to be executed manually. To avoid this manual step,
> external tables can be specified a table property based on which a background
> metastore thread can sync partitions periodically. Tables can also be
> specified with partition retention period. Any partition whose age exceeds
> the retention period will be dropped automatically.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654884#comment-16654884
]
Hive QA commented on HIVE-20707:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
24s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
51s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
4s{color} | {color:blue} standalone-metastore/metastore-server in master has
182 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
30s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 40 new + 515 unchanged - 6
fixed = 555 total (was 521) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
26s{color} | {color:red} standalone-metastore/metastore-common generated 1 new
+ 28 unchanged - 0 fixed = 29 total (was 28) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} hbase-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} ql generated 0 new + 2316 unchanged - 2 fixed = 2316
total (was 2318) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
13s{color} | {color:red} standalone-metastore/metastore-server generated 4 new
+ 182 unchanged - 0 fixed = 186 total (was 182) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore_metastore-server generated 1 new
+ 49 unchanged - 0 fixed = 50 total (was 49) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-common |
| | org.apache.hadoop.hive.metastore.CheckResult$PartitionResult defines
[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654066#comment-16654066 ] Prasanth Jayachandran commented on HIVE-20707: -- Only msck_repair_drop.q seems to be relevant. Added sorting to stabilize the output. > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch, > HIVE-20707.2.patch, HIVE-20707.4.patch, HIVE-20707.5.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16652067#comment-16652067
]
Hive QA commented on HIVE-20707:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944066/HIVE-20707.4.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14502/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14502/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14502/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12944066/HIVE-20707.4.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944066 - PreCommit-HIVE-Build
> Automatic partition management
> --
>
> Key: HIVE-20707
> URL: https://issues.apache.org/jira/browse/HIVE-20707
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch,
> HIVE-20707.2.patch, HIVE-20707.4.patch
>
>
> In current scenario, to add partitions for external tables to metastore, MSCK
> REPAIR command has to be executed manually. To avoid this manual step,
> external tables can be specified a table property based on which a background
> metastore thread can sync partitions periodically. Tables can also be
> specified with partition retention period. Any partition whose age exceeds
> the retention period will be dropped automatically.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651353#comment-16651353
]
Hive QA commented on HIVE-20707:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12944066/HIVE-20707.4.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 15095 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[msck_repair_drop]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_reduce]
(batchId=60)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert]
(batchId=195)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14496/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14496/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14496/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12944066 - PreCommit-HIVE-Build
> Automatic partition management
> --
>
> Key: HIVE-20707
> URL: https://issues.apache.org/jira/browse/HIVE-20707
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch,
> HIVE-20707.2.patch, HIVE-20707.4.patch
>
>
> In current scenario, to add partitions for external tables to metastore, MSCK
> REPAIR command has to be executed manually. To avoid this manual step,
> external tables can be specified a table property based on which a background
> metastore thread can sync partitions periodically. Tables can also be
> specified with partition retention period. Any partition whose age exceeds
> the retention period will be dropped automatically.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651323#comment-16651323
]
Hive QA commented on HIVE-20707:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
38s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
10s{color} | {color:blue} standalone-metastore/metastore-server in master has
182 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
32s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 40 new + 515 unchanged - 6
fixed = 555 total (was 521) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
41s{color} | {color:red} standalone-metastore/metastore-common generated 1 new
+ 28 unchanged - 0 fixed = 29 total (was 28) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} hbase-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} ql generated 0 new + 2316 unchanged - 2 fixed = 2316
total (was 2318) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
20s{color} | {color:red} standalone-metastore/metastore-server generated 4 new
+ 182 unchanged - 0 fixed = 186 total (was 182) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
19s{color} | {color:red} standalone-metastore_metastore-server generated 1 new
+ 49 unchanged - 0 fixed = 50 total (was 49) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 21s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-common |
| | org.apache.hadoop.hive.metastore.CheckResult$PartitionResult defines
[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650995#comment-16650995 ] Prasanth Jayachandran commented on HIVE-20707: -- Fixes some test failures. Druid tests are not running in my local machine still debugging it to make it run and regenerate golden files. > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20702.3.patch, HIVE-20707.1.patch, > HIVE-20707.2.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649152#comment-16649152
]
Hive QA commented on HIVE-20707:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943576/HIVE-20707.2.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 45 failed/errored test(s), 15059 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=171)
[dynamic_semijoin_reduction.q,materialized_view_create_rewrite_3.q,vectorization_pushdown.q,correlationoptimizer2.q,cbo_gby_empty.q,resourceplan.q,schema_evol_text_nonvec_part_all_complex_llap_io.q,vectorization_short_regress.q,identity_project_remove_skip.q,mapjoin3.q,cross_product_check_1.q,results_cache_quoted_identifiers.q,cbo_join.q,correlationoptimizer6.q,union_remove_26.q,cbo_rp_limit.q,convert_decimal64_to_decimal.q,vector_groupby_cube1.q,vectorization_13.q,union2.q,groupby2.q,dynpart_sort_opt_vectorization.q,exchgpartition2lel.q,schema_evol_text_vecrow_part_llap_io.q,sample10.q,vectorized_timestamp_ints_casts.q,auto_sortmerge_join_2.q,bucketizedhiveinputformat.q,cte_mat_2.q,vectorization_8.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_like] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_like_view]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[default_file_format]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_topn] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_locks]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[msck_repair_acid]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[msck_repair_drop]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[rename_external_partition_location]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[repl_2_exim_basic]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_create_table_alter]
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_create_table_partitioned]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_create_table_serde]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_noscan_2]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_display_colstats_tbllvl]
(batchId=84)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[external_table_ppd]
(batchId=102)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_binary_storage_queries]
(batchId=105)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_ddl]
(batchId=104)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_queries]
(batchId=104)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbasestats]
(batchId=103)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=194)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions]
(batchId=194)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert]
(batchId=194)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[external_table_purge]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[whroot_external1]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_exim]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[strict_managed_tables2]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=168)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats_noscan_2]
(batchId=126)
org.apache.hadoop.hive.metastore.TestCatalogNonDefaultClient.getTableMeta
(batchId=220)
org.apache.hadoop.hive.metastore.TestCatalogNonDefaultSvr.getTableMeta
(batchId=228)
org.apache.hadoop.hive.metastore.TestCatalogOldClient.getTableMeta (batchId=220)
org.apache.hadoop.hive.metastore.client.TestGetTableMeta.tablesInDifferentCatalog[Embedded]
(batchId=220)
org.apache.hadoop.hive.metastore.client.TestGetTableMeta.tablesInDifferentCatalog[Remote]
(batchId=220)
org.apache.hadoop.hive.metastore.client.TestGetTableMeta.testGetTableMetaCaseSensitive[Embedded]
(batchId=220)
org.apache.hadoop.hive.metastore.client.TestGetTableMeta.testGetTableMetaCaseSensitive[Remote]
(batchId=220)
org.apache.hadoop.hive.metastore.client.TestGetTableMeta.testGetTableMetaNullOrEmptyDb[Embedded]
(batchId=220)
org.apache.hadoop.hive.metastore.client.TestGetTableMeta.testGetTableMetaNullOrEmptyDb[Remote]
(batchId=220)
[jira] [Commented] (HIVE-20707) Automatic partition management
[
https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649148#comment-16649148
]
Hive QA commented on HIVE-20707:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
20s{color} | {color:blue} standalone-metastore/metastore-common in master has
28 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 2318 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
5s{color} | {color:blue} standalone-metastore/metastore-server in master has
182 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
32s{color} | {color:red} ql in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 169
unchanged - 1 fixed = 169 total (was 170) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 35 new + 461 unchanged - 6
fixed = 496 total (was 467) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s{color} | {color:green} common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
36s{color} | {color:red} standalone-metastore/metastore-common generated 1 new
+ 28 unchanged - 0 fixed = 29 total (was 28) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
1s{color} | {color:green} ql generated 0 new + 2316 unchanged - 2 fixed = 2316
total (was 2318) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
15s{color} | {color:red} standalone-metastore/metastore-server generated 4 new
+ 182 unchanged - 0 fixed = 186 total (was 182) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
19s{color} | {color:red} standalone-metastore_metastore-server generated 1 new
+ 49 unchanged - 0 fixed = 50 total (was 49) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
[jira] [Commented] (HIVE-20707) Automatic partition management
[ https://issues.apache.org/jira/browse/HIVE-20707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16647268#comment-16647268 ] Prasanth Jayachandran commented on HIVE-20707: -- There is more scope added to this patch 1) Support managed and external tables. Managed tables will get X lock for MSCK. 2) Retention period for partitions can be specified via table property. 3) MSCK repair can now handle retention periods via config. 4) All external tables will now automatically enable partition discovery. On top of these there are fixes related to ACID, MSCK was acquiring wrong lock, explain locks was not working for msck, fixes related to catalog handling (some metastore APIs was not handling catalog names correctly). > Automatic partition management > -- > > Key: HIVE-20707 > URL: https://issues.apache.org/jira/browse/HIVE-20707 > Project: Hive > Issue Type: New Feature >Affects Versions: 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-20707.1.patch > > > In current scenario, to add partitions for external tables to metastore, MSCK > REPAIR command has to be executed manually. To avoid this manual step, > external tables can be specified a table property based on which a background > metastore thread can sync partitions periodically. Tables can also be > specified with partition retention period. Any partition whose age exceeds > the retention period will be dropped automatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
