[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416682#comment-16416682
]
Hudson commented on HBASE-19441:
Results for branch HBASE-19064
[build #77 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/77/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/77//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/77//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-19064/77//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 19441-v5.patch, 19441-v6.patch, HBASE-19441-v1.patch,
> HBASE-19441-v2.patch, HBASE-19441-v3.patch, HBASE-19441-v4.patch,
> HBASE-19441-v5.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16409596#comment-16409596
]
Hudson commented on HBASE-19441:
Results for branch master
[build #269 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/269/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/269//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/269//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/269//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 19441-v5.patch, 19441-v6.patch, HBASE-19441-v1.patch,
> HBASE-19441-v2.patch, HBASE-19441-v3.patch, HBASE-19441-v4.patch,
> HBASE-19441-v5.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408189#comment-16408189
]
Hadoop QA commented on HBASE-19441:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} hbase-backup: The patch generated 3 new + 0 unchanged
- 0 fixed = 3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
48s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 49s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
23s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19441 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915505/19441-v6.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 2f804a651fa3 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8ab7b20f48 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/12060/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12060/testReport/ |
| Max. process+thread count | 4474 (vs. ulimit of 1) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408172#comment-16408172
]
Josh Elser commented on HBASE-19441:
{quote}
bq. Why is this illegal import, Ted Yu?
I think the package from com.google.common.util is not shaded :
{quote}
Right. The full answer is that we've moved dependencies which tend to have
compatibility issues into the hbase-thirdparty project. In this project, we
shade these dependencies to help insulate us from other JARs that happen to
make it onto the runtime classpath. Thus, we want to make sure that we're using
these shaded variants -- that's what this checkstyle rule is telling us: we're
using the non-shaded path to these classes instead of the shaded path we want.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 19441-v5.patch, 19441-v6.patch, HBASE-19441-v1.patch,
> HBASE-19441-v2.patch, HBASE-19441-v3.patch, HBASE-19441-v4.patch,
> HBASE-19441-v5.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408082#comment-16408082
]
Ted Yu commented on HBASE-19441:
Patch v6 includes the indentation fixes from Vlad's v5.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 19441-v5.patch, 19441-v6.patch, HBASE-19441-v1.patch,
> HBASE-19441-v2.patch, HBASE-19441-v3.patch, HBASE-19441-v4.patch,
> HBASE-19441-v5.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407648#comment-16407648
]
Hadoop QA commented on HBASE-19441:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
30s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} hbase-backup: The patch generated 8 new + 0 unchanged
- 0 fixed = 8 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
19m 16s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 15m
33s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 50m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19441 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915444/19441-v5.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux b97b3fa7656b 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 8ab7b20f48 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/12053/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12053/testReport/ |
| Max. process+thread count | 4535 (vs. ulimit of 1) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407573#comment-16407573
]
Ted Yu commented on HBASE-19441:
Patch v5 imports from shaded artifact.
TestBackupManager passes locally with v5.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 19441-v5.patch, HBASE-19441-v1.patch,
> HBASE-19441-v2.patch, HBASE-19441-v3.patch, HBASE-19441-v4.patch,
> HBASE-19441-v5.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407572#comment-16407572
]
Ted Yu commented on HBASE-19441:
I think the package from com.google.common.util is not shaded :
{code}
+import
org.apache.hbase.thirdparty.com.google.common.util.concurrent.Uninterruptibles;
{code}
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch, HBASE-19441-v2.patch,
> HBASE-19441-v3.patch, HBASE-19441-v4.patch, HBASE-19441-v5.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407479#comment-16407479
]
Vladimir Rodionov commented on HBASE-19441:
---
Why is this illegal import, [~yuzhih...@gmail.com]?
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch, HBASE-19441-v2.patch,
> HBASE-19441-v3.patch, HBASE-19441-v4.patch, HBASE-19441-v5.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407306#comment-16407306
]
Ted Yu commented on HBASE-19441:
Among the checkstyle warnings, this must be addressed:
{code}
./hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java:44:import
com.google.common.util.concurrent.Uninterruptibles;:1: Import from illegal
package - com.google.common.util.concurrent.Uninterruptibles. [IllegalImport]
{code}
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch, HBASE-19441-v2.patch,
> HBASE-19441-v3.patch, HBASE-19441-v4.patch, HBASE-19441-v5.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407303#comment-16407303
]
Hadoop QA commented on HBASE-19441:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
51s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} hbase-backup: The patch generated 5 new + 0 unchanged
- 0 fixed = 5 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
42s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 6m
59s{color} | {color:red} The patch causes 10 errors with Hadoop v2.6.5. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 9m
15s{color} | {color:red} The patch causes 10 errors with Hadoop v2.7.4. {color}
|
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 11m
43s{color} | {color:red} The patch causes 10 errors with Hadoop v3.0.0. {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m
10s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19441 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915396/HBASE-19441-v5.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 4c675205ed44 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / acbdb86bb4 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407194#comment-16407194
]
Vladimir Rodionov commented on HBASE-19441:
---
Patch v5. Fixed indentation issues.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch, HBASE-19441-v2.patch,
> HBASE-19441-v3.patch, HBASE-19441-v4.patch, HBASE-19441-v5.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407160#comment-16407160
]
Hadoop QA commented on HBASE-19441:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} hbase-backup: The patch generated 10 new + 0 unchanged
- 0 fixed = 10 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
25s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 5s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 16m
38s{color} | {color:green} hbase-backup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19441 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915383/HBASE-19441-v4.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 8f26608153e2 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12
21:23:04 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / acbdb86bb4 |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/12043/artifact/patchprocess/diff-checkstyle-hbase-backup.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/12043/testReport/ |
| Max. process+thread count | 4620 (vs. ulimit of 1) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407088#comment-16407088
]
Vladimir Rodionov commented on HBASE-19441:
---
Patch v4.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch, HBASE-19441-v2.patch,
> HBASE-19441-v3.patch, HBASE-19441-v4.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406961#comment-16406961
]
Ted Yu commented on HBASE-19441:
{code}
+public class ExclusiveOperationException extends IOException {
{code}
Please add audience annotation (Private).
Otherwise looks good.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch, HBASE-19441-v2.patch,
> HBASE-19441-v3.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406867#comment-16406867
]
Vladimir Rodionov commented on HBASE-19441:
---
Patch v3.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch, HBASE-19441-v2.patch,
> HBASE-19441-v3.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406687#comment-16406687
]
Josh Elser commented on HBASE-19441:
{code:java}
[WARNING]
/testptch/hbase/hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java:[87,19]
[MissingOverride] run implements method in Runnable; expected @Override
[WARNING]
/testptch/hbase/hbase-backup/src/test/java/org/apache/hadoop/hbase/backup/TestBackupManager.java:[120,24]
[ThreadJoinLoop] Thread.join needs to be surrounded by a loop until it
succeeds, as in Uninterruptibles.joinUninterruptibly.{code}
Also please fix the javac warnings
{code:java}
+ throw new IOException();
...
+} catch (IOException | InterruptedException e) {
+ fail("Unexpected exception: "+ e.getMessage());{code}
Can you please add a message to the IOException thrown in the test? If this
test would fail for some reason, we'd get a really non-intuitive explanation.
{noformat}
+assertTrue(startTimes.get(1) - startTimes.get(0) >= sleepTime);
+assertTrue(stopTimes.get(1) - stopTimes.get(0) >= sleepTime);{noformat}
Would be nice to print out the {{startTimes}} and {{stopTimes}} in the
respective messages for these assertions.
Otherwise, this looks fine to me. A nice improvement over what is presently in
master. Good work, Vlad.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch, HBASE-19441-v2.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406436#comment-16406436
]
Ted Yu commented on HBASE-19441:
{code}
[ERROR] TestBackupManager.org.apache.hadoop.hbase.backup.TestBackupManager
The HBaseClassTestRule ClassRule in
org.apache.hadoop.hbase.backup.TestBackupManager is for
org.apache.hadoop.hbase.backup.TestBackupSystemTable expected: but was:
{code}
Looks like copy-paste error.
Add license header to ExclusiveOperationException, please.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Assignee: Vladimir Rodionov
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch, HBASE-19441-v2.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405790#comment-16405790
]
Hadoop QA commented on HBASE-19441:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 23s{color}
| {color:red} hbase-backup generated 2 new + 62 unchanged - 0 fixed = 64 total
(was 62) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 17 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
11s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 17s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.5 2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 17m 33s{color}
| {color:red} hbase-backup in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
10s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 40s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.backup.TestBackupManager |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19441 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12915241/HBASE-19441-v2.patch |
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 65336e2d69ce 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12
21:23:04 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 2a3f4a0a4e |
| maven | version: Apache Maven 3.5.3
(3383c37e1f9e9b3bc3df5050c29c8aff9f295297; 2018-02-24T19:49:05Z) |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC3 |
| javac |
https://builds.apache.org/job/PreCommit-HBASE-Build/12026/artifact/patchprocess/diff-compile-javac-hbase-backup.txt
|
| whitespace |
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16400736#comment-16400736
]
Ted Yu commented on HBASE-19441:
Comparing the text of exception is vulnerable to future change w.r.t. the
message.
You can create a new exception class and check against the (new) class.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399704#comment-16399704
]
Ted Yu commented on HBASE-19441:
{code}
+ public final static String BACKUP_EXCLUSIVE_OPERATION_TIMEOUT_KEY =
+ "hbase.backup.exclusive.op.timeout";
{code}
You can add '.second' as suffix to the key name.
{code}
+ } catch (InterruptedException e1) {
+ }
{code}
Restore interrupt status in the catch block.
Is it possible to add a test ?
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399676#comment-16399676
]
Vladimir Rodionov commented on HBASE-19441:
---
Patch v1 is ready. It adds simple retry logic and timeout to acquiring backup
system table exclusive lock. No procV2 is required or needed.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backuprestore
>Reporter: Josh Elser
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-19441-v1.patch
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281070#comment-16281070
]
Ted Yu commented on HBASE-19441:
bq. reasonable to move things to procedure framework.
+1
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backup
>Reporter: Josh Elser
> Fix For: 3.0.0
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19441) Implement retry logic around starting exclusive backup operation
[
https://issues.apache.org/jira/browse/HBASE-19441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280819#comment-16280819
]
Appy commented on HBASE-19441:
--
bq. Remember that backups are client driven (per some design review from a long
time ago), so queuing is tough to reason about (we have no "centralized"
execution system to use).
We have centralized execution system - procedureV2. Queueing is totally
possible.
We have good amount of time before 2.1, wouldn't it be reasonable to move
things to procedure framework. The benefits would be- Backup operation can
avoid using BackupMetaTable to maintain intermediate state (since procs have
support for state, WAL, recovery on crash, etc). That in turn may help get rid
of snapshot-restore of backuptable. Which means multiple backups can progress
in parallel.
> Implement retry logic around starting exclusive backup operation
>
>
> Key: HBASE-19441
> URL: https://issues.apache.org/jira/browse/HBASE-19441
> Project: HBase
> Issue Type: Improvement
> Components: backup
>Reporter: Josh Elser
> Fix For: 3.0.0
>
>
> {quote}
> Specifically, the client does a checkAndPut to specifics coordinates in the
> backup table and throws an exception when that fails. Remember that backups
> are client driven (per some design review from a long time ago), so queuing
> is tough to reason about (we have no "centralized" execution system to use).
> At a glance, it seems pretty straightforward to add some retry/backoff
> semantics to BackupSystemTable#startBackupExclusiveOperation().
> {quote}
> While we are in a state in which backup operations cannot be executed in
> parallel, it would be nice to provide some retry logic + configuration. This
> would alleviate users from having to build this themselves.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
