[jira] [Updated] (HBASE-13468) hbase.zookeeper.quorum supports ipv6 address
[ https://issues.apache.org/jira/browse/HBASE-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling updated HBASE-13468: Attachment: HBASE-13468.master.001.patch > hbase.zookeeper.quorum supports ipv6 address > > > Key: HBASE-13468 > URL: https://issues.apache.org/jira/browse/HBASE-13468 > Project: HBase > Issue Type: Bug >Reporter: Mingtao Zhang >Assignee: maoling >Priority: Major > Attachments: HBASE-13468.master.001.patch > > > I put ipv6 address in hbase.zookeeper.quorum, by the time this string went to > zookeeper code, the address is messed up, i.e. only '[1234' left. > I started using pseudo mode with embedded zk = true. > I downloaded 1.0.0, not sure which affected version should be here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21291) Add a test for bypassing stuck state-machine procedures
[
https://issues.apache.org/jira/browse/HBASE-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648834#comment-16648834
]
Hadoop QA commented on HBASE-21291:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
48s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 13 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red} 0m
10s{color} | {color:red} patch has 7 errors when building our shaded downstream
artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 41s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color}
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}129m 29s{color}
| {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
46s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}180m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestBlockEvictionFromClient |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21291 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943746/HBASE-21291.master.005.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 4045ce499342 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Li
[jira] [Updated] (HBASE-13468) hbase.zookeeper.quorum supports ipv6 address
[ https://issues.apache.org/jira/browse/HBASE-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling updated HBASE-13468: Component/s: (was: Zookeeper) > hbase.zookeeper.quorum supports ipv6 address > > > Key: HBASE-13468 > URL: https://issues.apache.org/jira/browse/HBASE-13468 > Project: HBase > Issue Type: Bug >Reporter: Mingtao Zhang >Assignee: maoling >Priority: Major > Attachments: HBASE-13468.master.001.patch > > > I put ipv6 address in hbase.zookeeper.quorum, by the time this string went to > zookeeper code, the address is messed up, i.e. only '[1234' left. > I started using pseudo mode with embedded zk = true. > I downloaded 1.0.0, not sure which affected version should be here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13468) hbase.zookeeper.quorum supports ipv6 address
[ https://issues.apache.org/jira/browse/HBASE-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling updated HBASE-13468: Component/s: Zookeeper > hbase.zookeeper.quorum supports ipv6 address > > > Key: HBASE-13468 > URL: https://issues.apache.org/jira/browse/HBASE-13468 > Project: HBase > Issue Type: Bug >Reporter: Mingtao Zhang >Assignee: maoling >Priority: Major > Attachments: HBASE-13468.master.001.patch > > > I put ipv6 address in hbase.zookeeper.quorum, by the time this string went to > zookeeper code, the address is messed up, i.e. only '[1234' left. > I started using pseudo mode with embedded zk = true. > I downloaded 1.0.0, not sure which affected version should be here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13468) hbase.zookeeper.quorum supports ipv6 address
[ https://issues.apache.org/jira/browse/HBASE-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling updated HBASE-13468: Status: Patch Available (was: Open) > hbase.zookeeper.quorum supports ipv6 address > > > Key: HBASE-13468 > URL: https://issues.apache.org/jira/browse/HBASE-13468 > Project: HBase > Issue Type: Bug >Reporter: Mingtao Zhang >Assignee: maoling >Priority: Major > Attachments: HBASE-13468.master.001.patch > > > I put ipv6 address in hbase.zookeeper.quorum, by the time this string went to > zookeeper code, the address is messed up, i.e. only '[1234' left. > I started using pseudo mode with embedded zk = true. > I downloaded 1.0.0, not sure which affected version should be here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21278:
--
Attachment: HBASE-21278.patch
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Priority: Major
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21242) [amv2] Miscellaneous minor log and assign procedure create improvements
[
https://issues.apache.org/jira/browse/HBASE-21242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648850#comment-16648850
]
Hadoop QA commented on HBASE-21242:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
10s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
28s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
18s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
54s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
15s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
18s{color} | {color:red} hbase-server: The patch generated 1 new + 7 unchanged
- 0 fixed = 8 total (was 7) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
54s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 31s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
18s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
33s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}259m 9s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}326m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.master.procedure.TestServerCrashProcedure |
| | hadoop.hbase.coprocessor.TestRegionObserverInterface |
| | hadoop.hbase.TestFullLogReconstruction |
| | hadoop.hbase.regionserver.wal.TestSecureAsyncWALReplay |
| | hadoop.hbase.master.TestRollingRestart |
| | hadoop.hbase.TestIOFencing |
| | hadoop.hbase.regionserver.wal.TestAsyncWALReplayCompressed |
| | hadoop.hbase.regionserv
[jira] [Updated] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Duo Zhang updated HBASE-21278:
--
Assignee: Duo Zhang
Fix Version/s: 2.2.0
3.0.0
Status: Patch Available (was: Open)
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648852#comment-16648852
]
Duo Zhang commented on HBASE-21278:
---
Review board link:
https://reviews.apache.org/r/69014/
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648853#comment-16648853
]
Duo Zhang commented on HBASE-21278:
---
Ping [~stack] for reviewing. The old rollback implementation for
MergeTableRegionsProcedure and SplitTableRegionProcedure does not work,
actually...
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21309) Increase the waiting timeout for TestProcedurePriority
Duo Zhang created HBASE-21309: - Summary: Increase the waiting timeout for TestProcedurePriority Key: HBASE-21309 URL: https://issues.apache.org/jira/browse/HBASE-21309 Project: HBase Issue Type: Sub-task Components: test Reporter: Duo Zhang On loaded jenkins nodes it will be easily timed out, so let's increase the timeout value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21309) Increase the waiting timeout for TestProcedurePriority
[ https://issues.apache.org/jira/browse/HBASE-21309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21309: -- Attachment: HBASE-21309.patch > Increase the waiting timeout for TestProcedurePriority > -- > > Key: HBASE-21309 > URL: https://issues.apache.org/jira/browse/HBASE-21309 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Priority: Major > Attachments: HBASE-21309.patch > > > On loaded jenkins nodes it will be easily timed out, so let's increase the > timeout value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21309) Increase the waiting timeout for TestProcedurePriority
[ https://issues.apache.org/jira/browse/HBASE-21309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21309: -- Assignee: Duo Zhang Fix Version/s: 2.0.3 2.1.1 2.2.0 3.0.0 Status: Patch Available (was: Open) > Increase the waiting timeout for TestProcedurePriority > -- > > Key: HBASE-21309 > URL: https://issues.apache.org/jira/browse/HBASE-21309 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-21309.patch > > > On loaded jenkins nodes it will be easily timed out, so let's increase the > timeout value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-13468) hbase.zookeeper.quorum supports ipv6 address
[
https://issues.apache.org/jira/browse/HBASE-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648872#comment-16648872
]
Hadoop QA commented on HBASE-13468:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
27s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
38s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 25s{color}
| {color:red} hbase-zookeeper generated 2 new + 37 unchanged - 0 fixed = 39
total (was 37) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} hbase-common: The patch generated 1 new + 3 unchanged
- 0 fixed = 4 total (was 3) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} hbase-zookeeper: The patch generated 4 new + 0
unchanged - 0 fixed = 4 total (was 0) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
28s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
14m 36s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
41s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
50s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-13468 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943755/HBASE-13468.master.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 39934bcf2018 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
[jira] [Updated] (HBASE-21291) Add a test for bypassing stuck state-machine procedures
[
https://issues.apache.org/jira/browse/HBASE-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingyun Tian updated HBASE-21291:
-
Attachment: (was: HBASE-21291.master.005.patch)
> Add a test for bypassing stuck state-machine procedures
> ---
>
> Key: HBASE-21291
> URL: https://issues.apache.org/jira/browse/HBASE-21291
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Attachments: HBASE-21291.master.001.patch,
> HBASE-21291.master.002.patch, HBASE-21291.master.003.patch,
> HBASE-21291.master.004.patch
>
>
> {code}
> if (!procedure.isFailed()) {
> if (subprocs != null) {
> if (subprocs.length == 1 && subprocs[0] == procedure) {
> // Procedure returned itself. Quick-shortcut for a state
> machine-like procedure;
> // i.e. we go around this loop again rather than go back out on
> the scheduler queue.
> subprocs = null;
> reExecute = true;
> LOG.trace("Short-circuit to next step on pid={}",
> procedure.getProcId());
> } else {
> // Yield the current procedure, and make the subprocedure runnable
> // subprocs may come back 'null'.
> subprocs = initializeChildren(procStack, procedure, subprocs);
> LOG.info("Initialized subprocedures=" +
> (subprocs == null? null:
> Stream.of(subprocs).map(e -> "{" + e.toString() + "}").
> collect(Collectors.toList()).toString()));
> }
> } else if (procedure.getState() == ProcedureState.WAITING_TIMEOUT) {
> LOG.debug("Added to timeoutExecutor {}", procedure);
> timeoutExecutor.add(procedure);
> } else if (!suspended) {
> // No subtask, so we are done
> procedure.setState(ProcedureState.SUCCESS);
> }
> }
> {code}
> Currently implementation of ProcedureExecutor will set the reExcecute to true
> for state machine like procedure. Then if this procedure is stuck at one
> certain state, it will loop forever.
> {code}
> IdLock.Entry lockEntry =
> procExecutionLock.getLockEntry(proc.getProcId());
> try {
> executeProcedure(proc);
> } catch (AssertionError e) {
> LOG.info("ASSERT pid=" + proc.getProcId(), e);
> throw e;
> } finally {
> procExecutionLock.releaseLockEntry(lockEntry);
> {code}
> Since procedure will get the IdLock and release it after execution done,
> state machine procedure will never release IdLock until it is finished.
> Then bypassProcedure doesn't work because is will try to grab the IdLock at
> first.
> {code}
> IdLock.Entry lockEntry =
> procExecutionLock.tryLockEntry(procedure.getProcId(), lockWait);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21291) Add a test for bypassing stuck state-machine procedures
[
https://issues.apache.org/jira/browse/HBASE-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingyun Tian updated HBASE-21291:
-
Attachment: HBASE-21291.master.005.patch
> Add a test for bypassing stuck state-machine procedures
> ---
>
> Key: HBASE-21291
> URL: https://issues.apache.org/jira/browse/HBASE-21291
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Jingyun Tian
>Assignee: Jingyun Tian
>Priority: Major
> Attachments: HBASE-21291.master.001.patch,
> HBASE-21291.master.002.patch, HBASE-21291.master.003.patch,
> HBASE-21291.master.004.patch, HBASE-21291.master.005.patch
>
>
> {code}
> if (!procedure.isFailed()) {
> if (subprocs != null) {
> if (subprocs.length == 1 && subprocs[0] == procedure) {
> // Procedure returned itself. Quick-shortcut for a state
> machine-like procedure;
> // i.e. we go around this loop again rather than go back out on
> the scheduler queue.
> subprocs = null;
> reExecute = true;
> LOG.trace("Short-circuit to next step on pid={}",
> procedure.getProcId());
> } else {
> // Yield the current procedure, and make the subprocedure runnable
> // subprocs may come back 'null'.
> subprocs = initializeChildren(procStack, procedure, subprocs);
> LOG.info("Initialized subprocedures=" +
> (subprocs == null? null:
> Stream.of(subprocs).map(e -> "{" + e.toString() + "}").
> collect(Collectors.toList()).toString()));
> }
> } else if (procedure.getState() == ProcedureState.WAITING_TIMEOUT) {
> LOG.debug("Added to timeoutExecutor {}", procedure);
> timeoutExecutor.add(procedure);
> } else if (!suspended) {
> // No subtask, so we are done
> procedure.setState(ProcedureState.SUCCESS);
> }
> }
> {code}
> Currently implementation of ProcedureExecutor will set the reExcecute to true
> for state machine like procedure. Then if this procedure is stuck at one
> certain state, it will loop forever.
> {code}
> IdLock.Entry lockEntry =
> procExecutionLock.getLockEntry(proc.getProcId());
> try {
> executeProcedure(proc);
> } catch (AssertionError e) {
> LOG.info("ASSERT pid=" + proc.getProcId(), e);
> throw e;
> } finally {
> procExecutionLock.releaseLockEntry(lockEntry);
> {code}
> Since procedure will get the IdLock and release it after execution done,
> state machine procedure will never release IdLock until it is finished.
> Then bypassProcedure doesn't work because is will try to grab the IdLock at
> first.
> {code}
> IdLock.Entry lockEntry =
> procExecutionLock.tryLockEntry(procedure.getProcId(), lockWait);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21271) [amv2] Don't throw UnsupportedOperationException when rollback called on Assign/Unassign; spiral of death
[
https://issues.apache.org/jira/browse/HBASE-21271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648884#comment-16648884
]
Hudson commented on HBASE-21271:
Results for branch branch-2.1
[build #459 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/459/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/459//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/459//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/459//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [amv2] Don't throw UnsupportedOperationException when rollback called on
> Assign/Unassign; spiral of death
> -
>
> Key: HBASE-21271
> URL: https://issues.apache.org/jira/browse/HBASE-21271
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.1.1, 2.0.3
>
> Attachments: HBASE-21271.branch-2.1.001.patch,
> HBASE-21271.branch-2.1.002.patch
>
>
> I can't repro reliably but if an AssignProcedure or UnassignProcedure is a
> subprocedure of an Enable/Disable and for whatever reason the parent decides
> it needs to rollback -- can't get an entity lock -- it will ask the
> subprocedures to rollback. UP and AP don't support rollback on all steps. For
> steps where not supported, we have been throwing a
> UnsupportedOperationException The Framework reschedules the rollback. And
> so on filling logs and Procedure WALs.
> Instead just note no rollback supported and intervention may be needed (until
> we to to 2.2 when AP/UP go away).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21098) Improve Snapshot Performance with Temporary Snapshot Directory when rootDir on S3
[
https://issues.apache.org/jira/browse/HBASE-21098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648892#comment-16648892
]
Hudson commented on HBASE-21098:
Results for branch branch-1
[build #506 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/506/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/506//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/506//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-1/506//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Improve Snapshot Performance with Temporary Snapshot Directory when rootDir
> on S3
> -
>
> Key: HBASE-21098
> URL: https://issues.apache.org/jira/browse/HBASE-21098
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 3.0.0, 1.4.8, 2.1.1
>Reporter: Tyler Mi
>Assignee: Tyler Mi
>Priority: Major
> Fix For: 3.0.0, 1.5.0, 2.2.0
>
> Attachments: HBASE-21098.branch-1.001.patch,
> HBASE-21098.branch-1.002.patch, HBASE-21098.master.001.patch,
> HBASE-21098.master.002.patch, HBASE-21098.master.003.patch,
> HBASE-21098.master.004.patch, HBASE-21098.master.005.patch,
> HBASE-21098.master.006.patch, HBASE-21098.master.007.patch,
> HBASE-21098.master.008.patch, HBASE-21098.master.009.patch,
> HBASE-21098.master.010.patch, HBASE-21098.master.011.patch,
> HBASE-21098.master.012.patch, HBASE-21098.master.013.patch
>
>
> When using Apache HBase, the snapshot feature can be used to make a point in
> time recovery. To do this, HBase creates a manifest of all the files in all
> of the Regions so that those files can be referenced again when a user
> restores a snapshot. With HBase's S3 storage mode, developers can store their
> data off-cluster on Amazon S3. However, utilizing S3 as a file system is
> inefficient in some operations, namely renames. Most Hadoop ecosystem
> applications use an atomic rename as a method of committing data. However,
> with S3, a rename is a separate copy and then a delete of every file which is
> no longer atomic and, in fact, quite costly. In addition, puts and deletes on
> S3 have latency issues that traditional filesystems do not encounter when
> manipulating the region snapshots to consolidate into a single manifest. When
> HBase on S3 users have a significant amount of regions, puts, deletes, and
> renames (the final commit stage of the snapshot) become the bottleneck
> causing snapshots to take many minutes or even hours to complete.
> The purpose of this patch is to increase the overall performance of snapshots
> while utilizing HBase on S3 through the use of a temporary directory for the
> snapshots that exists on a traditional filesystem like HDFS to circumvent the
> bottlenecks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-21310) Split TestCloneSnapshotFromClient
Duo Zhang created HBASE-21310: - Summary: Split TestCloneSnapshotFromClient Key: HBASE-21310 URL: https://issues.apache.org/jira/browse/HBASE-21310 Project: HBase Issue Type: Sub-task Components: test Reporter: Duo Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21311) Split TestRestoreSnapshotFromClient
[ https://issues.apache.org/jira/browse/HBASE-21311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21311: -- Component/s: test > Split TestRestoreSnapshotFromClient > --- > > Key: HBASE-21311 > URL: https://issues.apache.org/jira/browse/HBASE-21311 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-21311) Split TestRestoreSnapshotFromClient
Duo Zhang created HBASE-21311: - Summary: Split TestRestoreSnapshotFromClient Key: HBASE-21311 URL: https://issues.apache.org/jira/browse/HBASE-21311 Project: HBase Issue Type: Sub-task Reporter: Duo Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21271) [amv2] Don't throw UnsupportedOperationException when rollback called on Assign/Unassign; spiral of death
[
https://issues.apache.org/jira/browse/HBASE-21271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648901#comment-16648901
]
Hudson commented on HBASE-21271:
Results for branch branch-2.0
[build #944 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/944/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/944//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/944//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/944//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [amv2] Don't throw UnsupportedOperationException when rollback called on
> Assign/Unassign; spiral of death
> -
>
> Key: HBASE-21271
> URL: https://issues.apache.org/jira/browse/HBASE-21271
> Project: HBase
> Issue Type: Bug
> Components: amv2
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 2.1.1, 2.0.3
>
> Attachments: HBASE-21271.branch-2.1.001.patch,
> HBASE-21271.branch-2.1.002.patch
>
>
> I can't repro reliably but if an AssignProcedure or UnassignProcedure is a
> subprocedure of an Enable/Disable and for whatever reason the parent decides
> it needs to rollback -- can't get an entity lock -- it will ask the
> subprocedures to rollback. UP and AP don't support rollback on all steps. For
> steps where not supported, we have been throwing a
> UnsupportedOperationException The Framework reschedules the rollback. And
> so on filling logs and Procedure WALs.
> Instead just note no rollback supported and intervention may be needed (until
> we to to 2.2 when AP/UP go away).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21291) Add a test for bypassing stuck state-machine procedures
[
https://issues.apache.org/jira/browse/HBASE-21291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648902#comment-16648902
]
Hadoop QA commented on HBASE-21291:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
25s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 51s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
2s{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21291 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943764/HBASE-21291.master.005.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 702b3cfe2db5 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7464e2ef9d |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14688/testReport/ |
| Max. process+thread count | 283 (vs. ulimit of 1) |
| modules | C: hbase-procedure U: hbase-procedure |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/14688/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
[jira] [Commented] (HBASE-21289) Remove the log "'hbase.regionserver.maxlogs' was deprecated." in AbstractFSWAL
[
https://issues.apache.org/jira/browse/HBASE-21289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648904#comment-16648904
]
Hudson commented on HBASE-21289:
Results for branch master
[build #543 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/543/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove the log "'hbase.regionserver.maxlogs' was deprecated." in AbstractFSWAL
> --
>
> Key: HBASE-21289
> URL: https://issues.apache.org/jira/browse/HBASE-21289
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21289.master.001.patch
>
>
> This log was added by HBASE-14951. And the description and release note never
> said this config was deprecated. I thought HBASE-14951 only changed the
> default value of maxlogs (Please correct me if I am wrong). And we still use
> this config in our hbase book. So the log "'hbase.regionserver.maxlogs' was
> deprecated." in AbstractFSWAL is confused. Let's remove it. FYI [~vrodionov]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21114) website should have a copy of 2.1 release docs
[
https://issues.apache.org/jira/browse/HBASE-21114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648908#comment-16648908
]
Hudson commented on HBASE-21114:
Results for branch master
[build #543 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/543/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> website should have a copy of 2.1 release docs
> --
>
> Key: HBASE-21114
> URL: https://issues.apache.org/jira/browse/HBASE-21114
> Project: HBase
> Issue Type: Task
> Components: community, documentation
>Affects Versions: 2.1.0
>Reporter: Sean Busbey
>Assignee: Mike Drob
>Priority: Major
> Fix For: 2.1.1
>
> Attachments: HBASE-21114.patch
>
>
> in the "Documentation and API" menu we have entries for 2.0 and 1.2. should
> also add in 2.1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21299) List counts of actual region states in master UI tables section
[
https://issues.apache.org/jira/browse/HBASE-21299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648906#comment-16648906
]
Hudson commented on HBASE-21299:
Results for branch master
[build #543 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/543/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> List counts of actual region states in master UI tables section
> ---
>
> Key: HBASE-21299
> URL: https://issues.apache.org/jira/browse/HBASE-21299
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1
>
> Attachments: HBASE-21299.branch-2.1.001.patch, Screen Shot 2018-10-11
> at 9.44.39 PM.png, Screen Shot 2018-10-11 at 9.44.56 PM.png
>
>
> It the tables panel, we list Open Regions, Offline Regions, Failed Regions,
> Split Region, and Other. This is not very useful. It is from a time before
> region states were edited down. Better to list OPEN, CLOSED, OPENING,
> CLOSING...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21178) [BC break] : Get and Scan operation with a custom converter_class not working
[
https://issues.apache.org/jira/browse/HBASE-21178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648905#comment-16648905
]
Hudson commented on HBASE-21178:
Results for branch master
[build #543 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/543/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [BC break] : Get and Scan operation with a custom converter_class not working
> -
>
> Key: HBASE-21178
> URL: https://issues.apache.org/jira/browse/HBASE-21178
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 2.0.0
>Reporter: Subrat Mishra
>Assignee: Subrat Mishra
>Priority: Critical
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21178.master.001.patch,
> HBASE-21178.master.002.patch, HBASE-21178.master.003.patch
>
>
> Consider a simple scenario:
> {code:java}
> create 'foo', {NAME => 'f1'}
> put 'foo','r1','f1:a',1000
> get 'foo','r1',{COLUMNS =>
> ['f1:a:c(org.apache.hadoop.hbase.util.Bytes).len']}
> scan 'foo',{COLUMNS =>
> ['f1:a:c(org.apache.hadoop.hbase.util.Bytes).len']}{code}
> Both get and scan fails with ERROR
> {code:java}
> ERROR: wrong number of arguments (3 for 1) {code}
> Looks like in table.rb file converter_method expects 3 arguments [(bytes,
> offset, len)] since version 2.0.0, prior to version 2.0.0 it was taking only
> 1 argument [(bytes)]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21303) [shell] clear_deadservers with no args fails
[
https://issues.apache.org/jira/browse/HBASE-21303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648907#comment-16648907
]
Hudson commented on HBASE-21303:
Results for branch master
[build #543 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/543/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/543//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [shell] clear_deadservers with no args fails
>
>
> Key: HBASE-21303
> URL: https://issues.apache.org/jira/browse/HBASE-21303
> Project: HBase
> Issue Type: Improvement
>Reporter: Stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3
>
> Attachments: HBASE-21303.branch-2.1.001.patch
>
>
> Tried running clear_deadservers on tip of branch-2.1 and it fails
> {code}
> hbase(main):007:0> help "clear_deadservers"
> Clear the dead region servers that are never used.
> Examples:
> Clear all dead region servers:
> hbase> clear_deadservers
> Clear the specified dead region servers:
> hbase> clear_deadservers 'host187.example.com,60020,1289493121758'
> or
> hbase> clear_deadservers 'host187.example.com,60020,1289493121758',
>'host188.example.com,60020,1289493121758'
> hbase(main):008:0> clear_deadservers
> ERROR: servers cannot be null or empty
> For usage try 'help "clear_deadservers"'
> Took 0.0094 seconds
> hbase(main):009:0> clear_deadservers
> ERROR: servers cannot be null or empty
> For usage try 'help "clear_deadservers"'
> Took 0.0084 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21281) Update bouncycastle dependency.
[ https://issues.apache.org/jira/browse/HBASE-21281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648925#comment-16648925 ] Ted Yu commented on HBASE-21281: TestSaslFanOutOneBlockAsyncDFSOutput passed during QA run: https://builds.apache.org/job/PreCommit-HBASE-Build/14681/testReport/org.apache.hadoop.hbase.io.asyncfs/TestSaslFanOutOneBlockAsyncDFSOutput/ whereas it failed during master branch build: https://builds.apache.org/job/HBase%20Nightly/job/master/543//testReport/junit/org.apache.hadoop.hbase.io.asyncfs/TestSaslFanOutOneBlockAsyncDFSOutput/health_checks___yetus_jdk8_hadoop3_checks___org_apache_hadoop_hbase_io_asyncfs_TestSaslFanOutOneBlockAsyncDFSOutput/ > Update bouncycastle dependency. > --- > > Key: HBASE-21281 > URL: https://issues.apache.org/jira/browse/HBASE-21281 > Project: HBase > Issue Type: Task > Components: dependencies, test >Reporter: Josh Elser >Assignee: Josh Elser >Priority: Major > Fix For: 3.0.0, 2.2.0 > > Attachments: 21281.addendum.patch, HBASE-21281.001.branch-2.0.patch > > > Looks like we still depend on bcprov-jdk16 for some x509 certificate > generation in our tests. Bouncycastle has moved beyond this in 1.47, changing > the artifact names. > [http://www.bouncycastle.org/wiki/display/JA1/Porting+from+earlier+BC+releases+to+1.47+and+later] > There are some API changes too, but it looks like we don't use any of these. > It seems like we also have vestiges in the POMs from when we were depending > on a specific BC version that came in from Hadoop. We now have a > KeyStoreTestUtil class in HBase, which makes me think we can also clean up > some dependencies. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-13468) hbase.zookeeper.quorum supports ipv6 address
[ https://issues.apache.org/jira/browse/HBASE-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maoling updated HBASE-13468: Attachment: HBASE-13468.master.002.patch > hbase.zookeeper.quorum supports ipv6 address > > > Key: HBASE-13468 > URL: https://issues.apache.org/jira/browse/HBASE-13468 > Project: HBase > Issue Type: Bug >Reporter: Mingtao Zhang >Assignee: maoling >Priority: Major > Attachments: HBASE-13468.master.001.patch, > HBASE-13468.master.002.patch > > > I put ipv6 address in hbase.zookeeper.quorum, by the time this string went to > zookeeper code, the address is messed up, i.e. only '[1234' left. > I started using pseudo mode with embedded zk = true. > I downloaded 1.0.0, not sure which affected version should be here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HBASE-21096) TestZKSecretWatcher::testKeyUpdate() fails intermittently on branch-2
[
https://issues.apache.org/jira/browse/HBASE-21096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maoling reassigned HBASE-21096:
---
Assignee: maoling
> TestZKSecretWatcher::testKeyUpdate() fails intermittently on branch-2
> -
>
> Key: HBASE-21096
> URL: https://issues.apache.org/jira/browse/HBASE-21096
> Project: HBase
> Issue Type: Bug
> Components: test, Zookeeper
>Affects Versions: 2.1.1
>Reporter: Mingliang Liu
>Assignee: maoling
>Priority: Major
>
> Failing Jenkins and the error message is like:
> {code}
> [ERROR] Failures:
> [ERROR]
> org.apache.hadoop.hbase.security.token.TestZKSecretWatcher.testKeyUpdate(org.apache.hadoop.hbase.security.token.TestZKSecretWatcher)
> [ERROR] Run 1: TestZKSecretWatcher.testKeyUpdate:185 expected null, but
> was:
> [ERROR] Run 2: TestZKSecretWatcher.testKeyUpdate:154
> expected: but
> was:
> [ERROR] Run 3: TestZKSecretWatcher.testKeyUpdate:154
> expected: but
> was:
> {code}
> It can pass locally for multiple runs. Reported also by [HBASE-21074] in
> [comment|https://issues.apache.org/jira/browse/HBASE-21074?focusedCommentId=16588200&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16588200].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648947#comment-16648947
]
Hadoop QA commented on HBASE-21278:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
13s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
23s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
10m 38s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 59s{color}
| {color:red} hbase-procedure in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}130m 43s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}192m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.procedure2.TestProcedureExecution |
| | hadoop.hbase.procedure2.TestStateMachineProcedure |
| | hadoop.hbase.master.procedure.TestModifyTableProcedure |
| | hadoop.hbase.master.procedure.TestDeleteNamespaceProcedure |
| | hadoop.hbase.master.procedure.TestEnableTableProcedure |
| | hadoop.hbase.master.procedure.TestCloneSnapshotProcedure |
| | hadoop.hbase.master.procedure.TestModifyNamespaceProcedure |
| | hadoop.hbase.master.assignment.TestRegionMoveAndAbandon |
| | hadoop.hbase.master.procedure.TestCreateTableProcedure |
| | hadoop.hbase.master.procedure.TestCreateNamespaceProcedure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21278 |
|
[jira] [Commented] (HBASE-13468) hbase.zookeeper.quorum supports ipv6 address
[
https://issues.apache.org/jira/browse/HBASE-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648950#comment-16648950
]
Hadoop QA commented on HBASE-13468:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} hbase-zookeeper generated 0 new + 35 unchanged - 2
fixed = 35 total (was 37) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
28s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 6s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
31s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
47s{color} | {color:green} hbase-zookeeper in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-13468 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943777/HBASE-13468.master.002.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 99afb9e7a130 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Bui
[jira] [Commented] (HBASE-13468) hbase.zookeeper.quorum supports ipv6 address
[ https://issues.apache.org/jira/browse/HBASE-13468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648955#comment-16648955 ] maoling commented on HBASE-13468: - Review Board:https://reviews.apache.org/r/69013/ > hbase.zookeeper.quorum supports ipv6 address > > > Key: HBASE-13468 > URL: https://issues.apache.org/jira/browse/HBASE-13468 > Project: HBase > Issue Type: Bug >Reporter: Mingtao Zhang >Assignee: maoling >Priority: Major > Attachments: HBASE-13468.master.001.patch, > HBASE-13468.master.002.patch > > > I put ipv6 address in hbase.zookeeper.quorum, by the time this string went to > zookeeper code, the address is messed up, i.e. only '[1234' left. > I started using pseudo mode with embedded zk = true. > I downloaded 1.0.0, not sure which affected version should be here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21309) Increase the waiting timeout for TestProcedurePriority
[
https://issues.apache.org/jira/browse/HBASE-21309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648980#comment-16648980
]
Hadoop QA commented on HBASE-21309:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
53s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
28s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
11m 22s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}177m 51s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}224m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21309 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12943762/HBASE-21309.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c6308fac4704 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 7464e2ef9d |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC3 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/14687/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/14687/testReport/ |
| Max. process+thread count | 4904 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/14687/console |
| Powered by | A
[jira] [Commented] (HBASE-21301) Heatmap for key access patterns
[ https://issues.apache.org/jira/browse/HBASE-21301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648982#comment-16648982 ] Allan Yang commented on HBASE-21301: Very nice feature, looking forward to it. I have a suggest, we can keep some statistics at data block level. For example, we can record the hit count for a certain data block and keep the data in a memory structure. So that we can generate a heatmap for data block. I think it can narrow down the hot key in a smaller granularity than hfile range,which is too big. Another thing is that if we need to write the statistics to another table, I'm afraid it will decrease the read performance dramatically, write them into a local memory structure is more doable I think. > Heatmap for key access patterns > --- > > Key: HBASE-21301 > URL: https://issues.apache.org/jira/browse/HBASE-21301 > Project: HBase > Issue Type: Improvement >Reporter: Archana Katiyar >Assignee: Archana Katiyar >Priority: Major > > Google recently released a beta feature for Cloud Bigtable which presents a > heat map of the keyspace. *Given how hotspotting comes up now and again here, > this is a good idea for giving HBase ops a tool to be proactive about it.* > >>> > Additionally, we are announcing the beta version of Key Visualizer, a > visualization tool for Cloud Bigtable key access patterns. Key Visualizer > helps debug performance issues due to unbalanced access patterns across the > key space, or single rows that are too large or receiving too much read or > write activity. With Key Visualizer, you get a heat map visualization of > access patterns over time, along with the ability to zoom into specific key > or time ranges, or select a specific row to find the full row key ID that's > responsible for a hotspot. Key Visualizer is automatically enabled for Cloud > Bigtable clusters with sufficient data or activity, and does not affect Cloud > Bigtable cluster performance. > <<< > From > [https://cloudplatform.googleblog.com/2018/07/on-gcp-your-database-your-way.html] > (Copied this description from the write-up by [~apurtell], thanks Andrew.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648992#comment-16648992
]
Allan Yang commented on HBASE-21278:
Just reviewed the patch, +1 for it. I think this patch can go to branch-2.0 and
branch-2.1 also.
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16648992#comment-16648992
]
Allan Yang edited comment on HBASE-21278 at 10/13/18 3:43 PM:
--
Just reviewed the patch, +1 for it. I like the idea that do not rollback the
already successful ones. But seems like some UTs are fail, still need to dig? I
think this patch can go to branch-2.0 and branch-2.1 also.
was (Author: allan163):
Just reviewed the patch, +1 for it. I think this patch can go to branch-2.0 and
branch-2.1 also.
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21301) Heatmap for key access patterns
[ https://issues.apache.org/jira/browse/HBASE-21301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649006#comment-16649006 ] stack commented on HBASE-21301: --- Thanks for working on this [~archana.katiyar]. Looks great. High-level comments: bq. Store data in a HBase table (new system table) We have poor support for system tables currently. The hbase:namespace is a system table that has caused us grief at assign time with a good while now. The 'fix' is to integrate it into the hbase:meta; i.e. have one system table only that does all. Keeping stats is a long-time request so excited to see progress. If they need to persist, then they'll need a dedicated table. Does it have to be a 'system' table? Does it have to be assigned before usertables? Or can stats recording go on though the table is not deployed promptly? Or perhaps for v1, do as [~allan163] suggests keeping in-memory circular buffers. Any visualization at all even if just a report on the last few minutes of an in-memory buffer would be a big massive win (smile). Keeping stats, may I suggest taking a look at https://datasketches.github.io/ Thanks. > Heatmap for key access patterns > --- > > Key: HBASE-21301 > URL: https://issues.apache.org/jira/browse/HBASE-21301 > Project: HBase > Issue Type: Improvement >Reporter: Archana Katiyar >Assignee: Archana Katiyar >Priority: Major > > Google recently released a beta feature for Cloud Bigtable which presents a > heat map of the keyspace. *Given how hotspotting comes up now and again here, > this is a good idea for giving HBase ops a tool to be proactive about it.* > >>> > Additionally, we are announcing the beta version of Key Visualizer, a > visualization tool for Cloud Bigtable key access patterns. Key Visualizer > helps debug performance issues due to unbalanced access patterns across the > key space, or single rows that are too large or receiving too much read or > write activity. With Key Visualizer, you get a heat map visualization of > access patterns over time, along with the ability to zoom into specific key > or time ranges, or select a specific row to find the full row key ID that's > responsible for a hotspot. Key Visualizer is automatically enabled for Cloud > Bigtable clusters with sufficient data or activity, and does not affect Cloud > Bigtable cluster performance. > <<< > From > [https://cloudplatform.googleblog.com/2018/07/on-gcp-your-database-your-way.html] > (Copied this description from the write-up by [~apurtell], thanks Andrew.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649033#comment-16649033
]
stack commented on HBASE-21278:
---
Left a few notes up on rb. Do we have to go through all Procedures and see how
they do with this new philosophy where the parent runs the rollback skipping
rollback of the subprocedures?
bq. The old rollback implementation for MergeTableRegionsProcedure and
SplitTableRegionProcedure does not work, actually
Say more. What to do here?
Thanks [~Apache9]
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649163#comment-16649163
]
Duo Zhang commented on HBASE-21278:
---
I've replied on RB, the old implementation is not idempotent. We will be in
trouble if there are crashes... The current testRollbackAndDoubleExecution can
cover this problem.
Let me see the failed UTs.
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21309) Increase the waiting timeout for TestProcedurePriority
[ https://issues.apache.org/jira/browse/HBASE-21309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649178#comment-16649178 ] Duo Zhang commented on HBASE-21309: --- Pushed to master first. Let's see how it works. > Increase the waiting timeout for TestProcedurePriority > -- > > Key: HBASE-21309 > URL: https://issues.apache.org/jira/browse/HBASE-21309 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.1.1, 2.0.3 > > Attachments: HBASE-21309.patch > > > On loaded jenkins nodes it will be easily timed out, so let's increase the > timeout value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21278) Do not rollback successful sub procedures when rolling back a procedure
[
https://issues.apache.org/jira/browse/HBASE-21278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649184#comment-16649184
]
Duo Zhang commented on HBASE-21278:
---
For TestEnableTableProcedure, it is because that I changed the implementation
of testRollbackAndDoubleExecution, and we used to pass 0 as the last step so it
is broken. But anyway the old UT just tests nothing, as we will quit
immediately... I'm afraid there is big hole in our UTs for testing rollback...
Will be back later after I fix all the failed UTs...
> Do not rollback successful sub procedures when rolling back a procedure
> ---
>
> Key: HBASE-21278
> URL: https://issues.apache.org/jira/browse/HBASE-21278
> Project: HBase
> Issue Type: Bug
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: HBASE-21278.patch,
> org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt
>
>
> https://builds.apache.org/job/HBase-Flaky-Tests/job/master/1235/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure-output.txt/*view*/
> I think the problem is
> {noformat}
> 2018-10-08 03:44:30,315 INFO [PEWorker-1]
> procedure.MasterProcedureScheduler(689): pid=43, ppid=42, state=SUCCESS,
> hasLock=false; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN checking lock on
> 9bac7c539ac0cff6dc5706ed375a3bfb
> 2018-10-08 03:44:30,320 ERROR [PEWorker-1] helpers.MarkerIgnoringBase(159):
> CODE-BUG: Uncaught runtime exception for pid=43, ppid=42, state=SUCCESS,
> hasLock=true; TransitRegionStateProcedure
> table=testRollbackAndDoubleExecution,
> region=9bac7c539ac0cff6dc5706ed375a3bfb, UNASSIGN
> java.lang.UnsupportedOperationException
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:458)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.rollbackState(TransitRegionStateProcedure.java:97)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doRollback(Procedure.java:957)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1605)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1567)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1446)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$900(ProcedureExecutor.java:76)
> {noformat}
> Typically there is no rollback for TRSP. Need to dig more.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21301) Heatmap for key access patterns
[ https://issues.apache.org/jira/browse/HBASE-21301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649249#comment-16649249 ] Archana Katiyar commented on HBASE-21301: - Thanks [~stack] and [~allan163] for your suggestions. We need to store stats for at-least couple of weeks; this is because we have observed cases in past where we have come to know that there was hotspotting, only after 1-2 weeks (this includes the time taken by the affected team to report the case, then SR team takes their own time, then our prod support takes some time to figure out the right direction to debug etc etc.). If there is continuous hot-spotting then things are easy to handle; but typically it happens when usage pattern changes suddenly (for example, huge data copy because of data migration). And until we realize that something went wrong, usage pattern comes back to normal. Also, just to add to the grief - typically affected team is not the one who caused the problem at the first hand. This is why we were thinking of having 30 days TTL for the stats data. With this much persistence requirement, keeping everything in memory doesn't look like a feasible option, specially when we start collecting stats at store file (or further at data block level). If system table is not an feasible option, then we can think of any of the following options - * Store data in a usertable (I am hoping, user tables have less overhead than system tables.); we can make sure to create this table from hbase start shell script. We need to have some fallback implementation though in case this table is not available. * Use a underlying file in HDFS (of local disc); implement some sort of append only write path. We also need to implement housekeeping to delete old data. Basically, we need a circular linked list type of storage which provides persistence as well. * Use memory mapped files (mentioned in [https://github.com/DataSketches/memory] ) and use it with underlying storage. Also, to answer [~allan163]'s concern of reducing read perf - since I am planning to record the stats periodically (like every 15 mins) and there is no additional call or statement per read operation (I am planning to to use existing counters only to collect region level stats), there should not be any observable perf difference. In future, when we extend it to store file level or block level, then we have to be vigilant about doing it at the right place and time. Comments\Suggestions\Thoughts are welcome. > Heatmap for key access patterns > --- > > Key: HBASE-21301 > URL: https://issues.apache.org/jira/browse/HBASE-21301 > Project: HBase > Issue Type: Improvement >Reporter: Archana Katiyar >Assignee: Archana Katiyar >Priority: Major > > Google recently released a beta feature for Cloud Bigtable which presents a > heat map of the keyspace. *Given how hotspotting comes up now and again here, > this is a good idea for giving HBase ops a tool to be proactive about it.* > >>> > Additionally, we are announcing the beta version of Key Visualizer, a > visualization tool for Cloud Bigtable key access patterns. Key Visualizer > helps debug performance issues due to unbalanced access patterns across the > key space, or single rows that are too large or receiving too much read or > write activity. With Key Visualizer, you get a heat map visualization of > access patterns over time, along with the ability to zoom into specific key > or time ranges, or select a specific row to find the full row key ID that's > responsible for a hotspot. Key Visualizer is automatically enabled for Cloud > Bigtable clusters with sufficient data or activity, and does not affect Cloud > Bigtable cluster performance. > <<< > From > [https://cloudplatform.googleblog.com/2018/07/on-gcp-your-database-your-way.html] > (Copied this description from the write-up by [~apurtell], thanks Andrew.) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20952) Re-visit the WAL API
[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649253#comment-16649253
]
Hudson commented on HBASE-20952:
Results for branch HBASE-20952
[build #17 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/17/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/17//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/17//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-20952/17//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Re-visit the WAL API
>
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: Josh Elser
>Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup&restore. Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B&R doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
