[jira] [Commented] (HBASE-16094) Procedure v2 - Improve cleaning up of proc wals
[
https://issues.apache.org/jira/browse/HBASE-16094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423998#comment-15423998
]
Hadoop QA commented on HBASE-16094:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 17s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
42s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
29m 10s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 51s {color}
| {color:red} hbase-procedure in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 25s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.procedure2.store.wal.TestWALProcedureStore |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.12.0 Server=1.12.0 Image:yetus/hbase:date2016-08-17 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12824087/HBASE-16094.master.004.patch
|
| JIRA Issue | HBASE-16094 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux b2b630004f43 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.s
[jira] [Commented] (HBASE-16399) Provide an API to get list of failed regions and servername in Canary
[
https://issues.apache.org/jira/browse/HBASE-16399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423996#comment-15423996
]
Hadoop QA commented on HBASE-16399:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m 2s {color}
| {color:red} Docker failed to build yetus/hbase:date2016-08-17. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12824095/HBASE-16399.branch-1.01.patch
|
| JIRA Issue | HBASE-16399 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3123/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Provide an API to get list of failed regions and servername in Canary
> -
>
> Key: HBASE-16399
> URL: https://issues.apache.org/jira/browse/HBASE-16399
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 1.3.1, 0.98.21
>Reporter: Vishal Khandelwal
>Assignee: Vishal Khandelwal
> Fix For: 1.3.1, 0.98.23
>
> Attachments: HBASE-16399.0.98.00.patch, HBASE-16399.0.98.01.patch,
> HBASE-16399.branch-1.00.patch, HBASE-16399.branch-1.01.patch
>
>
> At present HBase Canary tool only prints the failures as part of logs. It
> does not provide an API to get the list or summarizes it so caller can take
> action on the failed host. This Jira would additional API so caller can get
> read or write canary failures.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16388) Prevent client threads being blocked by only one slow region server
[
https://issues.apache.org/jira/browse/HBASE-16388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423990#comment-15423990
]
Hadoop QA commented on HBASE-16388:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 18s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
23s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 16s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
41s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 16s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 12s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 5s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
30m 36s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 6s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 17s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 48s
{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 3s
{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 94m 14s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
45s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 157m 43s {color}
| {color:black} {color} |
\\
\
[jira] [Updated] (HBASE-16399) Provide an API to get list of failed regions and servername in Canary
[ https://issues.apache.org/jira/browse/HBASE-16399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Khandelwal updated HBASE-16399: -- Attachment: HBASE-16399.branch-1.01.patch HBASE-16399.0.98.01.patch Added unit test fixed findbugs added failed list in RegionserverTask as well Removed whitespaces from patch > Provide an API to get list of failed regions and servername in Canary > - > > Key: HBASE-16399 > URL: https://issues.apache.org/jira/browse/HBASE-16399 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.3.1, 0.98.21 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal > Fix For: 1.3.1, 0.98.23 > > Attachments: HBASE-16399.0.98.00.patch, HBASE-16399.0.98.01.patch, > HBASE-16399.branch-1.00.patch, HBASE-16399.branch-1.01.patch > > > At present HBase Canary tool only prints the failures as part of logs. It > does not provide an API to get the list or summarizes it so caller can take > action on the failed host. This Jira would additional API so caller can get > read or write canary failures. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[ https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423827#comment-15423827 ] Duo Zhang edited comment on HBASE-15921 at 8/17/16 6:39 AM: OK, you use it in this patch... I left a comment on rb, I do not think it is a good idea to jump over the stub layer of protobuf to call a RpcChannel directly, the method name mapping is fragile, and we also bypass the type check which is not good... Thanks. was (Author: apache9): OK, you use it in this patch... I left a comment on rb, I do not think it is a good idea to jump over the stub layer of protobuf to call a RpcChannel directly, the method name mapping is fragile, and we also eliminate the type check... Thanks. > Add first AsyncTable impl and create TableImpl based on it > -- > > Key: HBASE-15921 > URL: https://issues.apache.org/jira/browse/HBASE-15921 > Project: HBase > Issue Type: Improvement >Reporter: Jurriaan Mous >Assignee: Jurriaan Mous > Attachments: HBASE-15921.patch, HBASE-15921.v1.patch > > > First we create an AsyncTable interface with implementation without the Scan > functionality. Those will land in a separate patch since they need a refactor > of existing scans. > Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl > internally and should be a bit faster because it does jump through less hoops > to do ProtoBuf transportation. This way we can run all existing tests on the > AsyncTableImpl to guarantee its quality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16384) Update report-flakies.py script to allow specifying a list of build ids to be excluded
[ https://issues.apache.org/jira/browse/HBASE-16384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-16384: - Attachment: HBASE-16384.master.007.patch > Update report-flakies.py script to allow specifying a list of build ids to be > excluded > -- > > Key: HBASE-16384 > URL: https://issues.apache.org/jira/browse/HBASE-16384 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-16384.master.001.patch, > HBASE-16384.master.002.patch, HBASE-16384.master.003.patch, > HBASE-16384.master.004.patch, HBASE-16384.master.005.patch, > HBASE-16384.master.006.patch, HBASE-16384.master.007.patch > > > Sometimes, builds fail mysteriously and leave lots of tests hanging. This > makes [flaky > list|https://builds.apache.org/job/HBase-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html] > go crazy. > This patch adds that feature to specify build ids to exclude in > report-flakies.py. > If we find that a build screwed up, we can exclude it using "exclude=" option > in --urls param and rerun the job to fix the flaky list. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16384) Update report-flakies.py script to allow specifying a list of build ids to be excluded
[ https://issues.apache.org/jira/browse/HBASE-16384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423977#comment-15423977 ] Appy commented on HBASE-16384: -- bq.Order imports alphabetically, but grouped as described in PEP 8. The current order didn't throw lint warning so i thought it was fine. Anyways, changed it for better. If its still not perfect, please ignore it as i really consider it a nit. bq. Can probably remove the brackets around the metavar for excluded_builds to be more clear on how to pass arguments. done bq. In your new get_bad_tests, you removed the assignment of result. I'm also confused about why the doc suggests you'll return a list, but then you actually return an empty dictionary sometimes. Good catch. i thought intellij's rename would rename every usage, but seems like it left this bug. bq. Use enumerate in place of for i in range(len(job_urls)) to stay Pythonic I noticed it in lint, but this can't be changed. i is used for getting corresponding elements in other args. There may be other fancy pythonic way to do this, but i'd rather not get hung up on that. > Update report-flakies.py script to allow specifying a list of build ids to be > excluded > -- > > Key: HBASE-16384 > URL: https://issues.apache.org/jira/browse/HBASE-16384 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-16384.master.001.patch, > HBASE-16384.master.002.patch, HBASE-16384.master.003.patch, > HBASE-16384.master.004.patch, HBASE-16384.master.005.patch, > HBASE-16384.master.006.patch > > > Sometimes, builds fail mysteriously and leave lots of tests hanging. This > makes [flaky > list|https://builds.apache.org/job/HBase-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html] > go crazy. > This patch adds that feature to specify build ids to exclude in > report-flakies.py. > If we find that a build screwed up, we can exclude it using "exclude=" option > in --urls param and rerun the job to fix the flaky list. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15635) Mean age of Blocks in cache (seconds) on webUI should be greater than zero
[
https://issues.apache.org/jira/browse/HBASE-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423969#comment-15423969
]
Hadoop QA commented on HBASE-15635:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 9s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 138m 29s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 181m 30s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestReplicaWithCluster |
| | hadoop.hbase.replication.regionserver.TestReplicationSink |
| | hadoop.hbase.security.access.TestAccessController |
| | hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery |
| | hadoop.hbase.replication.TestReplicationSyncUpToolWithBulkLoadedData |
| | hadoop.hbase.mapreduce.TestCopyTable |
| | hadoop.hbase.mapreduce.TestHFileOutputFormat2 |
| | hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery |
| | hadoop.hbase.coprocessor.TestRegionObserverInterface |
| | hado
[jira] [Updated] (HBASE-16094) Procedure v2 - Improve cleaning up of proc wals
[ https://issues.apache.org/jira/browse/HBASE-16094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-16094: - Attachment: HBASE-16094.master.004.patch > Procedure v2 - Improve cleaning up of proc wals > --- > > Key: HBASE-16094 > URL: https://issues.apache.org/jira/browse/HBASE-16094 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0 > > Attachments: HBASE-16094.master.001.patch, > HBASE-16094.master.002.patch, HBASE-16094.master.003.patch, > HBASE-16094.master.004.patch > > > Avoid accumulating too many wals. > We remove logs in 3 cases: > - No procedures are running. We can remove all the logs. > - All procedures are updated/written in the last log. We can remove all the > logs, aside the active one. > - Remove log if does not contains “active” procIds > https://github.com/apache/hbase/blob/master/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java#L865 > The last point, Remove log if does not contains “active” procIds, needs to be > improved. Basically, even if a log contains state of an active proc, we can > delete it if a later log contains a newer state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16179) Fix compilation errors when building hbase-spark against Spark 2.0
[ https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423939#comment-15423939 ] Euan de Kock commented on HBASE-16179: -- The Logging should never have been used - take a look at this commit to the cassandra project for an example of how we can build an internal Logging trait. https://github.com/datastax/spark-cassandra-connector/commit/98d610e866463a15e91f7e4a967e09ab338aa330 > Fix compilation errors when building hbase-spark against Spark 2.0 > -- > > Key: HBASE-16179 > URL: https://issues.apache.org/jira/browse/HBASE-16179 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu > > I tried building hbase-spark module against Spark-2.0 snapshot and got the > following compilation errors: > http://pastebin.com/bg3w247a > Some Spark classes such as DataTypeParser and Logging are no longer > accessible to downstream projects. > hbase-spark module should not depend on such classes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15635) Mean age of Blocks in cache (seconds) on webUI should be greater than zero
[
https://issues.apache.org/jira/browse/HBASE-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423938#comment-15423938
]
Hadoop QA commented on HBASE-15635:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m 2s {color}
| {color:red} Docker failed to build yetus/hbase:date2016-08-17. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12824084/HBASE-15635_v2.patch |
| JIRA Issue | HBASE-15635 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3120/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Mean age of Blocks in cache (seconds) on webUI should be greater than zero
> --
>
> Key: HBASE-15635
> URL: https://issues.apache.org/jira/browse/HBASE-15635
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.17
>Reporter: Heng Chen
>Assignee: Heng Chen
> Fix For: 2.0.0, 1.4.0, 1.0.5, 1.3.1, 1.2.3, 0.98.22, 1.1.7
>
> Attachments: 0.98.png, 7BFFAF68-0807-400C-853F-706B498449E1.png,
> HBASE-15635.0.98.patch, HBASE-15635.patch, HBASE-15635_v1.patch,
> HBASE-15635_v2.patch, master.png
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423937#comment-15423937
]
Hadoop QA commented on HBASE-16419:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 46s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 28s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 72m 16s
{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 107m 17s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-17 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12824061/HBASE-16419-branch-1.2-v1.patch
|
| JIRA Issue | HBASE-16419 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux c5a546502476 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/preco
[jira] [Commented] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423936#comment-15423936
]
Hadoop QA commented on HBASE-16419:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 0m 4s {color}
| {color:red} Docker failed to build yetus/hbase:date2016-08-17. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12824074/HBASE-16419-branch-1.2-v2.patch
|
| JIRA Issue | HBASE-16419 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3119/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> check REPLICATION_SCOPE's value more stringent
> --
>
> Key: HBASE-16419
> URL: https://issues.apache.org/jira/browse/HBASE-16419
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0, 1.2.2
>Reporter: Guangxu Cheng
> Fix For: 1.2.4
>
> Attachments: HBASE-16419-branch-1.2-v1.patch,
> HBASE-16419-branch-1.2-v2.patch, HBASE-16419-v1.patch
>
>
> When create table or modify table, the master will check if the value of
> REPLICATION_SCOPE is less than 0, however the value of REPLICATION_SCOPE must
> be 0 or 1. Otherwise will lead to regionserver shutdown, so I think should
> be check the values of REPLICATION_SCOPE more stringent.
> Beginning I don't fully understand the usage of REPLICATION_SCOPE, then set
> REPLICATION_SCOPE to 2 by mistake.when I insert data to table,the
> regionservers abort one by one,finanly
> the cluster abort，the exceptions as follow:
> {quote}
> 2016-08-16 12:34:45,245 WARN [regionserver/host:60023.append-pool1-t1]
> wal.FSHLog: Append sequenceId=94, requesting roll of WAL
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:118)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1886)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2016-08-16 12:34:45,293 INFO [MemStoreFlusher.0] regionserver.HStore: Added
> hdfs://hbase-test-27/hbase1.2.2/data/default/usertable/2aa98c17845c9c6d5c8760b87b3ba09a/i/35825c94e72945c0bf7df3f0adefa1b6,
> entries=1161600, sequenceid=59, filesize=167.6 M
> 2016-08-16 12:34:45,296 FATAL [MemStoreFlusher.0] regionserver.HRegionServer:
> ABORTING region server hbase-10-166-141-99,60023,1471262434177: Replay of WAL
> required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> usertable,,1471262560009.2aa98c17845c9c6d5c8760b87b3ba09a.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2427)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2105)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2067)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1958)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1884)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:510)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:75)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
> Append sequenceId=94, requesting roll of WAL
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1898)
> at
> o
[jira] [Updated] (HBASE-15635) Mean age of Blocks in cache (seconds) on webUI should be greater than zero
[ https://issues.apache.org/jira/browse/HBASE-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-15635: -- Attachment: HBASE-15635_v2.patch patch_v2 fix patch_v1 regionserver's modification > Mean age of Blocks in cache (seconds) on webUI should be greater than zero > -- > > Key: HBASE-15635 > URL: https://issues.apache.org/jira/browse/HBASE-15635 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.17 >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 1.4.0, 1.0.5, 1.3.1, 1.2.3, 0.98.22, 1.1.7 > > Attachments: 0.98.png, 7BFFAF68-0807-400C-853F-706B498449E1.png, > HBASE-15635.0.98.patch, HBASE-15635.patch, HBASE-15635_v1.patch, > HBASE-15635_v2.patch, master.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16431) Add missing method in class HTableWrapper
Biao Ma created HBASE-16431: --- Summary: Add missing method in class HTableWrapper Key: HBASE-16431 URL: https://issues.apache.org/jira/browse/HBASE-16431 Project: HBase Issue Type: Bug Components: Client Affects Versions: 1.2.2 Reporter: Biao Ma Class `HTableWrapper` must either declared abstract or implement abstract method 'setRpcTimeout(int)' in 'Table' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16426) Remove company affiliations from committer list
[
https://issues.apache.org/jira/browse/HBASE-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423911#comment-15423911
]
Hadoop QA commented on HBASE-16426:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 13s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 12s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 32s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 35s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 19s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 30s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 1m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 0s
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
26m 40s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 10s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 35s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 21s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 103m 12s
{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 156m 23s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-17 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12824051/HBASE-16426.patch |
| JIRA Issue | HBASE-16426 |
| Optional Tests | asflicense javac javadoc unit xml compile |
| uname | Linux 3e391c044ac7 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28
20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d5080e8 |
| Default Java | 1.7.0_101 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_101
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 |
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/3114/artifact/patchproce
[jira] [Commented] (HBASE-16430) RegionServer Group's bug when move tables
[
https://issues.apache.org/jira/browse/HBASE-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423873#comment-15423873
]
Guangxu Cheng commented on HBASE-16430:
---
Attached patch for correcting movetables.
> RegionServer Group's bug when move tables
> -
>
> Key: HBASE-16430
> URL: https://issues.apache.org/jira/browse/HBASE-16430
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0
>Reporter: Guangxu Cheng
> Attachments: HBASE-16430-v1.patch
>
>
> it is an obvious bug when moves table. Implementation of the moveTables as
> follows：
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> @Override
> public synchronized void moveTables(
> Set tableNames, String groupName) throws IOException {
> if (groupName != null && !rsGroupMap.containsKey(groupName)) {
> throw new DoNotRetryIOException("Group "+groupName+" does not exist or
> is a special group");
> }
> Map newGroupMap = Maps.newHashMap(rsGroupMap);
> for(TableName tableName: tableNames) {
> if (tableMap.containsKey(tableName)) {
> RSGroupInfo src = new
> RSGroupInfo(rsGroupMap.get(tableMap.get(tableName)));
> src.removeTable(tableName);
> newGroupMap.put(src.getName(), src);
> }
> if(groupName != null) {
> RSGroupInfo dst = new RSGroupInfo(newGroupMap.get(groupName));
> dst.addTable(tableName);
> newGroupMap.put(dst.getName(), dst);
> }
> }
> flushConfig(newGroupMap);
> }
> {code}
> Should use newGroupMap instead of rsGroupMap:
> {code}
> RSGroupInfo src = new RSGroupInfo(rsGroupMap.get(tableMap.get(tableName)));
> {code}
> ==>
> {code}
> RSGroupInfo src = new RSGroupInfo(newGroupMap.get(tableMap.get(tableName)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16430) RegionServer Group's bug when move tables
[
https://issues.apache.org/jira/browse/HBASE-16430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-16430:
--
Attachment: HBASE-16430-v1.patch
> RegionServer Group's bug when move tables
> -
>
> Key: HBASE-16430
> URL: https://issues.apache.org/jira/browse/HBASE-16430
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0
>Reporter: Guangxu Cheng
> Attachments: HBASE-16430-v1.patch
>
>
> it is an obvious bug when moves table. Implementation of the moveTables as
> follows：
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> @Override
> public synchronized void moveTables(
> Set tableNames, String groupName) throws IOException {
> if (groupName != null && !rsGroupMap.containsKey(groupName)) {
> throw new DoNotRetryIOException("Group "+groupName+" does not exist or
> is a special group");
> }
> Map newGroupMap = Maps.newHashMap(rsGroupMap);
> for(TableName tableName: tableNames) {
> if (tableMap.containsKey(tableName)) {
> RSGroupInfo src = new
> RSGroupInfo(rsGroupMap.get(tableMap.get(tableName)));
> src.removeTable(tableName);
> newGroupMap.put(src.getName(), src);
> }
> if(groupName != null) {
> RSGroupInfo dst = new RSGroupInfo(newGroupMap.get(groupName));
> dst.addTable(tableName);
> newGroupMap.put(dst.getName(), dst);
> }
> }
> flushConfig(newGroupMap);
> }
> {code}
> Should use newGroupMap instead of rsGroupMap:
> {code}
> RSGroupInfo src = new RSGroupInfo(rsGroupMap.get(tableMap.get(tableName)));
> {code}
> ==>
> {code}
> RSGroupInfo src = new RSGroupInfo(newGroupMap.get(tableMap.get(tableName)));
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423867#comment-15423867
]
Guangxu Cheng commented on HBASE-16419:
---
[~tedyu] I'v repaired it according to your suggestions. thanks
> check REPLICATION_SCOPE's value more stringent
> --
>
> Key: HBASE-16419
> URL: https://issues.apache.org/jira/browse/HBASE-16419
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0, 1.2.2
>Reporter: Guangxu Cheng
> Fix For: 1.2.4
>
> Attachments: HBASE-16419-branch-1.2-v1.patch,
> HBASE-16419-branch-1.2-v2.patch, HBASE-16419-v1.patch
>
>
> When create table or modify table, the master will check if the value of
> REPLICATION_SCOPE is less than 0, however the value of REPLICATION_SCOPE must
> be 0 or 1. Otherwise will lead to regionserver shutdown, so I think should
> be check the values of REPLICATION_SCOPE more stringent.
> Beginning I don't fully understand the usage of REPLICATION_SCOPE, then set
> REPLICATION_SCOPE to 2 by mistake.when I insert data to table,the
> regionservers abort one by one,finanly
> the cluster abort，the exceptions as follow:
> {quote}
> 2016-08-16 12:34:45,245 WARN [regionserver/host:60023.append-pool1-t1]
> wal.FSHLog: Append sequenceId=94, requesting roll of WAL
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:118)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1886)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2016-08-16 12:34:45,293 INFO [MemStoreFlusher.0] regionserver.HStore: Added
> hdfs://hbase-test-27/hbase1.2.2/data/default/usertable/2aa98c17845c9c6d5c8760b87b3ba09a/i/35825c94e72945c0bf7df3f0adefa1b6,
> entries=1161600, sequenceid=59, filesize=167.6 M
> 2016-08-16 12:34:45,296 FATAL [MemStoreFlusher.0] regionserver.HRegionServer:
> ABORTING region server hbase-10-166-141-99,60023,1471262434177: Replay of WAL
> required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> usertable,,1471262560009.2aa98c17845c9c6d5c8760b87b3ba09a.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2427)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2105)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2067)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1958)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1884)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:510)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:75)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
> Append sequenceId=94, requesting roll of WAL
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1898)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
>
[jira] [Updated] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-16419:
--
Attachment: HBASE-16419-branch-1.2-v2.patch
> check REPLICATION_SCOPE's value more stringent
> --
>
> Key: HBASE-16419
> URL: https://issues.apache.org/jira/browse/HBASE-16419
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0, 1.2.2
>Reporter: Guangxu Cheng
> Fix For: 1.2.4
>
> Attachments: HBASE-16419-branch-1.2-v1.patch,
> HBASE-16419-branch-1.2-v2.patch, HBASE-16419-v1.patch
>
>
> When create table or modify table, the master will check if the value of
> REPLICATION_SCOPE is less than 0, however the value of REPLICATION_SCOPE must
> be 0 or 1. Otherwise will lead to regionserver shutdown, so I think should
> be check the values of REPLICATION_SCOPE more stringent.
> Beginning I don't fully understand the usage of REPLICATION_SCOPE, then set
> REPLICATION_SCOPE to 2 by mistake.when I insert data to table,the
> regionservers abort one by one,finanly
> the cluster abort，the exceptions as follow:
> {quote}
> 2016-08-16 12:34:45,245 WARN [regionserver/host:60023.append-pool1-t1]
> wal.FSHLog: Append sequenceId=94, requesting roll of WAL
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:118)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1886)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2016-08-16 12:34:45,293 INFO [MemStoreFlusher.0] regionserver.HStore: Added
> hdfs://hbase-test-27/hbase1.2.2/data/default/usertable/2aa98c17845c9c6d5c8760b87b3ba09a/i/35825c94e72945c0bf7df3f0adefa1b6,
> entries=1161600, sequenceid=59, filesize=167.6 M
> 2016-08-16 12:34:45,296 FATAL [MemStoreFlusher.0] regionserver.HRegionServer:
> ABORTING region server hbase-10-166-141-99,60023,1471262434177: Replay of WAL
> required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> usertable,,1471262560009.2aa98c17845c9c6d5c8760b87b3ba09a.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2427)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2105)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2067)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1958)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1884)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:510)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:75)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
> Append sequenceId=94, requesting roll of WAL
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1898)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
>
[jira] [Commented] (HBASE-16420) Fix source incompatibility of Table interface
[ https://issues.apache.org/jira/browse/HBASE-16420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423863#comment-15423863 ] Josh Elser commented on HBASE-16420: Ditto, Stack. These changes look great, Phil. I really appreciate you finding the time to put up a patch so quickly! > Fix source incompatibility of Table interface > - > > Key: HBASE-16420 > URL: https://issues.apache.org/jira/browse/HBASE-16420 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.5, 1.2.2 >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 1.1.6, 1.2.3 > > Attachments: HBASE-16420-branch-1.2-v1.patch > > > HBASE-15645 broke source compatibility of Table interface. We need revert the > changing on branch-1.1 and branch-1.2. There is also a bugfix for timeout, so > we should keep the bugfix code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423856#comment-15423856
]
Ted Yu commented on HBASE-16419:
{code}
1652 String message = "Replication scope for column family "
1653 + hcd.getNameAsString() + " is invalid.";
1654
{code}
Consider adding value of the bad scope to message.
Change the log from warn to error().
> check REPLICATION_SCOPE's value more stringent
> --
>
> Key: HBASE-16419
> URL: https://issues.apache.org/jira/browse/HBASE-16419
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0, 1.2.2
>Reporter: Guangxu Cheng
> Fix For: 1.2.4
>
> Attachments: HBASE-16419-branch-1.2-v1.patch, HBASE-16419-v1.patch
>
>
> When create table or modify table, the master will check if the value of
> REPLICATION_SCOPE is less than 0, however the value of REPLICATION_SCOPE must
> be 0 or 1. Otherwise will lead to regionserver shutdown, so I think should
> be check the values of REPLICATION_SCOPE more stringent.
> Beginning I don't fully understand the usage of REPLICATION_SCOPE, then set
> REPLICATION_SCOPE to 2 by mistake.when I insert data to table,the
> regionservers abort one by one,finanly
> the cluster abort，the exceptions as follow:
> {quote}
> 2016-08-16 12:34:45,245 WARN [regionserver/host:60023.append-pool1-t1]
> wal.FSHLog: Append sequenceId=94, requesting roll of WAL
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:118)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1886)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2016-08-16 12:34:45,293 INFO [MemStoreFlusher.0] regionserver.HStore: Added
> hdfs://hbase-test-27/hbase1.2.2/data/default/usertable/2aa98c17845c9c6d5c8760b87b3ba09a/i/35825c94e72945c0bf7df3f0adefa1b6,
> entries=1161600, sequenceid=59, filesize=167.6 M
> 2016-08-16 12:34:45,296 FATAL [MemStoreFlusher.0] regionserver.HRegionServer:
> ABORTING region server hbase-10-166-141-99,60023,1471262434177: Replay of WAL
> required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> usertable,,1471262560009.2aa98c17845c9c6d5c8760b87b3ba09a.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2427)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2105)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2067)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1958)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1884)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:510)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:75)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
> Append sequenceId=94, requesting roll of WAL
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1898)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more
> Caused by: java.lang.NullPointerExcep
[jira] [Updated] (HBASE-16388) Prevent client threads being blocked by only one slow region server
[ https://issues.apache.org/jira/browse/HBASE-16388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated HBASE-16388: -- Attachment: HBASE-16388-v2.patch retry > Prevent client threads being blocked by only one slow region server > --- > > Key: HBASE-16388 > URL: https://issues.apache.org/jira/browse/HBASE-16388 > Project: HBase > Issue Type: New Feature >Reporter: Phil Yang >Assignee: Phil Yang > Attachments: HBASE-16388-v1.patch, HBASE-16388-v2.patch, > HBASE-16388-v2.patch > > > It is a general use case for HBase's users that they have several > threads/handlers in their service, and each handler has its own Table/HTable > instance. Generally users think each handler is independent and won't > interact each other. > However, in an extreme case, if a region server is very slow, every requests > to this RS will timeout, handlers of users' service may be occupied by the > long-waiting requests even requests belong to other RS will also be timeout. > For example: > If we have 100 handlers in a client service(timeout is 1000ms) and HBase has > 10 region servers whose average response time is 50ms. If no region server is > slow, we can handle 2000 requests per second. > Now this service's QPS is 1000. If there is one region server very slow and > all requests to it will be timeout. Users hope that only 10% requests failed, > and 90% requests' response time is still 50ms, because only 10% requests are > located to the slow RS. However, each second we have 100 long-waiting > requests which exactly occupies all 100 handles. So all handlers is blocked, > the availability of this service is almost zero. > To prevent this case, we can limit the max concurrent requests to one RS in > process-level. Requests exceeding the limit will throws > ServerBusyException(extends DoNotRetryIOE) immediately to users. In the above > case, if we set this limit to 20, only 20 handlers will be occupied and other > 80 handlers can still handle requests to other RS. The availability of this > service is 90% as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16429) FSHLog: deadlock if rollWriter called when ring buffer filled with appends
[
https://issues.apache.org/jira/browse/HBASE-16429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423835#comment-15423835
]
Yu Li commented on HBASE-16429:
---
[~stack] please let me know your thoughts boss, thanks.
> FSHLog: deadlock if rollWriter called when ring buffer filled with appends
> --
>
> Key: HBASE-16429
> URL: https://issues.apache.org/jira/browse/HBASE-16429
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2, 1.2.2
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
> Attachments: HBASE-16429.patch
>
>
> Recently we experienced an online problem that all handlers are stuck.
> Checking the jstack we could see all handler threads waiting for
> RingBuffer.next, while the single ring buffer consumer dead waiting for
> {{safePointReleasedLatch}} to count down:
> {noformat}
> Normal handler thread:
> "B.defaultRpcServer.handler=126,queue=9,port=16020" daemon prio=10
> tid=0x7efd4b44f800 nid=0x15f29 runnable [0x7efd3db7b000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
> at
> com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136)
> at
> com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105)
> at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1222)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3188)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2819)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:736)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:698)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2095)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:774)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:756)
> RingBufferEventHandler thread waiting for safePointReleasedLatch:
> "regionserver/hadoop0369.et2.tbsite.net/11.251.152.226:16020.append-pool2-t1"
> prio=10 tid=0x7efd320d nid=0x1777b waiting on condition
> [0x7efd2d2fa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7f01b48d9178> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$SafePointZigZagLatch.safePointAttained(FSHLog.java:1866)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.attainSafePoint(FSHLog.java:2066)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:2029)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1909)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:756)
> {noformat}
> {{FSHLog#replaceWriter}} will call {{SafePointZigZagLatch#releaseSafePoint}}
> to count down {{safePointReleasedLatch}}, but replaceWriter got stuck when
> trying to publish a sync onto ring buffer:
> {noformat}
> "regionserver/hadoop0369.et2.tbsite.net/11.251.152.226:16020.logRoller"
> daemon prio=10 tid=0x000
[jira] [Commented] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[
https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423837#comment-15423837
]
Hadoop QA commented on HBASE-15921:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color}
| {color:red} HBASE-15921 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.3.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12807113/HBASE-15921.v1.patch |
| JIRA Issue | HBASE-15921 |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3116/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Add first AsyncTable impl and create TableImpl based on it
> --
>
> Key: HBASE-15921
> URL: https://issues.apache.org/jira/browse/HBASE-15921
> Project: HBase
> Issue Type: Improvement
>Reporter: Jurriaan Mous
>Assignee: Jurriaan Mous
> Attachments: HBASE-15921.patch, HBASE-15921.v1.patch
>
>
> First we create an AsyncTable interface with implementation without the Scan
> functionality. Those will land in a separate patch since they need a refactor
> of existing scans.
> Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl
> internally and should be a bit faster because it does jump through less hoops
> to do ProtoBuf transportation. This way we can run all existing tests on the
> AsyncTableImpl to guarantee its quality.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-16429) FSHLog: deadlock if rollWriter called when ring buffer filled with appends
[
https://issues.apache.org/jira/browse/HBASE-16429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423831#comment-15423831
]
Yu Li edited comment on HBASE-16429 at 8/17/16 3:56 AM:
Some explanation of the fix: change the logic to get a sequence from the ring
buffer first before initializing {{zigzagLatch}} in {{doReplaceWriter}}, and
since disruptor deals with the events one by one following the sequence, as
long as we get the sequence in doReplaceWriter, the latter sync publish won't
get stuck. Meanwhile, the {{ringBufferEventHandler}} won't go into
{{zigzagLatch#safePointAttained}} if {{zigzagLatch}} is not initialized, so
this consumer of the ring buffer won't get suck, and we will get the sequence
in {{doReplaceWriter}} sooner or later even if ring buffer filled with appends.
This way no deadlock will happen.
The added UT case will only pass with the fix
was (Author: carp84):
Some explanation of the fix: change the logic to get a sequence from the ring
buffer first before initializing {{zigzagLatch}} in {{doReplaceWriter}}, and
since disruptor deals with the events one by one following the sequence, as
long as the we get the sequence in doReplaceWriter, the latter sync publish
won't get stuck. Meanwhile, the {{ringBufferEventHandler}} won't go into
{{zigzagLatch#safePointAttained}} if {{zigzagLatch}} is not initialized, so
this consumer of the ring buffer won't get suck, and we will get the sequence
in {{doReplaceWriter}} sooner or later even if ring buffer filled with appends.
This way no deadlock will happen.
> FSHLog: deadlock if rollWriter called when ring buffer filled with appends
> --
>
> Key: HBASE-16429
> URL: https://issues.apache.org/jira/browse/HBASE-16429
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2, 1.2.2
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
> Attachments: HBASE-16429.patch
>
>
> Recently we experienced an online problem that all handlers are stuck.
> Checking the jstack we could see all handler threads waiting for
> RingBuffer.next, while the single ring buffer consumer dead waiting for
> {{safePointReleasedLatch}} to count down:
> {noformat}
> Normal handler thread:
> "B.defaultRpcServer.handler=126,queue=9,port=16020" daemon prio=10
> tid=0x7efd4b44f800 nid=0x15f29 runnable [0x7efd3db7b000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
> at
> com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136)
> at
> com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105)
> at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1222)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3188)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2819)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:736)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:698)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2095)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:774)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:756)
> RingBufferEventHandler thread waiting for safePointReleasedLatch:
> "regionserver/hadoop0369.et2.tbsite.net/11.251.152.226:16020.append-pool2-t1"
> prio=10 tid=0x7efd320d nid=0x1777b waiting on condition
> [0x7efd2d2fa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7f01b48d9178> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQue
[jira] [Commented] (HBASE-16429) FSHLog: deadlock if rollWriter called when ring buffer filled with appends
[
https://issues.apache.org/jira/browse/HBASE-16429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423831#comment-15423831
]
Yu Li commented on HBASE-16429:
---
Some explanation of the fix: change the logic to get a sequence from the ring
buffer first before initializing {{zigzagLatch}} in {{doReplaceWriter}}, and
since disruptor deals with the events one by one following the sequence, as
long as the we get the sequence in doReplaceWriter, the latter sync publish
won't get stuck. Meanwhile, the {{ringBufferEventHandler}} won't go into
{{zigzagLatch#safePointAttained}} if {{zigzagLatch}} is not initialized, so
this consumer of the ring buffer won't get suck, and we will get the sequence
in {{doReplaceWriter}} sooner or later even if ring buffer filled with appends.
This way no deadlock will happen.
> FSHLog: deadlock if rollWriter called when ring buffer filled with appends
> --
>
> Key: HBASE-16429
> URL: https://issues.apache.org/jira/browse/HBASE-16429
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2, 1.2.2
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
> Attachments: HBASE-16429.patch
>
>
> Recently we experienced an online problem that all handlers are stuck.
> Checking the jstack we could see all handler threads waiting for
> RingBuffer.next, while the single ring buffer consumer dead waiting for
> {{safePointReleasedLatch}} to count down:
> {noformat}
> Normal handler thread:
> "B.defaultRpcServer.handler=126,queue=9,port=16020" daemon prio=10
> tid=0x7efd4b44f800 nid=0x15f29 runnable [0x7efd3db7b000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
> at
> com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136)
> at
> com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105)
> at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1222)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3188)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2819)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:736)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:698)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2095)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:774)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:756)
> RingBufferEventHandler thread waiting for safePointReleasedLatch:
> "regionserver/hadoop0369.et2.tbsite.net/11.251.152.226:16020.append-pool2-t1"
> prio=10 tid=0x7efd320d nid=0x1777b waiting on condition
> [0x7efd2d2fa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7f01b48d9178> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$SafePointZigZagLatch.safePointAttained(FSHLog.java:1866)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.attainSafePoint(FSHLog.java:2066)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:2029)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1909)
> at
> com.lmax.disruptor.BatchEventProcessor.run(Batc
[jira] [Commented] (HBASE-16420) Fix source incompatibility of Table interface
[ https://issues.apache.org/jira/browse/HBASE-16420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423830#comment-15423830 ] Hudson commented on HBASE-16420: FAILURE: Integrated in Jenkins build HBase-1.2-IT #579 (See [https://builds.apache.org/job/HBase-1.2-IT/579/]) HBASE-16420 Fix source incompatibility of Table interface (Phil Yang) (stack: rev 45beeec0fbfe99bf41d166f00973d0c5efd025c9) * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java > Fix source incompatibility of Table interface > - > > Key: HBASE-16420 > URL: https://issues.apache.org/jira/browse/HBASE-16420 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.5, 1.2.2 >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 1.1.6, 1.2.3 > > Attachments: HBASE-16420-branch-1.2-v1.patch > > > HBASE-15645 broke source compatibility of Table interface. We need revert the > changing on branch-1.1 and branch-1.2. There is also a bugfix for timeout, so > we should keep the bugfix code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15921) Add first AsyncTable impl and create TableImpl based on it
[ https://issues.apache.org/jira/browse/HBASE-15921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423827#comment-15423827 ] Duo Zhang commented on HBASE-15921: --- OK, you use it in this patch... I left a comment on rb, I do not think it is a good idea to jump over the stub layer of protobuf to call a RpcChannel directly, the method name mapping is fragile, and we also eliminate the type check... Thanks. > Add first AsyncTable impl and create TableImpl based on it > -- > > Key: HBASE-15921 > URL: https://issues.apache.org/jira/browse/HBASE-15921 > Project: HBase > Issue Type: Improvement >Reporter: Jurriaan Mous >Assignee: Jurriaan Mous > Attachments: HBASE-15921.patch, HBASE-15921.v1.patch > > > First we create an AsyncTable interface with implementation without the Scan > functionality. Those will land in a separate patch since they need a refactor > of existing scans. > Also added is a new TableImpl to replace HTable. It uses the AsyncTableImpl > internally and should be a bit faster because it does jump through less hoops > to do ProtoBuf transportation. This way we can run all existing tests on the > AsyncTableImpl to guarantee its quality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-16419:
--
Status: Patch Available (was: Open)
> check REPLICATION_SCOPE's value more stringent
> --
>
> Key: HBASE-16419
> URL: https://issues.apache.org/jira/browse/HBASE-16419
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.2.2, 2.0.0
>Reporter: Guangxu Cheng
> Fix For: 1.2.4
>
> Attachments: HBASE-16419-branch-1.2-v1.patch, HBASE-16419-v1.patch
>
>
> When create table or modify table, the master will check if the value of
> REPLICATION_SCOPE is less than 0, however the value of REPLICATION_SCOPE must
> be 0 or 1. Otherwise will lead to regionserver shutdown, so I think should
> be check the values of REPLICATION_SCOPE more stringent.
> Beginning I don't fully understand the usage of REPLICATION_SCOPE, then set
> REPLICATION_SCOPE to 2 by mistake.when I insert data to table,the
> regionservers abort one by one,finanly
> the cluster abort，the exceptions as follow:
> {quote}
> 2016-08-16 12:34:45,245 WARN [regionserver/host:60023.append-pool1-t1]
> wal.FSHLog: Append sequenceId=94, requesting roll of WAL
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:118)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1886)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2016-08-16 12:34:45,293 INFO [MemStoreFlusher.0] regionserver.HStore: Added
> hdfs://hbase-test-27/hbase1.2.2/data/default/usertable/2aa98c17845c9c6d5c8760b87b3ba09a/i/35825c94e72945c0bf7df3f0adefa1b6,
> entries=1161600, sequenceid=59, filesize=167.6 M
> 2016-08-16 12:34:45,296 FATAL [MemStoreFlusher.0] regionserver.HRegionServer:
> ABORTING region server hbase-10-166-141-99,60023,1471262434177: Replay of WAL
> required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> usertable,,1471262560009.2aa98c17845c9c6d5c8760b87b3ba09a.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2427)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2105)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2067)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1958)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1884)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:510)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:75)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
> Append sequenceId=94, requesting roll of WAL
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1898)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regi
[jira] [Updated] (HBASE-16429) FSHLog: deadlock if rollWriter called when ring buffer filled with appends
[
https://issues.apache.org/jira/browse/HBASE-16429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated HBASE-16429:
--
Attachment: HBASE-16429.patch
The initial patch
> FSHLog: deadlock if rollWriter called when ring buffer filled with appends
> --
>
> Key: HBASE-16429
> URL: https://issues.apache.org/jira/browse/HBASE-16429
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.1.2, 1.2.2
>Reporter: Yu Li
>Assignee: Yu Li
>Priority: Critical
> Attachments: HBASE-16429.patch
>
>
> Recently we experienced an online problem that all handlers are stuck.
> Checking the jstack we could see all handler threads waiting for
> RingBuffer.next, while the single ring buffer consumer dead waiting for
> {{safePointReleasedLatch}} to count down:
> {noformat}
> Normal handler thread:
> "B.defaultRpcServer.handler=126,queue=9,port=16020" daemon prio=10
> tid=0x7efd4b44f800 nid=0x15f29 runnable [0x7efd3db7b000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
> at
> com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136)
> at
> com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105)
> at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1222)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3188)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2879)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2819)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:736)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:698)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2095)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:774)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:756)
> RingBufferEventHandler thread waiting for safePointReleasedLatch:
> "regionserver/hadoop0369.et2.tbsite.net/11.251.152.226:16020.append-pool2-t1"
> prio=10 tid=0x7efd320d nid=0x1777b waiting on condition
> [0x7efd2d2fa000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x7f01b48d9178> (a
> java.util.concurrent.CountDownLatch$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$SafePointZigZagLatch.safePointAttained(FSHLog.java:1866)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.attainSafePoint(FSHLog.java:2066)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:2029)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1909)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:756)
> {noformat}
> {{FSHLog#replaceWriter}} will call {{SafePointZigZagLatch#releaseSafePoint}}
> to count down {{safePointReleasedLatch}}, but replaceWriter got stuck when
> trying to publish a sync onto ring buffer:
> {noformat}
> "regionserver/hadoop0369.et2.tbsite.net/11.251.152.226:16020.logRoller"
> daemon prio=10 tid=0x7efd320c8800 nid=0x16123 runnable
> [0x7efd311f6000]
>
[jira] [Created] (HBASE-16430) RegionServer Group's bug when move tables
Guangxu Cheng created HBASE-16430:
-
Summary: RegionServer Group's bug when move tables
Key: HBASE-16430
URL: https://issues.apache.org/jira/browse/HBASE-16430
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 2.0.0
Reporter: Guangxu Cheng
it is an obvious bug when moves table. Implementation of the moveTables as
follows：
{code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
@Override
public synchronized void moveTables(
Set tableNames, String groupName) throws IOException {
if (groupName != null && !rsGroupMap.containsKey(groupName)) {
throw new DoNotRetryIOException("Group "+groupName+" does not exist or is
a special group");
}
Map newGroupMap = Maps.newHashMap(rsGroupMap);
for(TableName tableName: tableNames) {
if (tableMap.containsKey(tableName)) {
RSGroupInfo src = new
RSGroupInfo(rsGroupMap.get(tableMap.get(tableName)));
src.removeTable(tableName);
newGroupMap.put(src.getName(), src);
}
if(groupName != null) {
RSGroupInfo dst = new RSGroupInfo(newGroupMap.get(groupName));
dst.addTable(tableName);
newGroupMap.put(dst.getName(), dst);
}
}
flushConfig(newGroupMap);
}
{code}
Should use newGroupMap instead of rsGroupMap:
{code}
RSGroupInfo src = new RSGroupInfo(rsGroupMap.get(tableMap.get(tableName)));
{code}
==>
{code}
RSGroupInfo src = new RSGroupInfo(newGroupMap.get(tableMap.get(tableName)));
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16429) FSHLog: deadlock if rollWriter called when ring buffer filled with appends
Yu Li created HBASE-16429:
-
Summary: FSHLog: deadlock if rollWriter called when ring buffer
filled with appends
Key: HBASE-16429
URL: https://issues.apache.org/jira/browse/HBASE-16429
Project: HBase
Issue Type: Bug
Affects Versions: 1.2.2, 1.1.2
Reporter: Yu Li
Assignee: Yu Li
Priority: Critical
Recently we experienced an online problem that all handlers are stuck. Checking
the jstack we could see all handler threads waiting for RingBuffer.next, while
the single ring buffer consumer dead waiting for {{safePointReleasedLatch}} to
count down:
{noformat}
Normal handler thread:
"B.defaultRpcServer.handler=126,queue=9,port=16020" daemon prio=10
tid=0x7efd4b44f800 nid=0x15f29 runnable [0x7efd3db7b000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
at
com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136)
at
com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105)
at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.append(FSHLog.java:1222)
at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3188)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2879)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2819)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:736)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:698)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2095)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:774)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:102)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:756)
RingBufferEventHandler thread waiting for safePointReleasedLatch:
"regionserver/hadoop0369.et2.tbsite.net/11.251.152.226:16020.append-pool2-t1"
prio=10 tid=0x7efd320d nid=0x1777b waiting on condition
[0x7efd2d2fa000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x7f01b48d9178> (a
java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$SafePointZigZagLatch.safePointAttained(FSHLog.java:1866)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.attainSafePoint(FSHLog.java:2066)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:2029)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1909)
at
com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:756)
{noformat}
{{FSHLog#replaceWriter}} will call {{SafePointZigZagLatch#releaseSafePoint}} to
count down {{safePointReleasedLatch}}, but replaceWriter got stuck when trying
to publish a sync onto ring buffer:
{noformat}
"regionserver/hadoop0369.et2.tbsite.net/11.251.152.226:16020.logRoller" daemon
prio=10 tid=0x7efd320c8800 nid=0x16123 runnable [0x7efd311f6000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:349)
at
com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:136)
at
com.lmax.disruptor.MultiProducerSequencer.next(MultiProducerSequencer.java:105)
at com.lmax.disruptor.RingBuffer.next(RingBuffer.java:246)
at
o
[jira] [Resolved] (HBASE-16428) hbase-spark will not compile on Spark 2.0.0 due to private Logger
[
https://issues.apache.org/jira/browse/HBASE-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-16428.
Resolution: Duplicate
> hbase-spark will not compile on Spark 2.0.0 due to private Logger
> -
>
> Key: HBASE-16428
> URL: https://issues.apache.org/jira/browse/HBASE-16428
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
> Environment: AWS EMR Spark 2.0.0
>Reporter: Euan de Kock
>
> Compiling a Spark application using Spark 2.0.0 will fail when the
> HBaseContext is referenced as this module references spark.Logging. In Spark
> 2.0.0 this was moved to a private location as it was never intended to be
> used outside of the Spark API.
> The actual error code is:
> {quote}
> [ERROR] error: missing or invalid dependency detected while loading class
> file 'HBaseContext.class'.
> [INFO] Could not access type Logging in package org.apache.spark,
> [INFO] because it (or its dependencies) are missing.
> {quote}
> See [https://issues.apache.org/jira/browse/SPARK-13928] for details on this
> change to Spark.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16428) hbase-spark will not compile on Spark 2.0.0 due to private Logger
[
https://issues.apache.org/jira/browse/HBASE-16428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423806#comment-15423806
]
Ted Yu commented on HBASE-16428:
Dupe of HBASE-16179
There're other compilation errors.
> hbase-spark will not compile on Spark 2.0.0 due to private Logger
> -
>
> Key: HBASE-16428
> URL: https://issues.apache.org/jira/browse/HBASE-16428
> Project: HBase
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.0.0
> Environment: AWS EMR Spark 2.0.0
>Reporter: Euan de Kock
>
> Compiling a Spark application using Spark 2.0.0 will fail when the
> HBaseContext is referenced as this module references spark.Logging. In Spark
> 2.0.0 this was moved to a private location as it was never intended to be
> used outside of the Spark API.
> The actual error code is:
> {quote}
> [ERROR] error: missing or invalid dependency detected while loading class
> file 'HBaseContext.class'.
> [INFO] Could not access type Logging in package org.apache.spark,
> [INFO] because it (or its dependencies) are missing.
> {quote}
> See [https://issues.apache.org/jira/browse/SPARK-13928] for details on this
> change to Spark.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16428) hbase-spark will not compile on Spark 2.0.0 due to private Logger
Euan de Kock created HBASE-16428:
Summary: hbase-spark will not compile on Spark 2.0.0 due to
private Logger
Key: HBASE-16428
URL: https://issues.apache.org/jira/browse/HBASE-16428
Project: HBase
Issue Type: Bug
Components: spark
Affects Versions: 2.0.0
Environment: AWS EMR Spark 2.0.0
Reporter: Euan de Kock
Compiling a Spark application using Spark 2.0.0 will fail when the HBaseContext
is referenced as this module references spark.Logging. In Spark 2.0.0 this was
moved to a private location as it was never intended to be used outside of the
Spark API.
The actual error code is:
{quote}
[ERROR] error: missing or invalid dependency detected while loading class file
'HBaseContext.class'.
[INFO] Could not access type Logging in package org.apache.spark,
[INFO] because it (or its dependencies) are missing.
{quote}
See [https://issues.apache.org/jira/browse/SPARK-13928] for details on this
change to Spark.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-16419:
--
Attachment: HBASE-16419-branch-1.2-v1.patch
> check REPLICATION_SCOPE's value more stringent
> --
>
> Key: HBASE-16419
> URL: https://issues.apache.org/jira/browse/HBASE-16419
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0, 1.2.2
>Reporter: Guangxu Cheng
> Fix For: 1.2.4
>
> Attachments: HBASE-16419-branch-1.2-v1.patch, HBASE-16419-v1.patch
>
>
> When create table or modify table, the master will check if the value of
> REPLICATION_SCOPE is less than 0, however the value of REPLICATION_SCOPE must
> be 0 or 1. Otherwise will lead to regionserver shutdown, so I think should
> be check the values of REPLICATION_SCOPE more stringent.
> Beginning I don't fully understand the usage of REPLICATION_SCOPE, then set
> REPLICATION_SCOPE to 2 by mistake.when I insert data to table,the
> regionservers abort one by one,finanly
> the cluster abort，the exceptions as follow:
> {quote}
> 2016-08-16 12:34:45,245 WARN [regionserver/host:60023.append-pool1-t1]
> wal.FSHLog: Append sequenceId=94, requesting roll of WAL
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:118)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1886)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2016-08-16 12:34:45,293 INFO [MemStoreFlusher.0] regionserver.HStore: Added
> hdfs://hbase-test-27/hbase1.2.2/data/default/usertable/2aa98c17845c9c6d5c8760b87b3ba09a/i/35825c94e72945c0bf7df3f0adefa1b6,
> entries=1161600, sequenceid=59, filesize=167.6 M
> 2016-08-16 12:34:45,296 FATAL [MemStoreFlusher.0] regionserver.HRegionServer:
> ABORTING region server hbase-10-166-141-99,60023,1471262434177: Replay of WAL
> required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> usertable,,1471262560009.2aa98c17845c9c6d5c8760b87b3ba09a.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2427)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2105)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2067)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1958)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1884)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:510)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:75)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
> Append sequenceId=94, requesting roll of WAL
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1898)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hba
[jira] [Commented] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423796#comment-15423796
]
Guangxu Cheng commented on HBASE-16419:
---
[~tedyu] that's a good idea.I'v attached new patch.thanks
> check REPLICATION_SCOPE's value more stringent
> --
>
> Key: HBASE-16419
> URL: https://issues.apache.org/jira/browse/HBASE-16419
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0, 1.2.2
>Reporter: Guangxu Cheng
> Fix For: 1.2.4
>
> Attachments: HBASE-16419-branch-1.2-v1.patch, HBASE-16419-v1.patch
>
>
> When create table or modify table, the master will check if the value of
> REPLICATION_SCOPE is less than 0, however the value of REPLICATION_SCOPE must
> be 0 or 1. Otherwise will lead to regionserver shutdown, so I think should
> be check the values of REPLICATION_SCOPE more stringent.
> Beginning I don't fully understand the usage of REPLICATION_SCOPE, then set
> REPLICATION_SCOPE to 2 by mistake.when I insert data to table,the
> regionservers abort one by one,finanly
> the cluster abort，the exceptions as follow:
> {quote}
> 2016-08-16 12:34:45,245 WARN [regionserver/host:60023.append-pool1-t1]
> wal.FSHLog: Append sequenceId=94, requesting roll of WAL
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:118)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1886)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2016-08-16 12:34:45,293 INFO [MemStoreFlusher.0] regionserver.HStore: Added
> hdfs://hbase-test-27/hbase1.2.2/data/default/usertable/2aa98c17845c9c6d5c8760b87b3ba09a/i/35825c94e72945c0bf7df3f0adefa1b6,
> entries=1161600, sequenceid=59, filesize=167.6 M
> 2016-08-16 12:34:45,296 FATAL [MemStoreFlusher.0] regionserver.HRegionServer:
> ABORTING region server hbase-10-166-141-99,60023,1471262434177: Replay of WAL
> required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> usertable,,1471262560009.2aa98c17845c9c6d5c8760b87b3ba09a.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2427)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2105)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2067)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1958)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1884)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:510)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:75)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
> Append sequenceId=94, requesting roll of WAL
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1898)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALK
[jira] [Updated] (HBASE-15635) Mean age of Blocks in cache (seconds) on webUI should be greater than zero
[ https://issues.apache.org/jira/browse/HBASE-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-15635: -- Attachment: HBASE-15635_v1.patch patch for master with some cleanup > Mean age of Blocks in cache (seconds) on webUI should be greater than zero > -- > > Key: HBASE-15635 > URL: https://issues.apache.org/jira/browse/HBASE-15635 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.17 >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 1.4.0, 1.0.5, 1.3.1, 1.2.3, 0.98.22, 1.1.7 > > Attachments: 0.98.png, 7BFFAF68-0807-400C-853F-706B498449E1.png, > HBASE-15635.0.98.patch, HBASE-15635.patch, HBASE-15635_v1.patch, master.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15635) Mean age of Blocks in cache (seconds) on webUI should be greater than zero
[ https://issues.apache.org/jira/browse/HBASE-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-15635: -- Attachment: master.png upload test pic on master, looks good to me. > Mean age of Blocks in cache (seconds) on webUI should be greater than zero > -- > > Key: HBASE-15635 > URL: https://issues.apache.org/jira/browse/HBASE-15635 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.17 >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 1.4.0, 1.0.5, 1.3.1, 1.2.3, 0.98.22, 1.1.7 > > Attachments: 0.98.png, 7BFFAF68-0807-400C-853F-706B498449E1.png, > HBASE-15635.0.98.patch, HBASE-15635.patch, master.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15798) Add Async RpcChannels to all RpcClients
[
https://issues.apache.org/jira/browse/HBASE-15798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423786#comment-15423786
]
Duo Zhang commented on HBASE-15798:
---
Sorry a bit late, but how do you use the {{AsyncRpcChannel}} in HBase?
The protobuf stub can only be instantiated using {{RpcChannel}} or
{{BlockingRpcChannel}}.
Thanks.
> Add Async RpcChannels to all RpcClients
> ---
>
> Key: HBASE-15798
> URL: https://issues.apache.org/jira/browse/HBASE-15798
> Project: HBase
> Issue Type: New Feature
>Reporter: Jurriaan Mous
>Assignee: Jurriaan Mous
> Fix For: 2.0.0
>
> Attachments: HBASE-15798-v1.patch, HBASE-15798-v1.patch,
> HBASE-15798-v2.patch, HBASE-15798.patch
>
>
> The RpcClients all need to expose an async protobuf RpcChannel and our own
> custom AsyncRpcChannel (without protobuf overhead) so an Async table
> implementation can be made.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16257) Move staging dir to be under hbase root dir
[
https://issues.apache.org/jira/browse/HBASE-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423778#comment-15423778
]
Jerry He commented on HBASE-16257:
--
Ping for comments. [~enis], [~mbertozzi]?
Thanks.
> Move staging dir to be under hbase root dir
> ---
>
> Key: HBASE-16257
> URL: https://issues.apache.org/jira/browse/HBASE-16257
> Project: HBase
> Issue Type: Sub-task
>Reporter: Jerry He
>Assignee: Jerry He
>Priority: Minor
> Fix For: 2.0.0
>
> Attachments: HBASE-16257-v1.patch, HBASE-16257-v2.patch
>
>
> The hbase.bulkload.staging.dir defaults to hbase.fs.tmp.dir which then
> defaults to
> {code}
> public static final String DEFAULT_TEMPORARY_HDFS_DIRECTORY = "/user/"
> + System.getProperty("user.name") + "/hbase-staging";
> {code}
> This default would have problem on local file system standalone case.
> We can move the staging dir to be under hbase.rootdir. We are bringing
> secure bulkload to the core. It makes sense to bring it under core control as
> well, instead of an optional property.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16427) After HBASE-13701, hbase standalone mode start failed due to mkdir failed
[
https://issues.apache.org/jira/browse/HBASE-16427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423773#comment-15423773
]
Heng Chen commented on HBASE-16427:
---
Thanks [~mbertozzi]
> After HBASE-13701, hbase standalone mode start failed due to mkdir failed
> --
>
> Key: HBASE-16427
> URL: https://issues.apache.org/jira/browse/HBASE-16427
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>
> {code}
> 2016-08-17 10:26:43,305 ERROR [main] regionserver.SecureBulkLoadManager:
> Failed to create or set permission on staging directory
> /user/chenheng/hbase-staging
> ExitCodeException exitCode=1: chmod: /user/chenheng/hbase-staging: No such
> file or directory
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
> at org.apache.hadoop.util.Shell.run(Shell.java:456)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:815)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:798)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:728)
> at
> org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager.start(SecureBulkLoadManager.java:124)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:626)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:406)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:307)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:140)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:221)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:156)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:226)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2421)
> 2016-08-17 10:26:43,306 ERROR [main] master.HMasterCommandLine: Master exiting
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-16427) After HBASE-13701, hbase standalone mode start failed due to mkdir failed
[
https://issues.apache.org/jira/browse/HBASE-16427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matteo Bertozzi resolved HBASE-16427.
-
Resolution: Duplicate
This is the same as HBASE-16292 and the fix will be in HBASE-16257
> After HBASE-13701, hbase standalone mode start failed due to mkdir failed
> --
>
> Key: HBASE-16427
> URL: https://issues.apache.org/jira/browse/HBASE-16427
> Project: HBase
> Issue Type: Bug
>Reporter: Heng Chen
>
> {code}
> 2016-08-17 10:26:43,305 ERROR [main] regionserver.SecureBulkLoadManager:
> Failed to create or set permission on staging directory
> /user/chenheng/hbase-staging
> ExitCodeException exitCode=1: chmod: /user/chenheng/hbase-staging: No such
> file or directory
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
> at org.apache.hadoop.util.Shell.run(Shell.java:456)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:815)
> at org.apache.hadoop.util.Shell.execCommand(Shell.java:798)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:728)
> at
> org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:502)
> at
> org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager.start(SecureBulkLoadManager.java:124)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:626)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:406)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:307)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:140)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:221)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:156)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:226)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at
> org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2421)
> 2016-08-17 10:26:43,306 ERROR [main] master.HMasterCommandLine: Master exiting
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-16426) Remove company affiliations from committer list
[ https://issues.apache.org/jira/browse/HBASE-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-16426: Component/s: website > Remove company affiliations from committer list > --- > > Key: HBASE-16426 > URL: https://issues.apache.org/jira/browse/HBASE-16426 > Project: HBase > Issue Type: Task > Components: documentation, website >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-16426.patch > > > An email thread on the dev mailing list came to the consensus that company > affiliations in the committer list are difficult to keep up to date and not > worth it. This JIRA is to track removing them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16426) Remove company affiliations from committer list
[ https://issues.apache.org/jira/browse/HBASE-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-16426: Status: Patch Available (was: Open) > Remove company affiliations from committer list > --- > > Key: HBASE-16426 > URL: https://issues.apache.org/jira/browse/HBASE-16426 > Project: HBase > Issue Type: Task > Components: documentation, website >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-16426.patch > > > An email thread on the dev mailing list came to the consensus that company > affiliations in the committer list are difficult to keep up to date and not > worth it. This JIRA is to track removing them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16426) Remove company affiliations from committer list
[ https://issues.apache.org/jira/browse/HBASE-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-16426: Component/s: documentation > Remove company affiliations from committer list > --- > > Key: HBASE-16426 > URL: https://issues.apache.org/jira/browse/HBASE-16426 > Project: HBase > Issue Type: Task > Components: documentation, website >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-16426.patch > > > An email thread on the dev mailing list came to the consensus that company > affiliations in the committer list are difficult to keep up to date and not > worth it. This JIRA is to track removing them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16426) Remove company affiliations from committer list
[ https://issues.apache.org/jira/browse/HBASE-16426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-16426: Attachment: HBASE-16426.patch Ready for review. [~apurtell] [~stack] [~busbey] > Remove company affiliations from committer list > --- > > Key: HBASE-16426 > URL: https://issues.apache.org/jira/browse/HBASE-16426 > Project: HBase > Issue Type: Task >Reporter: Misty Stanley-Jones >Assignee: Misty Stanley-Jones > Attachments: HBASE-16426.patch > > > An email thread on the dev mailing list came to the consensus that company > affiliations in the committer list are difficult to keep up to date and not > worth it. This JIRA is to track removing them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16427) After HBASE-13701, hbase standalone mode start failed due to mkdir failed
Heng Chen created HBASE-16427:
-
Summary: After HBASE-13701, hbase standalone mode start failed
due to mkdir failed
Key: HBASE-16427
URL: https://issues.apache.org/jira/browse/HBASE-16427
Project: HBase
Issue Type: Bug
Reporter: Heng Chen
{code}
2016-08-17 10:26:43,305 ERROR [main] regionserver.SecureBulkLoadManager: Failed
to create or set permission on staging directory /user/chenheng/hbase-staging
ExitCodeException exitCode=1: chmod: /user/chenheng/hbase-staging: No such file
or directory
at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:815)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:798)
at
org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:728)
at
org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:502)
at
org.apache.hadoop.hbase.regionserver.SecureBulkLoadManager.start(SecureBulkLoadManager.java:124)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:626)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:406)
at
org.apache.hadoop.hbase.master.HMasterCommandLine$LocalHMaster.(HMasterCommandLine.java:307)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at
org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:140)
at
org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:221)
at
org.apache.hadoop.hbase.LocalHBaseCluster.(LocalHBaseCluster.java:156)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:226)
at
org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:127)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2421)
2016-08-17 10:26:43,306 ERROR [main] master.HMasterCommandLine: Master exiting
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14450) HBase Backup/Restore Phase 3: Multiwal support
[ https://issues.apache.org/jira/browse/HBASE-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14450: --- Attachment: 14450-roll.v2.txt > HBase Backup/Restore Phase 3: Multiwal support > -- > > Key: HBASE-14450 > URL: https://issues.apache.org/jira/browse/HBASE-14450 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Ted Yu > Labels: backup > Fix For: 2.0.0 > > Attachments: 14450-roll.v2.txt, 14450.roll-wait.txt, 14450.v2.txt, > org.apache.hadoop.hbase.backup.TestIncrementalBackup-output.txt, > testIncrementalBackup-8-15.txt > > > We need to support multiwal configurations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15635) Mean age of Blocks in cache (seconds) on webUI should be greater than zero
[ https://issues.apache.org/jira/browse/HBASE-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-15635: -- Attachment: HBASE-15635.0.98.patch Test 0.98 patch on standalone mode, it looks good to me. Upload the pic and the patch > Mean age of Blocks in cache (seconds) on webUI should be greater than zero > -- > > Key: HBASE-15635 > URL: https://issues.apache.org/jira/browse/HBASE-15635 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.17 >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 1.4.0, 1.0.5, 1.3.1, 1.2.3, 0.98.22, 1.1.7 > > Attachments: 0.98.png, 7BFFAF68-0807-400C-853F-706B498449E1.png, > HBASE-15635.0.98.patch, HBASE-15635.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-15635) Mean age of Blocks in cache (seconds) on webUI should be greater than zero
[ https://issues.apache.org/jira/browse/HBASE-15635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-15635: -- Attachment: 0.98.png > Mean age of Blocks in cache (seconds) on webUI should be greater than zero > -- > > Key: HBASE-15635 > URL: https://issues.apache.org/jira/browse/HBASE-15635 > Project: HBase > Issue Type: Bug >Affects Versions: 0.98.17 >Reporter: Heng Chen >Assignee: Heng Chen > Fix For: 2.0.0, 1.4.0, 1.0.5, 1.3.1, 1.2.3, 0.98.22, 1.1.7 > > Attachments: 0.98.png, 7BFFAF68-0807-400C-853F-706B498449E1.png, > HBASE-15635.0.98.patch, HBASE-15635.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16419:
---
Status: Open (was: Patch Available)
Please utilize public enum ScopeType defined in WALProtos.java so that the
check is still valid with future addition of scope.
> check REPLICATION_SCOPE's value more stringent
> --
>
> Key: HBASE-16419
> URL: https://issues.apache.org/jira/browse/HBASE-16419
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 1.2.2, 2.0.0
>Reporter: Guangxu Cheng
> Fix For: 1.2.4
>
> Attachments: HBASE-16419-v1.patch
>
>
> When create table or modify table, the master will check if the value of
> REPLICATION_SCOPE is less than 0, however the value of REPLICATION_SCOPE must
> be 0 or 1. Otherwise will lead to regionserver shutdown, so I think should
> be check the values of REPLICATION_SCOPE more stringent.
> Beginning I don't fully understand the usage of REPLICATION_SCOPE, then set
> REPLICATION_SCOPE to 2 by mistake.when I insert data to table,the
> regionservers abort one by one,finanly
> the cluster abort，the exceptions as follow:
> {quote}
> 2016-08-16 12:34:45,245 WARN [regionserver/host:60023.append-pool1-t1]
> wal.FSHLog: Append sequenceId=94, requesting roll of WAL
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:118)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1886)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2016-08-16 12:34:45,293 INFO [MemStoreFlusher.0] regionserver.HStore: Added
> hdfs://hbase-test-27/hbase1.2.2/data/default/usertable/2aa98c17845c9c6d5c8760b87b3ba09a/i/35825c94e72945c0bf7df3f0adefa1b6,
> entries=1161600, sequenceid=59, filesize=167.6 M
> 2016-08-16 12:34:45,296 FATAL [MemStoreFlusher.0] regionserver.HRegionServer:
> ABORTING region server hbase-10-166-141-99,60023,1471262434177: Replay of WAL
> required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> usertable,,1471262560009.2aa98c17845c9c6d5c8760b87b3ba09a.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2427)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2105)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2067)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1958)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1884)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:510)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:75)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
> Append sequenceId=94, requesting roll of WAL
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1898)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.w
[jira] [Commented] (HBASE-7621) REST server doesn't support binary row keys
[ https://issues.apache.org/jira/browse/HBASE-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423735#comment-15423735 ] Keith David Winkler commented on HBASE-7621: Thanks Andrew, that makes sense. There are probably many folks like me that use RemoteHTable as a (valuable) tool in their own tests/tools for java codebases. And likely there some projects out there that use RemoteHTable in production. The fix for this issue is straightforward, I'll work on a patch. > REST server doesn't support binary row keys > --- > > Key: HBASE-7621 > URL: https://issues.apache.org/jira/browse/HBASE-7621 > Project: HBase > Issue Type: Bug > Components: REST >Affects Versions: 0.94.0, 0.95.2, 0.98.4 >Reporter: Craig Muchinsky > > The REST server doesn't seem to support using binary (MD5 for example) row > keys. I believe the root cause of this is the use of Bytes.toBytes() in the > RowSpec.parseRowKeys() method. Based on the use of Bytes.toStringBinary() > within RemoteHTable.buildRowSpec(), I believe the converse function > Bytes.toBytesBinary() should be used for row key parsing in > RowSpec.parseRowKeys(). > I also noticed that the RemoteHTable.buildRowSpec() method isn't URL encoding > the row key, which is a mismatch to the logic in RowSpec.parseRowKeys() which > performs URL decoding for both the start and stop row keys. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16423) Add re-compare option to VerifyReplication to avoid occasional inconsistent rows
[ https://issues.apache.org/jira/browse/HBASE-16423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423733#comment-15423733 ] Jianwei Cui commented on HBASE-16423: - [~churromorales], there are cases will change the versions between [startTime, endTime) during VerifyReplication scanning. For example, if there are new versions being written to source cluster, the total versions may exceed the family max version so that compaction will cause the versions between [startTime, endTime) deleted, the compaction may happen at different time in source and peer clusters, making VerifyReplication may report inconsistent rows. > Add re-compare option to VerifyReplication to avoid occasional inconsistent > rows > > > Key: HBASE-16423 > URL: https://issues.apache.org/jira/browse/HBASE-16423 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0 >Reporter: Jianwei Cui >Priority: Minor > > Because replication keeps eventually consistency, VerifyReplication may > report inconsistent rows if there are data being written to source or peer > clusters during scanning. These occasionally inconsistent rows will have the > same data if we do the comparison again after a short period. It is not easy > to find the really inconsistent rows if VerifyReplication report a large > number of such occasionally inconsistency. To avoid this case, we can add an > option to make VerifyReplication read out the inconsistent rows again after > sleeping a few seconds and re-compare the rows during scanning. This behavior > follows the eventually consistency of hbase's replication. Suggestions and > discussions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16425) [Operability] Autohandling 'bad data'
[ https://issues.apache.org/jira/browse/HBASE-16425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423675#comment-15423675 ] Jean-Marc Spaggiari commented on HBASE-16425: - I like this thread! Another thing related to the bulk load. If someone bulkloads a cell wich is WAY too big, the region server might not be able to load it. Like, a 2GB cell. And will fail. Might be nice to detect that and alert the user/log the issue/skip the cell... > [Operability] Autohandling 'bad data' > - > > Key: HBASE-16425 > URL: https://issues.apache.org/jira/browse/HBASE-16425 > Project: HBase > Issue Type: Brainstorming > Components: Operability >Reporter: stack > > This is a brainstorming issue. It came up chatting w/ a couple of operators > talking about 'bad data'; i.e. no matter how you control your clients, > someone by mistake or under a misconception will load an out-of-spec Cell or > Row. In this particular case, two types of 'bad data' were talked about: > (on) The Big Cell: An upload of a 'big cell' came in via bulkload but it so > happened that their frontend all arrived at the malignant Cell at the same > time so hundreds of threads requesting the big cell. The RS OOME'd. Then when > the region opened on the new RS, it OOME'd, etc. Could we switch to chunking > when a Server sees that it has a large Cell on its hands? I suppose bulk load > could defeat any Put chunking we had in place but would be good to have this > too. Chatting w/ Matteo, we probably want to just move to the streaming > Interface that we've talked of in the past at various times; the Get would > chunk out the big Cell for assembly on the Client, or just give back the Cell > in pieces -- an OutputStream for the Application to suck on. New API and/or > old API could use it when Cells are big. > (on) The user had a row with 29M Columns in it because the default entity had > id=-1 In this case chunking the Scan (v1.1+) helps but the operator was > having trouble finding the problem row. How could we surface anomalies like > this for operators? On flush, add even more meta data to the HFile (Yahoo! > Data Sketches as [~jleach] has been suggesting) and then an offline tool to > read metadata and run it through a few simple rules. Data Sketches are > mergeable so could build up a region-view or store-view > This is sketchy and I'm pretty sure repeats stuff in old issues but parking > this note here while the encounter still fresh. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16384) Update report-flakies.py script to allow specifying a list of build ids to be excluded
[
https://issues.apache.org/jira/browse/HBASE-16384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423641#comment-15423641
]
Dima Spivak commented on HBASE-16384:
-
Thanks for updating the script, [~appy]; way more readable IMHO. :) Some
suggestions:
- Order imports alphabetically, but grouped as described in PEP 8.
- Can probably remove the brackets around the {{metavar}} for
{{excluded_builds}} to be more clear on how to pass arguments.
- In your new {{get_bad_tests}}, you removed the assignment of {{result}}. I'm
also confused about why the doc suggests you'll return a list, but then you
actually return an empty dictionary sometimes.
- Use {{enumerate}} in place of {{for i in range(len(job_urls))}} to stay
Pythonic.
Other than that, basically looks good to me. It passing locally with and
without your new arguments in place?
> Update report-flakies.py script to allow specifying a list of build ids to be
> excluded
> --
>
> Key: HBASE-16384
> URL: https://issues.apache.org/jira/browse/HBASE-16384
> Project: HBase
> Issue Type: Bug
>Reporter: Appy
>Assignee: Appy
> Attachments: HBASE-16384.master.001.patch,
> HBASE-16384.master.002.patch, HBASE-16384.master.003.patch,
> HBASE-16384.master.004.patch, HBASE-16384.master.005.patch,
> HBASE-16384.master.006.patch
>
>
> Sometimes, builds fail mysteriously and leave lots of tests hanging. This
> makes [flaky
> list|https://builds.apache.org/job/HBase-Find-Flaky-Tests/lastSuccessfulBuild/artifact/dashboard.html]
> go crazy.
> This patch adds that feature to specify build ids to exclude in
> report-flakies.py.
> If we find that a build screwed up, we can exclude it using "exclude=" option
> in --urls param and rerun the job to fix the flaky list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-16426) Remove company affiliations from committer list
Misty Stanley-Jones created HBASE-16426: --- Summary: Remove company affiliations from committer list Key: HBASE-16426 URL: https://issues.apache.org/jira/browse/HBASE-16426 Project: HBase Issue Type: Task Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones An email thread on the dev mailing list came to the consensus that company affiliations in the committer list are difficult to keep up to date and not worth it. This JIRA is to track removing them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14753) TestShell is not invoked anymore
[
https://issues.apache.org/jira/browse/HBASE-14753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-14753.
---
Resolution: Duplicate
Resolving as dup of HBASE-15023 which enabled TestShell
> TestShell is not invoked anymore
>
>
> Key: HBASE-14753
> URL: https://issues.apache.org/jira/browse/HBASE-14753
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
> Fix For: 2.0.0, 1.3.1, 1.2.3
>
>
> Not sure whether it is due to HBASE-14679 or not.
> {code}
> mvn clean test -Dtest=TestShell
> {code}
> does not run any test at all.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-16420) Fix source incompatibility of Table interface
[ https://issues.apache.org/jira/browse/HBASE-16420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423628#comment-15423628 ] Hudson commented on HBASE-16420: FAILURE: Integrated in Jenkins build HBase-1.2 #701 (See [https://builds.apache.org/job/HBase-1.2/701/]) HBASE-16420 Fix source incompatibility of Table interface (Phil Yang) (stack: rev 45beeec0fbfe99bf41d166f00973d0c5efd025c9) * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTablePool.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Table.java * (edit) hbase-rest/src/main/java/org/apache/hadoop/hbase/rest/client/RemoteHTable.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/client/HTableWrapper.java > Fix source incompatibility of Table interface > - > > Key: HBASE-16420 > URL: https://issues.apache.org/jira/browse/HBASE-16420 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.5, 1.2.2 >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 1.1.6, 1.2.3 > > Attachments: HBASE-16420-branch-1.2-v1.patch > > > HBASE-15645 broke source compatibility of Table interface. We need revert the > changing on branch-1.1 and branch-1.2. There is also a bugfix for timeout, so > we should keep the bugfix code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16030) All Regions are flushed at about same time when MEMSTORE_PERIODIC_FLUSH is on, causing flush spike
[ https://issues.apache.org/jira/browse/HBASE-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423625#comment-15423625 ] stack commented on HBASE-16030: --- Moving out of 1.2.3 for now. Put it in 1.2.4. > All Regions are flushed at about same time when MEMSTORE_PERIODIC_FLUSH is > on, causing flush spike > -- > > Key: HBASE-16030 > URL: https://issues.apache.org/jira/browse/HBASE-16030 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.1 >Reporter: Tianying Chang >Assignee: Tianying Chang > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.4 > > Attachments: Screen Shot 2016-06-15 at 11.35.42 PM.png, Screen Shot > 2016-06-15 at 11.52.38 PM.png, hbase-16030-v2.patch, hbase-16030-v3.patch, > hbase-16030.patch > > > In our production cluster, we observed that memstore flush spike every hour > for all regions/RS. (we use the default memstore periodic flush time of 1 > hour). > This will happend when two conditions are met: > 1. the memstore does not have enough data to be flushed before 1 hour limit > reached; > 2. all regions are opened around the same time, (e.g. all RS are started at > the same time when start a cluster). > With above two conditions, all the regions will be flushed around the same > time at: startTime+1hour-delay again and again. > We added a flush jittering time to randomize the flush time of each region, > so that they don't get flushed at around the same time. We had this feature > running in our 94.7 and 94.26 cluster. Recently, we upgrade to 1.2, found > this issue still there in 1.2. So we are porting this into 1.2 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16030) All Regions are flushed at about same time when MEMSTORE_PERIODIC_FLUSH is on, causing flush spike
[ https://issues.apache.org/jira/browse/HBASE-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423622#comment-15423622 ] stack commented on HBASE-16030: --- [~tychang] Any chance of an update to make the patch apply to master? If for branch-1.2, add the name to the patch name to get it applied on the right branch... e.g. hbase-16030.branch-1.2.txt. Thanks! > All Regions are flushed at about same time when MEMSTORE_PERIODIC_FLUSH is > on, causing flush spike > -- > > Key: HBASE-16030 > URL: https://issues.apache.org/jira/browse/HBASE-16030 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.1 >Reporter: Tianying Chang >Assignee: Tianying Chang > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.4 > > Attachments: Screen Shot 2016-06-15 at 11.35.42 PM.png, Screen Shot > 2016-06-15 at 11.52.38 PM.png, hbase-16030-v2.patch, hbase-16030-v3.patch, > hbase-16030.patch > > > In our production cluster, we observed that memstore flush spike every hour > for all regions/RS. (we use the default memstore periodic flush time of 1 > hour). > This will happend when two conditions are met: > 1. the memstore does not have enough data to be flushed before 1 hour limit > reached; > 2. all regions are opened around the same time, (e.g. all RS are started at > the same time when start a cluster). > With above two conditions, all the regions will be flushed around the same > time at: startTime+1hour-delay again and again. > We added a flush jittering time to randomize the flush time of each region, > so that they don't get flushed at around the same time. We had this feature > running in our 94.7 and 94.26 cluster. Recently, we upgrade to 1.2, found > this issue still there in 1.2. So we are porting this into 1.2 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16030) All Regions are flushed at about same time when MEMSTORE_PERIODIC_FLUSH is on, causing flush spike
[ https://issues.apache.org/jira/browse/HBASE-16030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16030: -- Fix Version/s: (was: 1.2.3) 1.2.4 > All Regions are flushed at about same time when MEMSTORE_PERIODIC_FLUSH is > on, causing flush spike > -- > > Key: HBASE-16030 > URL: https://issues.apache.org/jira/browse/HBASE-16030 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.2.1 >Reporter: Tianying Chang >Assignee: Tianying Chang > Fix For: 2.0.0, 1.4.0, 1.3.1, 1.2.4 > > Attachments: Screen Shot 2016-06-15 at 11.35.42 PM.png, Screen Shot > 2016-06-15 at 11.52.38 PM.png, hbase-16030-v2.patch, hbase-16030-v3.patch, > hbase-16030.patch > > > In our production cluster, we observed that memstore flush spike every hour > for all regions/RS. (we use the default memstore periodic flush time of 1 > hour). > This will happend when two conditions are met: > 1. the memstore does not have enough data to be flushed before 1 hour limit > reached; > 2. all regions are opened around the same time, (e.g. all RS are started at > the same time when start a cluster). > With above two conditions, all the regions will be flushed around the same > time at: startTime+1hour-delay again and again. > We added a flush jittering time to randomize the flush time of each region, > so that they don't get flushed at around the same time. We had this feature > running in our 94.7 and 94.26 cluster. Recently, we upgrade to 1.2, found > this issue still there in 1.2. So we are porting this into 1.2 branch. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16424) I get this error when I try to run hbase in standalone mode ./bin/start-hbase.sh
[ https://issues.apache.org/jira/browse/HBASE-16424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423620#comment-15423620 ] Dima Spivak commented on HBASE-16424: - Yeah, I follow. Looking at that documentation in particular, it looks like our Cygwin tutorials haven't gotten much love in over 6 years, so I'm just suggesting you'll have better luck on the user mailing list than on JIRA, which is frequented by developers fixing bugs and adding features. If you (or someone else) would be interested in sorting it out and updating the docs, please feel free to reopen this issue and submit a patch. > I get this error when I try to run hbase in standalone mode > ./bin/start-hbase.sh > > > Key: HBASE-16424 > URL: https://issues.apache.org/jira/browse/HBASE-16424 > Project: HBase > Issue Type: Bug > Environment: Windows + Cygwin, Ubuntu >Reporter: Cheyenne > > cygpath: can't convert empty path > cygpath: can't convert empty path > 2016-08-16 13:25:30,999 ERROR [main] util.Shell: Failed to locate the > winutils binary in the hadoop binary path > java.io.IOException: Could not locate executable null\bin\winutils.exe in the > Hadoop binaries. > at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355) > at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370) > at org.apache.hadoop.util.Shell.(Shell.java:363) > at org.apache.hadoop.util.StringUtils.(StringUtils.java:78) > at > org.apache.hadoop.conf.Configuration.getStrings(Configuration.java:1699) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKPropsFromHbaseConfig(ZKConfig.java:151) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:67) > at > org.apache.hadoop.hbase.zookeeper.ZKServerTool.readZKNodes(ZKServerTool.java:47) > at > org.apache.hadoop.hbase.zookeeper.ZKServerTool.main(ZKServerTool.java:70) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-16419:
--
Fix Version/s: (was: 1.2.3)
1.2.4
> check REPLICATION_SCOPE's value more stringent
> --
>
> Key: HBASE-16419
> URL: https://issues.apache.org/jira/browse/HBASE-16419
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0, 1.2.2
>Reporter: Guangxu Cheng
> Fix For: 1.2.4
>
> Attachments: HBASE-16419-v1.patch
>
>
> When create table or modify table, the master will check if the value of
> REPLICATION_SCOPE is less than 0, however the value of REPLICATION_SCOPE must
> be 0 or 1. Otherwise will lead to regionserver shutdown, so I think should
> be check the values of REPLICATION_SCOPE more stringent.
> Beginning I don't fully understand the usage of REPLICATION_SCOPE, then set
> REPLICATION_SCOPE to 2 by mistake.when I insert data to table,the
> regionservers abort one by one,finanly
> the cluster abort，the exceptions as follow:
> {quote}
> 2016-08-16 12:34:45,245 WARN [regionserver/host:60023.append-pool1-t1]
> wal.FSHLog: Append sequenceId=94, requesting roll of WAL
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:118)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1886)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2016-08-16 12:34:45,293 INFO [MemStoreFlusher.0] regionserver.HStore: Added
> hdfs://hbase-test-27/hbase1.2.2/data/default/usertable/2aa98c17845c9c6d5c8760b87b3ba09a/i/35825c94e72945c0bf7df3f0adefa1b6,
> entries=1161600, sequenceid=59, filesize=167.6 M
> 2016-08-16 12:34:45,296 FATAL [MemStoreFlusher.0] regionserver.HRegionServer:
> ABORTING region server hbase-10-166-141-99,60023,1471262434177: Replay of WAL
> required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> usertable,,1471262560009.2aa98c17845c9c6d5c8760b87b3ba09a.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2427)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2105)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2067)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1958)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1884)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:510)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:75)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
> Append sequenceId=94, requesting roll of WAL
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1898)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWrite
[jira] [Commented] (HBASE-16419) check REPLICATION_SCOPE's value more stringent
[
https://issues.apache.org/jira/browse/HBASE-16419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423619#comment-15423619
]
stack commented on HBASE-16419:
---
Moving out. Discussion still ongoing.
> check REPLICATION_SCOPE's value more stringent
> --
>
> Key: HBASE-16419
> URL: https://issues.apache.org/jira/browse/HBASE-16419
> Project: HBase
> Issue Type: Bug
> Components: master
>Affects Versions: 2.0.0, 1.2.2
>Reporter: Guangxu Cheng
> Fix For: 1.2.4
>
> Attachments: HBASE-16419-v1.patch
>
>
> When create table or modify table, the master will check if the value of
> REPLICATION_SCOPE is less than 0, however the value of REPLICATION_SCOPE must
> be 0 or 1. Otherwise will lead to regionserver shutdown, so I think should
> be check the values of REPLICATION_SCOPE more stringent.
> Beginning I don't fully understand the usage of REPLICATION_SCOPE, then set
> REPLICATION_SCOPE to 2 by mistake.when I insert data to table,the
> regionservers abort one by one,finanly
> the cluster abort，the exceptions as follow:
> {quote}
> 2016-08-16 12:34:45,245 WARN [regionserver/host:60023.append-pool1-t1]
> wal.FSHLog: Append sequenceId=94, requesting roll of WAL
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.append(ProtobufLogWriter.java:118)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1886)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> 2016-08-16 12:34:45,293 INFO [MemStoreFlusher.0] regionserver.HStore: Added
> hdfs://hbase-test-27/hbase1.2.2/data/default/usertable/2aa98c17845c9c6d5c8760b87b3ba09a/i/35825c94e72945c0bf7df3f0adefa1b6,
> entries=1161600, sequenceid=59, filesize=167.6 M
> 2016-08-16 12:34:45,296 FATAL [MemStoreFlusher.0] regionserver.HRegionServer:
> ABORTING region server hbase-10-166-141-99,60023,1471262434177: Replay of WAL
> required. Forcing server shutdown
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> usertable,,1471262560009.2aa98c17845c9c6d5c8760b87b3ba09a.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2427)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2105)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2067)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:1958)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flush(HRegion.java:1884)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:510)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:471)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$900(MemStoreFlusher.java:75)
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:259)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: org.apache.hadoop.hbase.regionserver.wal.DamagedWALException:
> Append sequenceId=94, requesting roll of WAL
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.append(FSHLog.java:1898)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1750)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog$RingBufferEventHandler.onEvent(FSHLog.java:1672)
> at
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> ... 1 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.protobuf.generated.WALProtos$FamilyScope$Builder.setScopeType(WALProtos.java:3939)
> at org.apache.hadoop.hbase.wal.WALKey.getBuilder(WALKey.java:618)
> at
> org.apache.hadoop.hbas
[jira] [Updated] (HBASE-16378) Procedure v2 - Make ProcedureException extend HBaseException
[ https://issues.apache.org/jira/browse/HBASE-16378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16378: -- Fix Version/s: (was: 1.2.3) 1.2.4 > Procedure v2 - Make ProcedureException extend HBaseException > > > Key: HBASE-16378 > URL: https://issues.apache.org/jira/browse/HBASE-16378 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.1.7, 1.2.4 > > Attachments: HBASE-16378-v0.patch, HBASE-16378-v1.patch > > > Make ProcedureException extend HBaseException, so we can avoid stuff like > HBASE-15591. and avoid try/catch ProcedureException and direct rethrows -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16378) Procedure v2 - Make ProcedureException extend HBaseException
[ https://issues.apache.org/jira/browse/HBASE-16378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423615#comment-15423615 ] stack commented on HBASE-16378: --- Moving out for of 1.2.3 for now after chat w/ [~mbertozzi] > Procedure v2 - Make ProcedureException extend HBaseException > > > Key: HBASE-16378 > URL: https://issues.apache.org/jira/browse/HBASE-16378 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0, 1.3.0, 1.1.5, 1.2.2 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.3.0, 1.1.7, 1.2.4 > > Attachments: HBASE-16378-v0.patch, HBASE-16378-v1.patch > > > Make ProcedureException extend HBaseException, so we can avoid stuff like > HBASE-15591. and avoid try/catch ProcedureException and direct rethrows -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16082) Procedure v2 - Move out helpers from MasterProcedureScheduler
[ https://issues.apache.org/jira/browse/HBASE-16082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423614#comment-15423614 ] stack commented on HBASE-16082: --- Moving out for now after chat w/ [~mbertozzi] > Procedure v2 - Move out helpers from MasterProcedureScheduler > - > > Key: HBASE-16082 > URL: https://issues.apache.org/jira/browse/HBASE-16082 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0, 1.3.0, 1.2.1 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.3.1, 1.2.4 > > Attachments: HBASE-16082-v0.patch > > > Move out the helper classes from MasterProcedureScheduler. I plan to use them > in other places. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16082) Procedure v2 - Move out helpers from MasterProcedureScheduler
[ https://issues.apache.org/jira/browse/HBASE-16082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16082: -- Fix Version/s: (was: 1.2.3) 1.2.4 > Procedure v2 - Move out helpers from MasterProcedureScheduler > - > > Key: HBASE-16082 > URL: https://issues.apache.org/jira/browse/HBASE-16082 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Affects Versions: 2.0.0, 1.3.0, 1.2.1 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Trivial > Fix For: 2.0.0, 1.3.1, 1.2.4 > > Attachments: HBASE-16082-v0.patch > > > Move out the helper classes from MasterProcedureScheduler. I plan to use them > in other places. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16420) Fix source incompatibility of Table interface
[ https://issues.apache.org/jira/browse/HBASE-16420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-16420: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 1.2.3 1.1.6 Status: Resolved (was: Patch Available) Pushed to branch-1.1 and to branch-1.2. FYI [~busbey] and [~ndimiduk] (Nick am I presumption marking this fixed in 1.1.6?) Thanks for the fast turnaround Phil. > Fix source incompatibility of Table interface > - > > Key: HBASE-16420 > URL: https://issues.apache.org/jira/browse/HBASE-16420 > Project: HBase > Issue Type: Bug >Affects Versions: 1.1.5, 1.2.2 >Reporter: Phil Yang >Assignee: Phil Yang > Fix For: 1.1.6, 1.2.3 > > Attachments: HBASE-16420-branch-1.2-v1.patch > > > HBASE-15645 broke source compatibility of Table interface. We need revert the > changing on branch-1.1 and branch-1.2. There is also a bugfix for timeout, so > we should keep the bugfix code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16424) I get this error when I try to run hbase in standalone mode ./bin/start-hbase.sh
[ https://issues.apache.org/jira/browse/HBASE-16424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423607#comment-15423607 ] Cheyenne commented on HBASE-16424: -- I am not using it in production, I dont even have hadoop installed, I am trying to use it in standalone mode I followed the instructions here: https://hbase.apache.org/cygwin.html > I get this error when I try to run hbase in standalone mode > ./bin/start-hbase.sh > > > Key: HBASE-16424 > URL: https://issues.apache.org/jira/browse/HBASE-16424 > Project: HBase > Issue Type: Bug > Environment: Windows + Cygwin, Ubuntu >Reporter: Cheyenne > > cygpath: can't convert empty path > cygpath: can't convert empty path > 2016-08-16 13:25:30,999 ERROR [main] util.Shell: Failed to locate the > winutils binary in the hadoop binary path > java.io.IOException: Could not locate executable null\bin\winutils.exe in the > Hadoop binaries. > at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355) > at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370) > at org.apache.hadoop.util.Shell.(Shell.java:363) > at org.apache.hadoop.util.StringUtils.(StringUtils.java:78) > at > org.apache.hadoop.conf.Configuration.getStrings(Configuration.java:1699) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKPropsFromHbaseConfig(ZKConfig.java:151) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:67) > at > org.apache.hadoop.hbase.zookeeper.ZKServerTool.readZKNodes(ZKServerTool.java:47) > at > org.apache.hadoop.hbase.zookeeper.ZKServerTool.main(ZKServerTool.java:70) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-16424) I get this error when I try to run hbase in standalone mode ./bin/start-hbase.sh
[ https://issues.apache.org/jira/browse/HBASE-16424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dima Spivak resolved HBASE-16424. - Resolution: Invalid > I get this error when I try to run hbase in standalone mode > ./bin/start-hbase.sh > > > Key: HBASE-16424 > URL: https://issues.apache.org/jira/browse/HBASE-16424 > Project: HBase > Issue Type: Bug > Environment: Windows + Cygwin, Ubuntu >Reporter: Cheyenne > > cygpath: can't convert empty path > cygpath: can't convert empty path > 2016-08-16 13:25:30,999 ERROR [main] util.Shell: Failed to locate the > winutils binary in the hadoop binary path > java.io.IOException: Could not locate executable null\bin\winutils.exe in the > Hadoop binaries. > at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355) > at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370) > at org.apache.hadoop.util.Shell.(Shell.java:363) > at org.apache.hadoop.util.StringUtils.(StringUtils.java:78) > at > org.apache.hadoop.conf.Configuration.getStrings(Configuration.java:1699) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKPropsFromHbaseConfig(ZKConfig.java:151) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:67) > at > org.apache.hadoop.hbase.zookeeper.ZKServerTool.readZKNodes(ZKServerTool.java:47) > at > org.apache.hadoop.hbase.zookeeper.ZKServerTool.main(ZKServerTool.java:70) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16094) Procedure v2 - Improve cleaning up of proc wals
[
https://issues.apache.org/jira/browse/HBASE-16094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423594#comment-15423594
]
Hadoop QA commented on HBASE-16094:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s {color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} master passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} master passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 58s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 48s
{color} | {color:green} hbase-procedure in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 11s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-16 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12824009/HBASE-16094.master.003.patch
|
| JIRA Issue | HBASE-16094 |
| Optional Tests | asflicense javac javadoc unit findbugs hadoopcheck
hbaseanti checkstyle compile |
| uname | Linux af4e5c869731 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d5080e8 |
| Default Java | 1.7.0_101 |
| Multi-JDK versions |
[jira] [Commented] (HBASE-13094) Consider Filters that are evaluated before deletes and see delete markers
[ https://issues.apache.org/jira/browse/HBASE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423556#comment-15423556 ] Andrew Purtell commented on HBASE-13094: bq. When SQM detects this class, it is will evaluate the wrapped filter's filterKeyValue method before delete markers and columns are filtered. Hmm. Feels like special magic to me. I'd go for #4. It's explicit. It allows for registration of a filter list, although I suppose "PreDeleteFilter" could become an empty interface used for tagging and then something could extend FilterList and implement that tagging iface... (EVen more hacky) > Consider Filters that are evaluated before deletes and see delete markers > - > > Key: HBASE-13094 > URL: https://issues.apache.org/jira/browse/HBASE-13094 > Project: HBase > Issue Type: Brainstorming > Components: regionserver, Scanners >Reporter: Lars Hofhansl > Attachments: 13094-0.98.txt > > > That would be good for full control filtering of all cells, such as needed > for some transaction implementations. > [~ghelmling] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16094) Procedure v2 - Improve cleaning up of proc wals
[ https://issues.apache.org/jira/browse/HBASE-16094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-16094: - Attachment: HBASE-16094.master.003.patch > Procedure v2 - Improve cleaning up of proc wals > --- > > Key: HBASE-16094 > URL: https://issues.apache.org/jira/browse/HBASE-16094 > Project: HBase > Issue Type: Sub-task > Components: proc-v2 >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0 > > Attachments: HBASE-16094.master.001.patch, > HBASE-16094.master.002.patch, HBASE-16094.master.003.patch > > > Avoid accumulating too many wals. > We remove logs in 3 cases: > - No procedures are running. We can remove all the logs. > - All procedures are updated/written in the last log. We can remove all the > logs, aside the active one. > - Remove log if does not contains “active” procIds > https://github.com/apache/hbase/blob/master/hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/store/wal/WALProcedureStore.java#L865 > The last point, Remove log if does not contains “active” procIds, needs to be > improved. Basically, even if a log contains state of an active proc, we can > delete it if a later log contains a newer state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14450) HBase Backup/Restore Phase 3: Multiwal support
[
https://issues.apache.org/jira/browse/HBASE-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423524#comment-15423524
]
Ted Yu commented on HBASE-14450:
Gone through one test output from today looking for WALs involved with
incremental restore:
{code}
2016-08-16 10:13:35,907 INFO [ProcedureExecutor-13]
master.IncrementalTableBackupProcedure(176): Incremental copy from
hdfs://localhost:55522/user/hbase/test-data/03056445-e200-43be-b5b9-
a64f6466dd12/WALs/cn012.l42scl.hortonworks.com,39243,1471367526761/cn012.l42scl.hortonworks.com%2C39243%2C1471367526761.regiongroup-1.1471367533183,hdfs://localhost:55522/user/hbase/test-data/03056445-e200-43be-b5b9-a64f6466dd12/WALs/cn012.l42scl.hortonworks.com,39243,1471367526761/cn012.l42scl.hortonworks.com%2C39243%2C1471367526761.regiongroup-1.1471367567459,hdfs://localhost:
55522/user/hbase/test-data/03056445-e200-43be-b5b9-a64f6466dd12/WALs/cn012.l42scl.hortonworks.com,39243,1471367526761/cn012.l42scl.hortonworks.com%2C39243%2C1471367526761.regiongroup-2.
1471367567039,hdfs://localhost:55522/user/hbase/test-data/03056445-e200-43be-b5b9-a64f6466dd12/WALs/cn012.l42scl.hortonworks.com,43196,1471367526436/cn012.l42scl.hortonworks.
com%2C43196%2C1471367526436.regiongroup-1.1471367530064,hdfs://localhost:55522/user/hbase/test-data/03056445-e200-43be-b5b9-a64f6466dd12/oldWALs/cn012.l42scl.hortonworks.
com%2C39243%2C1471367526761.regiongroup-0.1471367567893,hdfs://localhost:55522/user/hbase/test-data/03056445-e200-43be-b5b9-a64f6466dd12/oldWALs/cn012.l42scl.hortonworks.
com%2C39243%2C1471367526761.regiongroup-2.1471367558565,hdfs://localhost:55522/user/hbase/test-data/03056445-e200-43be-b5b9-a64f6466dd12/oldWALs/cn012.l42scl.hortonworks.
com%2C39243%2C1471367526761.regiongroup-3.1471367561067,hdfs://localhost:55522/user/hbase/test-data/03056445-e200-43be-b5b9-a64f6466dd12/oldWALs/cn012.l42scl.hortonworks.
com%2C43196%2C1471367526436.regiongroup-0.1471367567459,hdfs://localhost:55522/user/hbase/test-data/03056445-e200-43be-b5b9-a64f6466dd12/oldWALs/cn012.l42scl.hortonworks.
com%2C43196%2C1471367526436.regiongroup-1.1471367567039
to hdfs://localhost:55522/backupUT/backup_1471367605362/WALs finished.
...
2016-08-16 10:13:58,053 INFO [main] impl.RestoreClientImpl(284): Restoring
'ns1:test-1471367554256' to 'ns1:table1_restore' from log dirs:
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs
...
2016-08-16 10:14:08,310 DEBUG [main] mapreduce.WALInputFormat(263): Scanning
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs for WAL files
2016-08-16 10:14:08,311 WARN [main] mapreduce.WALInputFormat(286): File
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs/.backup.manifest does
not appear to be an WAL file. Skipping...
2016-08-16 10:14:08,311 INFO [main] mapreduce.WALInputFormat(278): Found:
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs/cn012.l42scl.hortonworks.com%2C39243%2C1471367526761.
regiongroup-0.1471367567893
2016-08-16 10:14:08,311 INFO [main] mapreduce.WALInputFormat(278): Found:
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs/cn012.l42scl.hortonworks.com%2C39243%2C1471367526761.
regiongroup-1.1471367533183
2016-08-16 10:14:08,311 INFO [main] mapreduce.WALInputFormat(278): Found:
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs/cn012.l42scl.hortonworks.com%2C39243%2C1471367526761.
regiongroup-1.1471367567459
2016-08-16 10:14:08,312 INFO [main] mapreduce.WALInputFormat(278): Found:
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs/cn012.l42scl.hortonworks.com%2C39243%2C1471367526761.
regiongroup-2.1471367558565
2016-08-16 10:14:08,312 INFO [main] mapreduce.WALInputFormat(278): Found:
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs/cn012.l42scl.hortonworks.com%2C39243%2C1471367526761.
regiongroup-2.1471367567039
2016-08-16 10:14:08,312 INFO [main] mapreduce.WALInputFormat(278): Found:
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs/cn012.l42scl.hortonworks.com%2C39243%2C1471367526761.
regiongroup-3.1471367561067
2016-08-16 10:14:08,312 INFO [main] mapreduce.WALInputFormat(278): Found:
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs/cn012.l42scl.hortonworks.com%2C43196%2C1471367526436.
regiongroup-0.1471367567459
2016-08-16 10:14:08,312 INFO [main] mapreduce.WALInputFormat(278): Found:
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs/cn012.l42scl.hortonworks.com%2C43196%2C1471367526436.
regiongroup-1.1471367530064
2016-08-16 10:14:08,312 INFO [main] mapreduce.WALInputFormat(278): Found:
hdfs://localhost:55522/backupUT/backup_1471367605362/WALs/cn012.l42scl.hortonworks.com%2C43196%2C1471367526436.
regiongroup-1.1471367567039
...
2016
[jira] [Commented] (HBASE-16295) InvalidFamilyOperationException while deleting a column family in shell
[
https://issues.apache.org/jira/browse/HBASE-16295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423495#comment-15423495
]
Stephen Yuan Jiang commented on HBASE-16295:
I could not repro this. My guess just as before that the column family
probably was deleted when you issued the command. If you cannot repro, we can
close the issue for now until we have a repro.
> InvalidFamilyOperationException while deleting a column family in shell
> ---
>
> Key: HBASE-16295
> URL: https://issues.apache.org/jira/browse/HBASE-16295
> Project: HBase
> Issue Type: Bug
> Components: master, shell
>Affects Versions: 1.2.0
>Reporter: Ashu Pachauri
>Priority: Minor
>
> The column family exists and is actually deleted, the regions are also
> reopened. But, the following exception is thrown in the shell:
> {code}
> alter 't1', 'delete' => 'cf'
> ERROR: org.apache.hadoop.hbase.InvalidFamilyOperationException: Family 'cf'
> does not exist, so it cannot be deleted
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
> at
> org.apache.hadoop.hbase.util.ForeignExceptionUtil.toIOException(ForeignExceptionUtil.java:45)
> at
> org.apache.hadoop.hbase.procedure2.RemoteProcedureException.fromProto(RemoteProcedureException.java:114)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.waitForProcedureToComplete(ProcedureSyncWait.java:85)
> at
> org.apache.hadoop.hbase.master.HMaster.deleteColumn(HMaster.java:1916)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.deleteColumn(MasterRpcServices.java:474)
> at
> org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:55658)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2170)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:109)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:137)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:112)
> at java.lang.Thread.run(Thread.java:745)
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hbase.InvalidFamilyOperationException):
> Family 'cf' does not exist, so it cannot be deleted
> at
> org.apache.hadoop.hbase.master.procedure.DeleteColumnFamilyProcedure.prepareDelete(DeleteColumnFamilyProcedure.java:281)
> at
> org.apache.hadoop.hbase.master.procedure.DeleteColumnFamilyProcedure.executeFromState(DeleteColumnFamilyProcedure.java:93)
> at
> org.apache.hadoop.hbase.master.procedure.DeleteColumnFamilyProcedure.executeFromState(DeleteColumnFamilyProcedure.java:48)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:119)
> at
> org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:465)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1061)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:856)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execLoop(ProcedureExecutor.java:809)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$400(ProcedureExecutor.java:75)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$2.run(ProcedureExecutor.java:495)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13094) Consider Filters that are evaluated before deletes and see delete markers
[ https://issues.apache.org/jira/browse/HBASE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-13094: -- Attachment: 13094-0.98.txt Here's a pretty trivial patch doing #1. Defines a PreDeleteFilter class that can wrap any other filter. When SQM detects this class, it is will evaluate the wrapped filter's filterKeyValue method before delete markers and columns are filtered. Also creates a convenience BaseDelegateFilter class that can be used to implement Filter delegates. Warning: COMPLETELY UNTESTED. Just want to park it here. Probably a it hacky. > Consider Filters that are evaluated before deletes and see delete markers > - > > Key: HBASE-13094 > URL: https://issues.apache.org/jira/browse/HBASE-13094 > Project: HBase > Issue Type: Brainstorming > Components: regionserver, Scanners >Reporter: Lars Hofhansl > Attachments: 13094-0.98.txt > > > That would be good for full control filtering of all cells, such as needed > for some transaction implementations. > [~ghelmling] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16425) [Operability] Autohandling 'bad data'
stack created HBASE-16425: - Summary: [Operability] Autohandling 'bad data' Key: HBASE-16425 URL: https://issues.apache.org/jira/browse/HBASE-16425 Project: HBase Issue Type: Brainstorming Components: Operability Reporter: stack This is a brainstorming issue. It came up chatting w/ a couple of operators talking about 'bad data'; i.e. no matter how you control your clients, someone by mistake or under a misconception will load an out-of-spec Cell or Row. In this particular case, two types of 'bad data' were talked about: (on) The Big Cell: An upload of a 'big cell' came in via bulkload but it so happened that their frontend all arrived at the malignant Cell at the same time so hundreds of threads requesting the big cell. The RS OOME'd. Then when the region opened on the new RS, it OOME'd, etc. Could we switch to chunking when a Server sees that it has a large Cell on its hands? I suppose bulk load could defeat any Put chunking we had in place but would be good to have this too. Chatting w/ Matteo, we probably want to just move to the streaming Interface that we've talked of in the past at various times; the Get would chunk out the big Cell for assembly on the Client, or just give back the Cell in pieces -- an OutputStream for the Application to suck on. New API and/or old API could use it when Cells are big. (on) The user had a row with 29M Columns in it because the default entity had id=-1 In this case chunking the Scan (v1.1+) helps but the operator was having trouble finding the problem row. How could we surface anomalies like this for operators? On flush, add even more meta data to the HFile (Yahoo! Data Sketches as [~jleach] has been suggesting) and then an offline tool to read metadata and run it through a few simple rules. Data Sketches are mergeable so could build up a region-view or store-view This is sketchy and I'm pretty sure repeats stuff in old issues but parking this note here while the encounter still fresh. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16423) Add re-compare option to VerifyReplication to avoid occasional inconsistent rows
[ https://issues.apache.org/jira/browse/HBASE-16423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423444#comment-15423444 ] churro morales commented on HBASE-16423: VerifyReplication has a startTime and endTime parameter. You know your replication lag from the metrics on the source. So if you are running your VerifyReplication job and your time is NOW and the max(lag) = x. You can just set endTime=NOW - x > Add re-compare option to VerifyReplication to avoid occasional inconsistent > rows > > > Key: HBASE-16423 > URL: https://issues.apache.org/jira/browse/HBASE-16423 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 2.0.0 >Reporter: Jianwei Cui >Priority: Minor > > Because replication keeps eventually consistency, VerifyReplication may > report inconsistent rows if there are data being written to source or peer > clusters during scanning. These occasionally inconsistent rows will have the > same data if we do the comparison again after a short period. It is not easy > to find the really inconsistent rows if VerifyReplication report a large > number of such occasionally inconsistency. To avoid this case, we can add an > option to make VerifyReplication read out the inconsistent rows again after > sleeping a few seconds and re-compare the rows during scanning. This behavior > follows the eventually consistency of hbase's replication. Suggestions and > discussions are welcomed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16304) regionserver should shutdown but it is blocked
[
https://issues.apache.org/jira/browse/HBASE-16304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423362#comment-15423362
]
Ted Yu commented on HBASE-16304:
Ran TestMasterOperationsForRegionReplicas and TestMultiTableSnapshotInputFormat
with patch in branch-1.2
They passed.
> regionserver should shutdown but it is blocked
> --
>
> Key: HBASE-16304
> URL: https://issues.apache.org/jira/browse/HBASE-16304
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.2
>Reporter: mingmin xu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: 16304.branch-1.2.v4.txt, 16304.branch-1.2.v5.txt,
> 16304.branch-1.2.v5.txt, 16304.branch-1.v1.txt, 16304.v1.txt, 16304.v3.txt,
> 16304.v4.txt, 16304.v4.txt, 16304.v5.txt
>
>
> here is my jvm stack:
> {code}
> 2016-07-29 16:36:56
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.72-b04 mixed mode):
> "Timer for 'HBase' metrics system" daemon prio=10 tid=0x7f205cf38000
> nid=0xafa5 in Object.wait() [0x7f203b353000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.util.TimerThread.mainLoop(Timer.java:552)
> - locked <0x00063503c790> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:505)
> "Attach Listener" daemon prio=10 tid=0x7f205d017800 nid=0x1300 waiting on
> condition [0x]
>java.lang.Thread.State: RUNNABLE
> "IPC Parameter Sending Thread #2" daemon prio=10 tid=0x7f205c7c4000
> nid=0x4f1a waiting on condition [0x7f20362e1000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00066f996718> (a
> java.util.concurrent.SynchronousQueue$TransferStack)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
> at
> java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
> at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> "RS_LOG_REPLAY_OPS-hadoop-datanode-0042:16020-1" prio=10
> tid=0x7f2054ec8000 nid=0x832d waiting on condition [0x7f2039a18000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00066ffb5950> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> "RS_LOG_REPLAY_OPS-hadoop-datanode-0042:16020-0" prio=10
> tid=0x7f20542ca800 nid=0x5a5d waiting on condition [0x7f2033bba000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00066ffb5950> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> "hadoop-datanode-0042.corp.cootek.com,16020,1469690065288_ChoreService_2"
> daemon prio=10 tid=0x7f205d0d4000 nid=0x72af waiting on condition
> [0x7f203b151000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait
[jira] [Updated] (HBASE-16411) BackupDistCp#execute() should throw exception if the distcp job fails
[ https://issues.apache.org/jira/browse/HBASE-16411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-16411: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Thanks for the review, Jerry. > BackupDistCp#execute() should throw exception if the distcp job fails > - > > Key: HBASE-16411 > URL: https://issues.apache.org/jira/browse/HBASE-16411 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Labels: backup > Attachments: 16411.v1.txt > > > In some case, IncrementalTableBackupProcedure#incrementalCopy() wouldn't > detect the failed job, leading to potential data loss. > BackupDistCp#execute() should throw exception if the wrapped distcp job > fails. This would allow IncrementalTableBackupProcedure#incrementalCopy() to > properly respond to job failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423303#comment-15423303
]
Hadoop QA commented on HBASE-12721:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 3s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} pylint {color} | {color:red} 0m 3s {color}
| {color:red} The patch generated 9 new + 0 unchanged - 0 fixed = 9 total (was
0) {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 9s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 13s
{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 1m 34s
{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 24s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-16 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12823973/HBASE-12721_v7.patch |
| JIRA Issue | HBASE-12721 |
| Optional Tests | asflicense pylint shellcheck shelldocs |
| uname | Linux a6437a449803 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d5080e8 |
| shellcheck | v0.4.4 |
| pylint | v1.6.4 |
| pylint |
https://builds.apache.org/job/PreCommit-HBASE-Build/3111/artifact/patchprocess/diff-patch-pylint.txt
|
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/3111/artifact/patchprocess/patch-hbaseprotoc-root.txt
|
| asflicense |
https://builds.apache.org/job/PreCommit-HBASE-Build/3111/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3111/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
> Components: build, community, documentation, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
> Attachments: HBASE-12721_v5.patch, HBASE-12721_v6.patch,
> HBASE-12721_v7.patch
>
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
[jira] [Commented] (HBASE-16411) BackupDistCp#execute() should throw exception if the distcp job fails
[ https://issues.apache.org/jira/browse/HBASE-16411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423292#comment-15423292 ] Jerry He commented on HBASE-16411: -- Patch looks good. > BackupDistCp#execute() should throw exception if the distcp job fails > - > > Key: HBASE-16411 > URL: https://issues.apache.org/jira/browse/HBASE-16411 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Labels: backup > Attachments: 16411.v1.txt > > > In some case, IncrementalTableBackupProcedure#incrementalCopy() wouldn't > detect the failed job, leading to potential data loss. > BackupDistCp#execute() should throw exception if the wrapped distcp job > fails. This would allow IncrementalTableBackupProcedure#incrementalCopy() to > properly respond to job failure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16304) regionserver should shutdown but it is blocked
[
https://issues.apache.org/jira/browse/HBASE-16304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423295#comment-15423295
]
Hadoop QA commented on HBASE-16304:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} patch {color} | {color:blue} 0m 1s
{color} | {color:blue} The patch file was not named according to hbase's naming
conventions. Please see
https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for
instructions. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
9s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
11s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
26s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
56s {color} | {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} branch-1.2 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 34s
{color} | {color:green} branch-1.2 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 33s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 37s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
17s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 26s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green} 0m
14s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 32s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 72m 33s {color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 110m 48s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hbase.master.TestMasterOperationsForRegionReplicas |
| | hadoop.hbase.mapreduce.TestMultiTableSnapshotInputFormat |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetu
[jira] [Updated] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak updated HBASE-12721:
Attachment: HBASE-12721_v7.patch
One last patch... 1 warning was real, but the others are locally disabled. I
think I'll need to open a separate JIRA for our Yetus personality, [~busbey],
to not consider those intentionally-ignored warnings to be problems.
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
> Components: build, community, documentation, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
> Attachments: HBASE-12721_v5.patch, HBASE-12721_v6.patch,
> HBASE-12721_v7.patch
>
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423263#comment-15423263
]
Hadoop QA commented on HBASE-12721:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 3s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} pylint {color} | {color:red} 0m 3s {color}
| {color:red} The patch generated 10 new + 0 unchanged - 0 fixed = 10 total
(was 0) {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 22s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 10s
{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 13s
{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 11s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-16 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12823969/HBASE-12721_v6.patch |
| JIRA Issue | HBASE-12721 |
| Optional Tests | asflicense pylint shellcheck shelldocs |
| uname | Linux 0fd99684a984 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d5080e8 |
| shellcheck | v0.4.4 |
| pylint | v1.6.4 |
| pylint |
https://builds.apache.org/job/PreCommit-HBASE-Build/3110/artifact/patchprocess/diff-patch-pylint.txt
|
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/3110/artifact/patchprocess/patch-hbaseprotoc-root.txt
|
| asflicense |
https://builds.apache.org/job/PreCommit-HBASE-Build/3110/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3110/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
> Components: build, community, documentation, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
> Attachments: HBASE-12721_v5.patch, HBASE-12721_v6.patch
>
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Ap
[jira] [Commented] (HBASE-16424) I get this error when I try to run hbase in standalone mode ./bin/start-hbase.sh
[ https://issues.apache.org/jira/browse/HBASE-16424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423225#comment-15423225 ] Dima Spivak commented on HBASE-16424: - Hi Cheyenne, Running HBase in Windows + Cygwin is not supported in production, so is not well taken care of much in practice. If you'd like to get advice on how to get this working, you're probably best off subscribing to and then sending an email to the [user@ mailing list|mailto:u...@hbase.apache.org]. > I get this error when I try to run hbase in standalone mode > ./bin/start-hbase.sh > > > Key: HBASE-16424 > URL: https://issues.apache.org/jira/browse/HBASE-16424 > Project: HBase > Issue Type: Bug > Environment: Windows + Cygwin, Ubuntu >Reporter: Cheyenne > > cygpath: can't convert empty path > cygpath: can't convert empty path > 2016-08-16 13:25:30,999 ERROR [main] util.Shell: Failed to locate the > winutils binary in the hadoop binary path > java.io.IOException: Could not locate executable null\bin\winutils.exe in the > Hadoop binaries. > at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355) > at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370) > at org.apache.hadoop.util.Shell.(Shell.java:363) > at org.apache.hadoop.util.StringUtils.(StringUtils.java:78) > at > org.apache.hadoop.conf.Configuration.getStrings(Configuration.java:1699) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKPropsFromHbaseConfig(ZKConfig.java:151) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:67) > at > org.apache.hadoop.hbase.zookeeper.ZKServerTool.readZKNodes(ZKServerTool.java:47) > at > org.apache.hadoop.hbase.zookeeper.ZKServerTool.main(ZKServerTool.java:70) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-16424) I get this error when I try to run hbase in standalone mode ./bin/start-hbase.sh
Cheyenne created HBASE-16424: Summary: I get this error when I try to run hbase in standalone mode ./bin/start-hbase.sh Key: HBASE-16424 URL: https://issues.apache.org/jira/browse/HBASE-16424 Project: HBase Issue Type: Bug Environment: Windows Cygwin Ubuntu Reporter: Cheyenne cygpath: can't convert empty path cygpath: can't convert empty path 2016-08-16 13:25:30,999 ERROR [main] util.Shell: Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355) at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370) at org.apache.hadoop.util.Shell.(Shell.java:363) at org.apache.hadoop.util.StringUtils.(StringUtils.java:78) at org.apache.hadoop.conf.Configuration.getStrings(Configuration.java:1699) at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKPropsFromHbaseConfig(ZKConfig.java:151) at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:67) at org.apache.hadoop.hbase.zookeeper.ZKServerTool.readZKNodes(ZKServerTool.java:47) at org.apache.hadoop.hbase.zookeeper.ZKServerTool.main(ZKServerTool.java:70) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16424) I get this error when I try to run hbase in standalone mode ./bin/start-hbase.sh
[ https://issues.apache.org/jira/browse/HBASE-16424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheyenne updated HBASE-16424: - Environment: Windows + Cygwin, Ubuntu (was: Windows Cygwin Ubuntu) > I get this error when I try to run hbase in standalone mode > ./bin/start-hbase.sh > > > Key: HBASE-16424 > URL: https://issues.apache.org/jira/browse/HBASE-16424 > Project: HBase > Issue Type: Bug > Environment: Windows + Cygwin, Ubuntu >Reporter: Cheyenne > > cygpath: can't convert empty path > cygpath: can't convert empty path > 2016-08-16 13:25:30,999 ERROR [main] util.Shell: Failed to locate the > winutils binary in the hadoop binary path > java.io.IOException: Could not locate executable null\bin\winutils.exe in the > Hadoop binaries. > at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355) > at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370) > at org.apache.hadoop.util.Shell.(Shell.java:363) > at org.apache.hadoop.util.StringUtils.(StringUtils.java:78) > at > org.apache.hadoop.conf.Configuration.getStrings(Configuration.java:1699) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKPropsFromHbaseConfig(ZKConfig.java:151) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:67) > at > org.apache.hadoop.hbase.zookeeper.ZKServerTool.readZKNodes(ZKServerTool.java:47) > at > org.apache.hadoop.hbase.zookeeper.ZKServerTool.main(ZKServerTool.java:70) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak updated HBASE-12721:
Attachment: HBASE-12721_v6.patch
Let's try to clean up those pesky pylint warnings...
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
> Components: build, community, documentation, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
> Attachments: HBASE-12721_v5.patch, HBASE-12721_v6.patch
>
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Work started] (HBASE-15448) HBase Backup Phase 3: Restore optimization 2
[ https://issues.apache.org/jira/browse/HBASE-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-15448 started by Vladimir Rodionov. - > HBase Backup Phase 3: Restore optimization 2 > > > Key: HBASE-15448 > URL: https://issues.apache.org/jira/browse/HBASE-15448 > Project: HBase > Issue Type: Task >Affects Versions: 2.0.0 >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Critical > Labels: backup > Fix For: 2.0.0 > > > JIRA opened to continue work on restore optimization. > This will focus on the following > # During incremental backup image restore - restoring full backup into region > boundaries of the most recent incremental backup image. > # Combining multiple tables into single M/R job -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423184#comment-15423184
]
Hadoop QA commented on HBASE-12721:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s
{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m 3s
{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s {color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} pylint {color} | {color:red} 0m 3s {color}
| {color:red} The patch generated 15 new + 0 unchanged - 0 fixed = 15 total
(was 0) {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
4s {color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
27m 34s {color} | {color:green} Patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 10s
{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 12s
{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 23s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=1.11.2 Server=1.11.2 Image:yetus/hbase:date2016-08-16 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12823959/HBASE-12721_v5.patch |
| JIRA Issue | HBASE-12721 |
| Optional Tests | asflicense pylint shellcheck shelldocs |
| uname | Linux a4e33b7794fd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / d5080e8 |
| shellcheck | v0.4.4 |
| pylint | v1.6.4 |
| pylint |
https://builds.apache.org/job/PreCommit-HBASE-Build/3108/artifact/patchprocess/diff-patch-pylint.txt
|
| hbaseprotoc |
https://builds.apache.org/job/PreCommit-HBASE-Build/3108/artifact/patchprocess/patch-hbaseprotoc-root.txt
|
| asflicense |
https://builds.apache.org/job/PreCommit-HBASE-Build/3108/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/3108/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
> Components: build, community, documentation, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
> Attachments: HBASE-12721_v5.patch
>
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
[jira] [Updated] (HBASE-16304) regionserver should shutdown but it is blocked
[
https://issues.apache.org/jira/browse/HBASE-16304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-16304:
---
Attachment: 16304.branch-1.2.v5.txt
> regionserver should shutdown but it is blocked
> --
>
> Key: HBASE-16304
> URL: https://issues.apache.org/jira/browse/HBASE-16304
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.2
>Reporter: mingmin xu
>Assignee: Ted Yu
>Priority: Critical
> Fix For: 2.0.0, 1.3.0, 1.4.0
>
> Attachments: 16304.branch-1.2.v4.txt, 16304.branch-1.2.v5.txt,
> 16304.branch-1.2.v5.txt, 16304.branch-1.v1.txt, 16304.v1.txt, 16304.v3.txt,
> 16304.v4.txt, 16304.v4.txt, 16304.v5.txt
>
>
> here is my jvm stack:
> {code}
> 2016-07-29 16:36:56
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.72-b04 mixed mode):
> "Timer for 'HBase' metrics system" daemon prio=10 tid=0x7f205cf38000
> nid=0xafa5 in Object.wait() [0x7f203b353000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.util.TimerThread.mainLoop(Timer.java:552)
> - locked <0x00063503c790> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:505)
> "Attach Listener" daemon prio=10 tid=0x7f205d017800 nid=0x1300 waiting on
> condition [0x]
>java.lang.Thread.State: RUNNABLE
> "IPC Parameter Sending Thread #2" daemon prio=10 tid=0x7f205c7c4000
> nid=0x4f1a waiting on condition [0x7f20362e1000]
>java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00066f996718> (a
> java.util.concurrent.SynchronousQueue$TransferStack)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
> at
> java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
> at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> "RS_LOG_REPLAY_OPS-hadoop-datanode-0042:16020-1" prio=10
> tid=0x7f2054ec8000 nid=0x832d waiting on condition [0x7f2039a18000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00066ffb5950> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> "RS_LOG_REPLAY_OPS-hadoop-datanode-0042:16020-0" prio=10
> tid=0x7f20542ca800 nid=0x5a5d waiting on condition [0x7f2033bba000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00066ffb5950> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> "hadoop-datanode-0042.corp.cootek.com,16020,1469690065288_ChoreService_2"
> daemon prio=10 tid=0x7f205d0d4000 nid=0x72af waiting on condition
> [0x7f203b151000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00066fd70dd8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.lo
[jira] [Updated] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak updated HBASE-12721:
Attachment: HBASE-12721_v5.patch
Ooh, sorry about that [~busbey]. Lemme post it up here to get a Yetus run.
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
> Components: build, community, documentation, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
> Attachments: HBASE-12721_v5.patch
>
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12721) Create Docker container cluster infrastructure to enable better testing
[
https://issues.apache.org/jira/browse/HBASE-12721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dima Spivak updated HBASE-12721:
Status: Patch Available (was: In Progress)
> Create Docker container cluster infrastructure to enable better testing
> ---
>
> Key: HBASE-12721
> URL: https://issues.apache.org/jira/browse/HBASE-12721
> Project: HBase
> Issue Type: New Feature
> Components: build, community, documentation, test
>Reporter: Dima Spivak
>Assignee: Dima Spivak
> Attachments: HBASE-12721_v5.patch
>
>
> Some simple work on using HBase with Docker was committed into /dev-support
> as "hbase_docker;" all this did was stand up a standalone cluster from source
> and start a shell. Now seems like a good time to extend this to be useful for
> applications that could actual benefit the community, especially around
> testing. Some ideas:
> - Integration testing would be much more accessible if people could stand up
> distributed HBase clusters on a single host machine in a couple minutes and
> run our awesome hbase-it suite against it.
> - Binary compatibility testing of an HBase client is easiest when standing up
> an HBase cluster can be done once and then different client source/binary
> permutations run against it.
> - Upgrade testing, and especially rolling upgrade testing, doesn't have any
> upstream automation on build.apache.org, in part because it's a pain to set
> up x-node clusters on Apache infrastructure.
> This proposal, whether it stays under /dev-support or moves out into it's own
> top-level module ("hbase-docker" would conveniently fit the existing schema
> :-)), strives to create a simple framework for deploying "distributed,"
> multi-container Apache HBase clusters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14450) HBase Backup/Restore Phase 3: Multiwal support
[
https://issues.apache.org/jira/browse/HBASE-14450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14450:
---
Attachment: testIncrementalBackup-8-15.txt
Test output corresponding to calling LogRoller#requestRollAll().
{code}
hTable = (HTable) conn.getTable(table1_restore);
Assert.assertThat(TEST_UTIL.countRows(hTable),
CoreMatchers.equalTo(NB_ROWS_IN_BATCH * 2));
{code}
The above assertion after incremental restore didn't pass.
> HBase Backup/Restore Phase 3: Multiwal support
> --
>
> Key: HBASE-14450
> URL: https://issues.apache.org/jira/browse/HBASE-14450
> Project: HBase
> Issue Type: Task
>Affects Versions: 2.0.0
>Reporter: Vladimir Rodionov
>Assignee: Ted Yu
> Labels: backup
> Fix For: 2.0.0
>
> Attachments: 14450.roll-wait.txt, 14450.v2.txt,
> org.apache.hadoop.hbase.backup.TestIncrementalBackup-output.txt,
> testIncrementalBackup-8-15.txt
>
>
> We need to support multiwal configurations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13094) Consider Filters that are evaluated before deletes and see delete markers
[ https://issues.apache.org/jira/browse/HBASE-13094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423087#comment-15423087 ] Lars Hofhansl commented on HBASE-13094: --- I'd like to warm this up again. There are several ways to do this: # Add a special Filter that can wrap any other filter. This filter would be evaluated before the delete markers are checked. This needs to be the only Filter used. # Variation of #1, which would allow to add this wrapper to a FilterList, then some logic to evaluate the right parts before or after delete markers are handled. # Add a new API to filter, in order to indicate whether this filter is to be evaluated before or after delete markers. # Add a new API to Scan and Get to allow adding a 2nd filter, that is evaluated before deletes. #1 is simplest and can be done 100% backwards compatible. #4 seems cleanest. Comments? [~stack], [~apurtell] > Consider Filters that are evaluated before deletes and see delete markers > - > > Key: HBASE-13094 > URL: https://issues.apache.org/jira/browse/HBASE-13094 > Project: HBase > Issue Type: Brainstorming > Components: regionserver, Scanners >Reporter: Lars Hofhansl > > That would be good for full control filtering of all cells, such as needed > for some transaction implementations. > [~ghelmling] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-16399) Provide an API to get list of failed regions and servername in Canary
[
https://issues.apache.org/jira/browse/HBASE-16399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422982#comment-15422982
]
Hadoop QA commented on HBASE-16399:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 52s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 40m
46s {color} | {color:green} 0.98 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s
{color} | {color:green} 0.98 passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} 0.98 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
41s {color} | {color:green} 0.98 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s {color} | {color:green} 0.98 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 16s
{color} | {color:red} hbase-server in 0.98 has 85 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 45s
{color} | {color:red} hbase-server in 0.98 failed with JDK v1.8.0_101. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} 0.98 passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed with JDK v1.8.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 42s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 39s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s
{color} | {color:red} The patch has 16 line(s) that end in whitespace. Use git
apply --whitespace=fix. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
13m 56s {color} | {color:green} The patch does not cause any errors with Hadoop
2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. {color} |
| {color:red}-1{color} | {color:red} hbaseprotoc {color} | {color:red} 0m 19s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 34s
{color} | {color:red} hbase-server generated 3 new + 85 unchanged - 0 fixed =
88 total (was 85) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 34s
{color} | {color:red} hbase-server in the patch failed with JDK v1.8.0_101.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed with JDK v1.7.0_101 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 165m 25s
{color} | {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 237m 34s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Read of unwritten field serverName in
org.apache.hadoop.hbase.tool.Canary$RegionTask.read() At Canary.java:in
org.apache.hadoop.hbase.tool.Canary$RegionTask.read() At Canary.java:[line
393] |
| | Read of unwritten field serverName in
org.apache.hadoop.hbase.tool.Canary$RegionTask.write() At Canary.java:in
org.apache.hadoop.hbase.tool.Canary$RegionTask.write() At Canary.java:[line
450] |
| | Unwritten field:Canary.java:[line 393] |
| Failed junit tests | hadoop.hbase.security.access.TestAccessController |
\\
\\
