[jira] [Created] (HBASE-22710) Wrong result in one case of scan that use raw and versions and filter together
Zheng Wang created HBASE-22710: -- Summary: Wrong result in one case of scan that use raw and versions and filter together Key: HBASE-22710 URL: https://issues.apache.org/jira/browse/HBASE-22710 Project: HBase Issue Type: Bug Components: Scanners Affects Versions: 2.0.0 Reporter: Zheng Wang -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] pankaj72981 opened a new pull request #391: HBASE-22628 Document the custom WAL directory (hbase.wal.dir) usage
pankaj72981 opened a new pull request #391: HBASE-22628 Document the custom WAL directory (hbase.wal.dir) usage URL: https://github.com/apache/hbase/pull/391 @anoopsjohn @zyork please review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22527: --- Attachment: HBASE-22527.branch-2.1.002.patch > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Sub-task > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-22527.branch-2.1.001.patch, > HBASE-22527.branch-2.1.002.patch, HBASE-22527.master.001.patch, > HBASE-22527.master.002.patch, HBASE-22527.master.003.patch, > HBASE-22527.master.004.patch, HBASE-22527.master.addendum.patch > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (HBASE-22639) Unexpected split when a big table has only one region on a regionServer
[ https://issues.apache.org/jira/browse/HBASE-22639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887678#comment-16887678 ] Zheng Wang edited comment on HBASE-22639 at 7/18/19 6:37 AM: - To Toshihiro Suzuk: Yes,it is similiar. To Xu Cang: The table has enough regions,but that regions is not big enough,so I don't want it to split. It may cause too many regions,and some of these too small. was (Author: filtertip): [~xucang] The table has enough regions,but that regions is not big enough,so I don't want it to split. It may cause too many regions,and some of these too small. > Unexpected split when a big table has only one region on a regionServer > > > Key: HBASE-22639 > URL: https://issues.apache.org/jira/browse/HBASE-22639 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Zheng Wang >Priority: Minor > > I am using the default policy named SteppingSplitPolicy. > If restart some nodes,it may occur,because this policy didnot judge if the > table is big enough actually. > It brings some unexpected small regions. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22537) Split happened Replica region can not be deleted after deleting table successfully and restarting RegionServer
[ https://issues.apache.org/jira/browse/HBASE-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887680#comment-16887680 ] Y. SREENIVASULU REDDY commented on HBASE-22537: --- [~wchevreuil], Thanks for your suggestion. Here I am providing are my details for Author info Name: sreenivasulureddy email: sreenivasul...@huawei.com And I have attached the master patch after fixing the checkstyle issues. > Split happened Replica region can not be deleted after deleting table > successfully and restarting RegionServer > -- > > Key: HBASE-22537 > URL: https://issues.apache.org/jira/browse/HBASE-22537 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.1.1 >Reporter: Y. SREENIVASULU REDDY >Assignee: Y. SREENIVASULU REDDY >Priority: Minor > Fix For: 2.1.6 > > Attachments: HBASE-22537.002.patch, HBASE-22537.branch-2.1.002.patch, > HBASE-22537.branch-2.1.003.patch, HBASE-22537.branch-2.1.004.patch, > HBASE-22537.branch-2.1.005.patch, HBASE-22537.branch-2.1.patch, > HBASE-22537.branch-2.patch, HBASE-22537.patch > > > [Test step] > 1.create a table (set RegionReplication=2). > 2.insert data to the table utill region be splitted. > 3.Disable and Drop the table. > 4.Parent replica region holding Regionserver, Kill forcefully > 5.HBase WebUI will show that the replica regions will be in RIT. > [Expect Output] > Parent replica region should be deleted. > [Actual Output] > Parent replica region still exists. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[
https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887679#comment-16887679
]
HBase QA commented on HBASE-22527:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
39s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
21s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
23s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
22s{color} | {color:red} hbase-server: The patch generated 40 new + 26
unchanged - 3 fixed = 66 total (was 29) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
19s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 0s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.7 2.8.5 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}203m 5s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}255m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=18.09.7 Server=18.09.7 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/648/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22527 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12975113/HBASE-22527.branch-2.1.001.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux a50efae5e1ed 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4
22:49:08 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.1 / c5af92b644 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/648/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/648/artifact/patch
[jira] [Commented] (HBASE-22639) Unexpected split when a big table has only one region on a regionServer
[ https://issues.apache.org/jira/browse/HBASE-22639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887678#comment-16887678 ] Zheng Wang commented on HBASE-22639: [~xucang] The table has enough regions,but that regions is not big enough,so I don't want it to split. It may cause too many regions,and some of these too small. > Unexpected split when a big table has only one region on a regionServer > > > Key: HBASE-22639 > URL: https://issues.apache.org/jira/browse/HBASE-22639 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 2.0.0 >Reporter: Zheng Wang >Priority: Minor > > I am using the default policy named SteppingSplitPolicy. > If restart some nodes,it may occur,because this policy didnot judge if the > table is big enough actually. > It brings some unexpected small regions. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22537) Split happened Replica region can not be deleted after deleting table successfully and restarting RegionServer
[ https://issues.apache.org/jira/browse/HBASE-22537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Y. SREENIVASULU REDDY updated HBASE-22537: -- Attachment: HBASE-22537.002.patch > Split happened Replica region can not be deleted after deleting table > successfully and restarting RegionServer > -- > > Key: HBASE-22537 > URL: https://issues.apache.org/jira/browse/HBASE-22537 > Project: HBase > Issue Type: Bug > Components: Region Assignment >Affects Versions: 2.1.1 >Reporter: Y. SREENIVASULU REDDY >Assignee: Y. SREENIVASULU REDDY >Priority: Minor > Fix For: 2.1.6 > > Attachments: HBASE-22537.002.patch, HBASE-22537.branch-2.1.002.patch, > HBASE-22537.branch-2.1.003.patch, HBASE-22537.branch-2.1.004.patch, > HBASE-22537.branch-2.1.005.patch, HBASE-22537.branch-2.1.patch, > HBASE-22537.branch-2.patch, HBASE-22537.patch > > > [Test step] > 1.create a table (set RegionReplication=2). > 2.insert data to the table utill region be splitted. > 3.Disable and Drop the table. > 4.Parent replica region holding Regionserver, Kill forcefully > 5.HBase WebUI will show that the replica regions will be in RIT. > [Expect Output] > Parent replica region should be deleted. > [Actual Output] > Parent replica region still exists. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22704) Avoid NPE when access table.jsp and snapshot.jsp but master not finish initialization
[
https://issues.apache.org/jira/browse/HBASE-22704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887664#comment-16887664
]
Hudson commented on HBASE-22704:
Results for branch branch-2.0
[build #1768 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1768/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1768//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1768//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1768//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Avoid NPE when access table.jsp and snapshot.jsp but master not finish
> initialization
> -
>
> Key: HBASE-22704
> URL: https://issues.apache.org/jira/browse/HBASE-22704
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
>
> {code:java}
> 2019-07-16,11:11:56,895 WARN [qtp1930319715-581]
> org.eclipse.jetty.servlet.ServletHandler: /table.jsp
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:154)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:840)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1780)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at
> org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1428)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:50)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:50)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:539)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPo
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887665#comment-16887665
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #16 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/16/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/16//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/16//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/16//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22527: --- Attachment: HBASE-22527.master.addendum.patch > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Sub-task > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-22527.branch-2.1.001.patch, > HBASE-22527.master.001.patch, HBASE-22527.master.002.patch, > HBASE-22527.master.003.patch, HBASE-22527.master.004.patch, > HBASE-22527.master.addendum.patch > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[
https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887649#comment-16887649
]
Guanghao Zhang commented on HBASE-22527:
{quote}Commit w/o test to branch-2.1 and 2.0.
{quote}
I attatched a new patch for branch-2.1 which split the TestAssignmentManager to
a base class and test class.
> [hbck2] Add a master web ui to show the problematic regions
> ---
>
> Key: HBASE-22527
> URL: https://issues.apache.org/jira/browse/HBASE-22527
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-operator-tools, hbck2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-22527.branch-2.1.001.patch,
> HBASE-22527.master.001.patch, HBASE-22527.master.002.patch,
> HBASE-22527.master.003.patch, HBASE-22527.master.004.patch
>
>
> On our cluster which based 2.2.0, we found one problem: there are some opened
> regions which had wrong regionserver in meta. The regionserver is not exist.
> We used hbck2 to fix them by the following steps.
> # disable table
> # bypass the stucked close region procedure (as the target regionserver is
> not exist) and disable table procedure.
> # setRegionState to CLOSED.
> # setTableState to DISABLED.
> # enable table
> We found this problem by scan the hbase:meta. I thought we should add this
> feature to hbck2. The we can use hbck2 to find this problem. Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-21393) Add an API ScheduleSCP() to HBCK2
[ https://issues.apache.org/jira/browse/HBASE-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887638#comment-16887638 ] stack commented on HBASE-21393: --- This change is coming in at around the same time (HBASE-22673). It deprecates the API we are using here. Will need to change what we use down the road. WIll file follow-on. > Add an API ScheduleSCP() to HBCK2 > -- > > Key: HBASE-21393 > URL: https://issues.apache.org/jira/browse/HBASE-21393 > Project: HBase > Issue Type: Sub-task > Components: hbase-operator-tools, hbck2 >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Attachments: 0001-HBASE-21393-Add-an-API-ScheduleSCP-to-HBCK2.patch, > 0002-HBASE-21393-Add-an-API-ScheduleSCP-to-HBCK2.patch > > > Add the API of ScheduleSCP() to hbase-operator-tools so that operators can > schedule ServerCrashProcedures of specified regionservers. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase-operator-tools] saintstack commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
saintstack commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#issuecomment-512671609 @infraio Thanks. Sorry for confusion over in HBASE-22673. Lets go w/ this as is for now. Can switch to new API later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887636#comment-16887636 ] stack commented on HBASE-22673: --- Ooops. I missed the PR. +1'd it. Ignore the above [~zghaobac] hbck2 will just use the old API. Can later switch it to use the new one. > Avoid to expose protobuf stuff in Hbck interface > > > Key: HBASE-22673 > URL: https://issues.apache.org/jira/browse/HBASE-22673 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887624#comment-16887624 ] stack commented on HBASE-22673: --- But lets play it safe. Let me add a method and deprecate the old [~zghaobac] > Avoid to expose protobuf stuff in Hbck interface > > > Key: HBASE-22673 > URL: https://issues.apache.org/jira/browse/HBASE-22673 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887621#comment-16887621 ] stack commented on HBASE-22673: --- bq. You mean only HBCK2 tool will use this api? Yeah, thought this API only for HBCK2. Correct me if wrong. If HBCK2 client not in place, then yes, no one can be using the API so lets just change it? > Avoid to expose protobuf stuff in Hbck interface > > > Key: HBASE-22673 > URL: https://issues.apache.org/jira/browse/HBASE-22673 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22704) Avoid NPE when access table.jsp and snapshot.jsp but master not finish initialization
[
https://issues.apache.org/jira/browse/HBASE-22704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887618#comment-16887618
]
Hudson commented on HBASE-22704:
Results for branch master
[build #1238 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1238/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Avoid NPE when access table.jsp and snapshot.jsp but master not finish
> initialization
> -
>
> Key: HBASE-22704
> URL: https://issues.apache.org/jira/browse/HBASE-22704
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
>
> {code:java}
> 2019-07-16,11:11:56,895 WARN [qtp1930319715-581]
> org.eclipse.jetty.servlet.ServletHandler: /table.jsp
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:154)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:840)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1780)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at
> org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1428)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:50)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:50)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:539)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
> at
> org.eclipse.jet
[jira] [Commented] (HBASE-21426) TestEncryptionKeyRotation.testCFKeyRotation is flaky
[
https://issues.apache.org/jira/browse/HBASE-21426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887620#comment-16887620
]
Hudson commented on HBASE-21426:
Results for branch master
[build #1238 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1238/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestEncryptionKeyRotation.testCFKeyRotation is flaky
>
>
> Key: HBASE-21426
> URL: https://issues.apache.org/jira/browse/HBASE-21426
> Project: HBase
> Issue Type: Bug
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Attachments: testCFKeyRotation [Jenkins test result].pdf
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22685) Add to migration doc that meta should be healthy before upgrade
[
https://issues.apache.org/jira/browse/HBASE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887617#comment-16887617
]
Hudson commented on HBASE-22685:
Results for branch master
[build #1238 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1238/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add to migration doc that meta should be healthy before upgrade
> ---
>
> Key: HBASE-22685
> URL: https://issues.apache.org/jira/browse/HBASE-22685
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
>Reporter: stack
>Assignee: stack
>Priority: Major
> Labels: documentation
> Fix For: 3.0.0
>
> Attachments: HBASE-22685.master.001.patch
>
>
> Add a note to doc that operator ensures healthy meta before upgrade.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[
https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887619#comment-16887619
]
Hudson commented on HBASE-22527:
Results for branch master
[build #1238 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1238/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1238//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [hbck2] Add a master web ui to show the problematic regions
> ---
>
> Key: HBASE-22527
> URL: https://issues.apache.org/jira/browse/HBASE-22527
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-operator-tools, hbck2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-22527.branch-2.1.001.patch,
> HBASE-22527.master.001.patch, HBASE-22527.master.002.patch,
> HBASE-22527.master.003.patch, HBASE-22527.master.004.patch
>
>
> On our cluster which based 2.2.0, we found one problem: there are some opened
> regions which had wrong regionserver in meta. The regionserver is not exist.
> We used hbck2 to fix them by the following steps.
> # disable table
> # bypass the stucked close region procedure (as the target regionserver is
> not exist) and disable table procedure.
> # setRegionState to CLOSED.
> # setTableState to DISABLED.
> # enable table
> We found this problem by scan the hbase:meta. I thought we should add this
> feature to hbck2. The we can use hbck2 to find this problem. Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[
https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887609#comment-16887609
]
Guanghao Zhang commented on HBASE-22673:
{quote}Regards server-side, I meant the api w/ pb in it was added only on the
server-side. There was no shipping client that was capable of querying it, not
until HBASE-21322 goes in... So I'm thinking we can just change it.
{quote}
You mean only HBCK2 tool will use this api? And now HBASE-21393 is not
committed to HBCK2 tool? So just change this protobuf method directly?
> Avoid to expose protobuf stuff in Hbck interface
>
>
> Key: HBASE-22673
> URL: https://issues.apache.org/jira/browse/HBASE-22673
> Project: HBase
> Issue Type: Sub-task
> Components: hbck2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887605#comment-16887605 ] stack commented on HBASE-22673: --- [~zghaobac] I'm working on a patch. WIll put it up in a sec. Regards server-side, I meant the api w/ pb in it was added only on the server-side. There was no shipping client that was capable of querying it, not until HBASE-21322 goes in... So I'm thinking we can just change it. Means that HBASE-21322 won't be able to go in until shipping version that has this item in it. > Avoid to expose protobuf stuff in Hbck interface > > > Key: HBASE-22673 > URL: https://issues.apache.org/jira/browse/HBASE-22673 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[
https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887600#comment-16887600
]
Guanghao Zhang commented on HBASE-22673:
{quote}Client-side hasn't gone in yet (HBASE-21393). Fix server-side first
[~zghaobac]? Makes sense to me.
{quote}
I mean that we should avoid expose protobuf stuff in Hbck interface.
server-side means?
> Avoid to expose protobuf stuff in Hbck interface
>
>
> Key: HBASE-22673
> URL: https://issues.apache.org/jira/browse/HBASE-22673
> Project: HBase
> Issue Type: Sub-task
> Components: hbck2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22704) Avoid NPE when access table.jsp and snapshot.jsp but master not finish initialization
[
https://issues.apache.org/jira/browse/HBASE-22704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887597#comment-16887597
]
Hudson commented on HBASE-22704:
Results for branch branch-2
[build #2084 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2084/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2084//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2084//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2084//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Avoid NPE when access table.jsp and snapshot.jsp but master not finish
> initialization
> -
>
> Key: HBASE-22704
> URL: https://issues.apache.org/jira/browse/HBASE-22704
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
>
> {code:java}
> 2019-07-16,11:11:56,895 WARN [qtp1930319715-581]
> org.eclipse.jetty.servlet.ServletHandler: /table.jsp
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:154)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:840)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1780)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at
> org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1428)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:50)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:50)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:539)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
> at
> org.ecl
[jira] [Commented] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[
https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887598#comment-16887598
]
Hudson commented on HBASE-22527:
Results for branch branch-2
[build #2084 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2084/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2084//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2084//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2084//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [hbck2] Add a master web ui to show the problematic regions
> ---
>
> Key: HBASE-22527
> URL: https://issues.apache.org/jira/browse/HBASE-22527
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-operator-tools, hbck2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
> Attachments: HBASE-22527.branch-2.1.001.patch,
> HBASE-22527.master.001.patch, HBASE-22527.master.002.patch,
> HBASE-22527.master.003.patch, HBASE-22527.master.004.patch
>
>
> On our cluster which based 2.2.0, we found one problem: there are some opened
> regions which had wrong regionserver in meta. The regionserver is not exist.
> We used hbck2 to fix them by the following steps.
> # disable table
> # bypass the stucked close region procedure (as the target regionserver is
> not exist) and disable table procedure.
> # setRegionState to CLOSED.
> # setTableState to DISABLED.
> # enable table
> We found this problem by scan the hbase:meta. I thought we should add this
> feature to hbck2. The we can use hbck2 to find this problem. Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[
https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887594#comment-16887594
]
Guanghao Zhang commented on HBASE-22673:
{quote}It went into 3.0.0, 2.2.0, 2.0.3, 2.1.2 but client was only added
yesterday.
{quote}
It was added to Hbck interface. This is at client side and marked as
LimitPrivate.
> Avoid to expose protobuf stuff in Hbck interface
>
>
> Key: HBASE-22673
> URL: https://issues.apache.org/jira/browse/HBASE-22673
> Project: HBase
> Issue Type: Sub-task
> Components: hbck2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[
https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887593#comment-16887593
]
Guanghao Zhang commented on HBASE-22673:
{quote}Only in unreleased versions so far... 3.0.0, 2.3.0. No client yet. Safe
to remove!
{quote}
Let me remove it.
> Avoid to expose protobuf stuff in Hbck interface
>
>
> Key: HBASE-22673
> URL: https://issues.apache.org/jira/browse/HBASE-22673
> Project: HBase
> Issue Type: Sub-task
> Components: hbck2
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] infraio commented on a change in pull request #373: HBASE-22527 [hbck2] Add a master web ui to show the problematic regions
infraio commented on a change in pull request #373: HBASE-22527 [hbck2] Add a master web ui to show the problematic regions URL: https://github.com/apache/hbase/pull/373#discussion_r304727389 ## File path: hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/AssignmentManagerStatusTmpl.jamon ## @@ -17,27 +17,98 @@ See the License for the specific language governing permissions and limitations under the License. <%import> -org.apache.hadoop.hbase.master.assignment.AssignmentManager; -org.apache.hadoop.hbase.master.assignment.AssignmentManager.RegionInTransitionStat; -org.apache.hadoop.hbase.master.assignment.RegionStates.RegionFailedOpen; -org.apache.hadoop.hbase.master.RegionState; +java.util.Map; +java.util.Set; +java.util.SortedSet; +java.util.concurrent.atomic.AtomicInteger; +java.util.stream.Collectors; org.apache.hadoop.conf.Configuration; org.apache.hadoop.hbase.HBaseConfiguration; org.apache.hadoop.hbase.HConstants; +org.apache.hadoop.hbase.ServerName; +org.apache.hadoop.hbase.client.RegionInfo; org.apache.hadoop.hbase.client.RegionInfoDisplay; -java.util.HashSet; -java.util.SortedSet; -java.util.Map; -java.util.concurrent.atomic.AtomicInteger; +org.apache.hadoop.hbase.master.RegionState; +org.apache.hadoop.hbase.master.assignment.AssignmentManager; +org.apache.hadoop.hbase.master.assignment.AssignmentManager.RegionInTransitionStat; +org.apache.hadoop.hbase.master.assignment.RegionStates.RegionFailedOpen; +org.apache.hadoop.hbase.util.Pair; <%args> AssignmentManager assignmentManager; int limit = 100; -<%java SortedSet rit = assignmentManager - .getRegionStates().getRegionsInTransitionOrderedByTimestamp(); -%> +<%java> +SortedSet rit = assignmentManager.getRegionStates() +.getRegionsInTransitionOrderedByTimestamp(); +Map>> problematicRegions = assignmentManager +.getProblematicRegions(); + + +<%if !problematicRegions.isEmpty() %> +<%java> +int totalSize = problematicRegions.size(); +int sizePerPage = Math.min(10, totalSize); +int numOfPages = (int) Math.ceil(totalSize * 1.0 / sizePerPage); + + +Problematic Regions Review comment: Already committed it. Will add addendum for this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] saintstack commented on a change in pull request #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
saintstack commented on a change in pull request #11: HBASE-21322 Add a
scheduleServerCrashProcedure() API to HbckService
URL:
https://github.com/apache/hbase-operator-tools/pull/11#discussion_r304721905
##
File path: hbase-hbck2/src/main/java/org/apache/hbase/Version.java
##
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import org.apache.commons.lang3.StringUtils;
+
+/**
+ * Check versions.
+ */
+public final class Version {
+ // Copied from hbase VersionInfo.
+ private static final int VERY_LARGE_NUMBER = 10;
+ private static final int MAJOR = 0;
+ private static final int MINOR = 1;
+ private static final int PATCH = 2;
+
+ private Version() {}
+
+ /**
+ * @param thresholdVersions List of versions from oldest to newest.
+ * @return true if version is less-than or equal to
thresholdVersions.
+ * For example, if passed threshold list is {"2.0.2", "2.1.3",
"2.2.1"}
+ * and the version is 2.1.2 then the result should be false since 2.1.2
+ * is less than the matching passed-in 2.1.3 but if version is 2.1.5 then
we return true.
+ */
+ static boolean check(final String version, String ... thresholdVersions) {
+if (thresholdVersions == null) {
+ return true;
+}
+boolean supported = false;
+// Components of the server version string.
+String [] versionComponents = getVersionComponents(version);
+boolean excessiveMajor = false;
+boolean excessiveMinor = false;
+for (String thresholdVersion: thresholdVersions) {
+ // Get components of current threshold version.
+ String[] thresholdVersionComponents =
getVersionComponents(thresholdVersion);
+ int serverMajor = Integer.parseInt(versionComponents[MAJOR]);
+ int thresholdMajor = Integer.parseInt(thresholdVersionComponents[MAJOR]);
+ if (serverMajor > thresholdMajor) {
+excessiveMajor = true;
+continue;
+ }
+ excessiveMajor = false;
+ if (serverMajor < thresholdMajor) {
+continue;
+ }
+ int serverMinor = Integer.parseInt(versionComponents[MINOR]);
+ int thresholdMinor = Integer.parseInt(thresholdVersionComponents[MINOR]);
+ if (serverMinor > thresholdMinor) {
+excessiveMinor = true;
+continue;
+ }
+ excessiveMinor = false;
+ if (serverMinor < thresholdMinor) {
+continue;
+ }
+ if (Integer.parseInt(versionComponents[PATCH]) >=
+ Integer.parseInt(thresholdVersionComponents[PATCH])) {
+supported = true;
+ }
+ break;
+}
+return supported || excessiveMajor || excessiveMinor;
+ }
+
+ /**
+ * Copied from hbase VersionInfo.
Review comment:
The getComponents in private in VersionInfo so have to copy it.
VI doesn't have the check we do here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Comment Edited] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887578#comment-16887578 ] stack edited comment on HBASE-22673 at 7/18/19 3:25 AM: Here is what added pbs... Author: Jingyun Tian Date: Mon Apr 8 17:26:29 2019 +0800 HBASE-21965 Fix failed split and merge transactions that have failed to roll back Its a list of TableNames... Can do list of Strings instead. Only in unreleased versions so far... 3.0.0, 2.3.0. No client yet. Safe to remove! For this one... commit 857abfb08221e3d313262e5812078ed4a686bdd9 Author: tianjingyun Date: Sat Oct 27 10:06:56 2018 +0800 HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService Signed-off-by: Michael Stack Can do a list of Strings too for the List of ServerNames It went into 3.0.0, 2.2.0, 2.0.3, 2.1.2 but client was only added yesterday. You working on it [~zghaobac] ? If not, I can have a go at it sir. was (Author: stack): Here is what added pbs... Author: Jingyun Tian Date: Mon Apr 8 17:26:29 2019 +0800 HBASE-21965 Fix failed split and merge transactions that have failed to roll back Its a list of TableNames... Can do list of Strings instead. Only in unreleased versions so far... 3.0.0, 2.3.0. No client yet. For this one... commit 857abfb08221e3d313262e5812078ed4a686bdd9 Author: tianjingyun Date: Sat Oct 27 10:06:56 2018 +0800 HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService Signed-off-by: Michael Stack Can do a list of Strings too for the List of ServerNames It went into 3.0.0, 2.2.0, 2.0.3, 2.1.2 but client was only added yesterday. You working on it [~zghaobac] ? If not, I can have a go at it sir. > Avoid to expose protobuf stuff in Hbck interface > > > Key: HBASE-22673 > URL: https://issues.apache.org/jira/browse/HBASE-22673 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887579#comment-16887579 ] stack commented on HBASE-22527: --- Commit w/o test to branch-2.1 and 2.0 [~zghaobac]? > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Sub-task > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-22527.branch-2.1.001.patch, > HBASE-22527.master.001.patch, HBASE-22527.master.002.patch, > HBASE-22527.master.003.patch, HBASE-22527.master.004.patch > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887578#comment-16887578 ] stack commented on HBASE-22673: --- Here is what added pbs... Author: Jingyun Tian Date: Mon Apr 8 17:26:29 2019 +0800 HBASE-21965 Fix failed split and merge transactions that have failed to roll back Its a list of TableNames... Can do list of Strings instead. Only in unreleased versions so far... 3.0.0, 2.3.0. No client yet. For this one... commit 857abfb08221e3d313262e5812078ed4a686bdd9 Author: tianjingyun Date: Sat Oct 27 10:06:56 2018 +0800 HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService Signed-off-by: Michael Stack Can do a list of Strings too for the List of ServerNames It went into 3.0.0, 2.2.0, 2.0.3, 2.1.2 but client was only added yesterday. You working on it [~zghaobac] ? If not, I can have a go at it sir. > Avoid to expose protobuf stuff in Hbck interface > > > Key: HBASE-22673 > URL: https://issues.apache.org/jira/browse/HBASE-22673 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887576#comment-16887576 ] stack commented on HBASE-22673: --- bq. The "checking" work can be done ... master. We can add a chore thread in master to check the wrong thing. And show them in master web ui. +1 We've committed an hbck that uses pbs. I suppose only the server-side was added by HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService ... Client-side hasn't gone in yet (HBASE-21393). Fix server-side first [~zghaobac]? Makes sense to me. > Avoid to expose protobuf stuff in Hbck interface > > > Key: HBASE-22673 > URL: https://issues.apache.org/jira/browse/HBASE-22673 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase-operator-tools] infraio commented on a change in pull request #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
infraio commented on a change in pull request #11: HBASE-21322 Add a
scheduleServerCrashProcedure() API to HbckService
URL:
https://github.com/apache/hbase-operator-tools/pull/11#discussion_r304715501
##
File path: hbase-hbck2/src/main/java/org/apache/hbase/Version.java
##
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hbase;
+
+import org.apache.commons.lang3.StringUtils;
+
+/**
+ * Check versions.
+ */
+public final class Version {
+ // Copied from hbase VersionInfo.
+ private static final int VERY_LARGE_NUMBER = 10;
+ private static final int MAJOR = 0;
+ private static final int MINOR = 1;
+ private static final int PATCH = 2;
+
+ private Version() {}
+
+ /**
+ * @param thresholdVersions List of versions from oldest to newest.
+ * @return true if version is less-than or equal to
thresholdVersions.
+ * For example, if passed threshold list is {"2.0.2", "2.1.3",
"2.2.1"}
+ * and the version is 2.1.2 then the result should be false since 2.1.2
+ * is less than the matching passed-in 2.1.3 but if version is 2.1.5 then
we return true.
+ */
+ static boolean check(final String version, String ... thresholdVersions) {
+if (thresholdVersions == null) {
+ return true;
+}
+boolean supported = false;
+// Components of the server version string.
+String [] versionComponents = getVersionComponents(version);
+boolean excessiveMajor = false;
+boolean excessiveMinor = false;
+for (String thresholdVersion: thresholdVersions) {
+ // Get components of current threshold version.
+ String[] thresholdVersionComponents =
getVersionComponents(thresholdVersion);
+ int serverMajor = Integer.parseInt(versionComponents[MAJOR]);
+ int thresholdMajor = Integer.parseInt(thresholdVersionComponents[MAJOR]);
+ if (serverMajor > thresholdMajor) {
+excessiveMajor = true;
+continue;
+ }
+ excessiveMajor = false;
+ if (serverMajor < thresholdMajor) {
+continue;
+ }
+ int serverMinor = Integer.parseInt(versionComponents[MINOR]);
+ int thresholdMinor = Integer.parseInt(thresholdVersionComponents[MINOR]);
+ if (serverMinor > thresholdMinor) {
+excessiveMinor = true;
+continue;
+ }
+ excessiveMinor = false;
+ if (serverMinor < thresholdMinor) {
+continue;
+ }
+ if (Integer.parseInt(versionComponents[PATCH]) >=
+ Integer.parseInt(thresholdVersionComponents[PATCH])) {
+supported = true;
+ }
+ break;
+}
+return supported || excessiveMajor || excessiveMinor;
+ }
+
+ /**
+ * Copied from hbase VersionInfo.
Review comment:
Why don't use hbase VersionInfo?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Assigned] (HBASE-22709) Add a web ui to show the failed splited/merged regions
[ https://issues.apache.org/jira/browse/HBASE-22709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang reassigned HBASE-22709: -- Assignee: Guanghao Zhang > Add a web ui to show the failed splited/merged regions > -- > > Key: HBASE-22709 > URL: https://issues.apache.org/jira/browse/HBASE-22709 > Project: HBase > Issue Type: Sub-task >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > See HBASE-21965. There may have two type of failed splited/merged regions. > One is orphan region on filesystem. Another one is unassigned daughter > regions or merged regions. Plan to add a seperate hbck.jsp for them. Will add > more hbck problems to this UI in future. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HBASE-22709) Add a web ui to show the failed splited/merged regions
Guanghao Zhang created HBASE-22709: -- Summary: Add a web ui to show the failed splited/merged regions Key: HBASE-22709 URL: https://issues.apache.org/jira/browse/HBASE-22709 Project: HBase Issue Type: Sub-task Reporter: Guanghao Zhang See HBASE-21965. There may have two type of failed splited/merged regions. One is orphan region on filesystem. Another one is unassigned daughter regions or merged regions. Plan to add a seperate hbck.jsp for them. Will add more hbck problems to this UI in future. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] infraio commented on a change in pull request #372: HBASE-22673 Avoid to expose protobuf stuff in Hbck interface
infraio commented on a change in pull request #372: HBASE-22673 Avoid to expose
protobuf stuff in Hbck interface
URL: https://github.com/apache/hbase/pull/372#discussion_r304712180
##
File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/Hbck.java
##
@@ -105,8 +109,18 @@
List bypassProcedure(List pids, long waitTime, boolean
override, boolean recursive)
throws IOException;
- List scheduleServerCrashProcedure(List
serverNames)
- throws IOException;
+ /**
+ * Use {@link #scheduleServerCrashProcedures(List)} instead.
+ * @deprecated since 2.2.1. Will removed in 2.3.0.
Review comment:
Open a new issue HBASE-22708 for it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-22708) Remove the deprecated methods in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22708: --- Description: Only for 3.0.0. Remove the methods which mark deprecated in HBASE-22673. > Remove the deprecated methods in Hbck interface > --- > > Key: HBASE-22708 > URL: https://issues.apache.org/jira/browse/HBASE-22708 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > > Only for 3.0.0. Remove the methods which mark deprecated in HBASE-22673. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HBASE-22708) Remove the deprecated methods in Hbck interface
Guanghao Zhang created HBASE-22708: -- Summary: Remove the deprecated methods in Hbck interface Key: HBASE-22708 URL: https://issues.apache.org/jira/browse/HBASE-22708 Project: HBase Issue Type: Sub-task Affects Versions: 3.0.0 Reporter: Guanghao Zhang Assignee: Guanghao Zhang -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22673) Avoid to expose protobuf stuff in Hbck interface
[ https://issues.apache.org/jira/browse/HBASE-22673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22673: --- Issue Type: Sub-task (was: Improvement) Parent: HBASE-21745 > Avoid to expose protobuf stuff in Hbck interface > > > Key: HBASE-22673 > URL: https://issues.apache.org/jira/browse/HBASE-22673 > Project: HBase > Issue Type: Sub-task > Components: hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Guanghao Zhang updated HBASE-22527: --- Attachment: HBASE-22527.branch-2.1.001.patch > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Sub-task > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-22527.branch-2.1.001.patch, > HBASE-22527.master.001.patch, HBASE-22527.master.002.patch, > HBASE-22527.master.003.patch, HBASE-22527.master.004.patch > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] infraio closed pull request #373: HBASE-22527 [hbck2] Add a master web ui to show the problematic regions
infraio closed pull request #373: HBASE-22527 [hbck2] Add a master web ui to show the problematic regions URL: https://github.com/apache/hbase/pull/373 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-22704) Avoid NPE when access table.jsp and snapshot.jsp but master not finish initialization
[
https://issues.apache.org/jira/browse/HBASE-22704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guanghao Zhang resolved HBASE-22704.
Resolution: Fixed
Fix Version/s: 2.1.6
2.2.1
2.0.6
2.3.0
3.0.0
Pushed to branch-2.0+.
> Avoid NPE when access table.jsp and snapshot.jsp but master not finish
> initialization
> -
>
> Key: HBASE-22704
> URL: https://issues.apache.org/jira/browse/HBASE-22704
> Project: HBase
> Issue Type: Improvement
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
>
> {code:java}
> 2019-07-16,11:11:56,895 WARN [qtp1930319715-581]
> org.eclipse.jetty.servlet.ServletHandler: /table.jsp
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:154)
> at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:111)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:840)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1780)
> at
> org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:112)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at
> org.apache.hadoop.hbase.http.ClickjackingPreventionFilter.doFilter(ClickjackingPreventionFilter.java:48)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at
> org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1428)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:50)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.apache.hadoop.hbase.http.NoCacheFilter.doFilter(NoCacheFilter.java:50)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1767)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:539)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
> at
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-21426) TestEncryptionKeyRotation.testCFKeyRotation is flaky
[ https://issues.apache.org/jira/browse/HBASE-21426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887523#comment-16887523 ] Guanghao Zhang commented on HBASE-21426: Pushed to master to see the result. > TestEncryptionKeyRotation.testCFKeyRotation is flaky > > > Key: HBASE-21426 > URL: https://issues.apache.org/jira/browse/HBASE-21426 > Project: HBase > Issue Type: Bug >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: testCFKeyRotation [Jenkins test result].pdf > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] infraio merged pull request #375: HBASE-21426 TestEncryptionKeyRotation.testCFKeyRotation is flaky
infraio merged pull request #375: HBASE-21426 TestEncryptionKeyRotation.testCFKeyRotation is flaky URL: https://github.com/apache/hbase/pull/375 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22527) [hbck2] Add a master web ui to show the problematic regions
[ https://issues.apache.org/jira/browse/HBASE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887520#comment-16887520 ] Guanghao Zhang commented on HBASE-22527: Pushed to branch-2.2+. For branch-2.1 and 2.0, need a new patch as there are no TestAssignmentManagerBase. > [hbck2] Add a master web ui to show the problematic regions > --- > > Key: HBASE-22527 > URL: https://issues.apache.org/jira/browse/HBASE-22527 > Project: HBase > Issue Type: Sub-task > Components: hbase-operator-tools, hbck2 >Reporter: Guanghao Zhang >Assignee: Guanghao Zhang >Priority: Major > Attachments: HBASE-22527.master.001.patch, > HBASE-22527.master.002.patch, HBASE-22527.master.003.patch, > HBASE-22527.master.004.patch > > > On our cluster which based 2.2.0, we found one problem: there are some opened > regions which had wrong regionserver in meta. The regionserver is not exist. > We used hbck2 to fix them by the following steps. > # disable table > # bypass the stucked close region procedure (as the target regionserver is > not exist) and disable table procedure. > # setRegionState to CLOSED. > # setTableState to DISABLED. > # enable table > We found this problem by scan the hbase:meta. I thought we should add this > feature to hbck2. The we can use hbck2 to find this problem. Thanks. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-21745) Make HBCK2 be able to fix issues other than region assignment
[ https://issues.apache.org/jira/browse/HBASE-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21745: -- Description: This is what [~apurtell] posted on mailing-list, HBCK2 should support * -Rebuild meta from region metadata in the filesystem, aka offline meta rebuild.- * Fix assignment errors (undeployed regions, double assignments (yes, should not be possible), etc) * Fix region holes, overlaps, and other errors in the region chain * Fix failed split and merge transactions that have failed to roll back due to some bug (related to previous) * -Enumerate store files to determine file level corruption and sideline corrupt files- * -Fix hfile link problems (dangling / broken)- was: This is what [~apurtell] posted on mailing-list, HBCK2 should support * -Rebuild meta from region metadata in the filesystem, aka offline meta rebuild.- * - Fix assignment errors (undeployed regions, double assignments (yes, should not be possible), etc) * - Fix region holes, overlaps, and other errors in the region chain * - Fix failed split and merge transactions that have failed to roll back due to some bug (related to previous) * - Enumerate store files to determine file level corruption and sideline corrupt files- * - Fix hfile link problems (dangling / broken)- > Make HBCK2 be able to fix issues other than region assignment > - > > Key: HBASE-21745 > URL: https://issues.apache.org/jira/browse/HBASE-21745 > Project: HBase > Issue Type: Umbrella > Components: hbase-operator-tools, hbck2 >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > > This is what [~apurtell] posted on mailing-list, HBCK2 should support > * -Rebuild meta from region metadata in the filesystem, aka offline meta > rebuild.- > * Fix assignment errors (undeployed regions, double assignments (yes, should > not be possible), etc) > * Fix region holes, overlaps, and other errors in the region chain > * Fix failed split and merge transactions that have failed to roll back due > to some bug (related to previous) > * -Enumerate store files to determine file level corruption and sideline > corrupt files- > * -Fix hfile link problems (dangling / broken)- -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-21745) Make HBCK2 be able to fix issues other than region assignment
[ https://issues.apache.org/jira/browse/HBASE-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21745: -- Description: This is what [~apurtell] posted on mailing-list, HBCK2 should support * -Rebuild meta from region metadata in the filesystem, aka offline meta rebuild.- * - Fix assignment errors (undeployed regions, double assignments (yes, should not be possible), etc) * - Fix region holes, overlaps, and other errors in the region chain * - Fix failed split and merge transactions that have failed to roll back due to some bug (related to previous) * - Enumerate store files to determine file level corruption and sideline corrupt files- * - Fix hfile link problems (dangling / broken)- was: This is what [~apurtell] posted on mailing-list, HBCK2 should support * - Rebuild meta from region metadata in the filesystem, aka offline meta rebuild.- * - Fix assignment errors (undeployed regions, double assignments (yes, should not be possible), etc) * - Fix region holes, overlaps, and other errors in the region chain * - Fix failed split and merge transactions that have failed to roll back due to some bug (related to previous) * - Enumerate store files to determine file level corruption and sideline corrupt files- * - Fix hfile link problems (dangling / broken)- > Make HBCK2 be able to fix issues other than region assignment > - > > Key: HBASE-21745 > URL: https://issues.apache.org/jira/browse/HBASE-21745 > Project: HBase > Issue Type: Umbrella > Components: hbase-operator-tools, hbck2 >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > > This is what [~apurtell] posted on mailing-list, HBCK2 should support > * -Rebuild meta from region metadata in the filesystem, aka offline meta > rebuild.- > * - Fix assignment errors (undeployed regions, double assignments (yes, > should not be possible), etc) > * - Fix region holes, overlaps, and other errors in the region chain > * - Fix failed split and merge transactions that have failed to roll back >due to some bug (related to previous) > * - Enumerate store files to determine file level corruption and sideline > corrupt files- > * - Fix hfile link problems (dangling / broken)- -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-21745) Make HBCK2 be able to fix issues other than region assignment
[
https://issues.apache.org/jira/browse/HBASE-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-21745:
--
Description:
This is what [~apurtell] posted on mailing-list, HBCK2 should support
* - Rebuild meta from region metadata in the filesystem, aka offline meta
rebuild.-
* - Fix assignment errors (undeployed regions, double assignments (yes, should
not be possible), etc)
* - Fix region holes, overlaps, and other errors in the region chain
* - Fix failed split and merge transactions that have failed to roll back
due to some bug (related to previous)
* - Enumerate store files to determine file level corruption and sideline
corrupt files-
* - Fix hfile link problems (dangling / broken)-
was:
This is what [~apurtell] posted on mailing-list, HBCK2 should support
{quote}
* - Rebuild meta from region metadata in the filesystem, aka offline meta
rebuild.-
* - Fix assignment errors (undeployed regions, double assignments (yes, should
not be possible), etc)
* - Fix region holes, overlaps, and other errors in the region chain
* - Fix failed split and merge transactions that have failed to roll back
due to some bug (related to previous)
* - Enumerate store files to determine file level corruption and sideline
corrupt files-
* - Fix hfile link problems (dangling / broken)-
> Make HBCK2 be able to fix issues other than region assignment
> -
>
> Key: HBASE-21745
> URL: https://issues.apache.org/jira/browse/HBASE-21745
> Project: HBase
> Issue Type: Umbrella
> Components: hbase-operator-tools, hbck2
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
>
> This is what [~apurtell] posted on mailing-list, HBCK2 should support
> * - Rebuild meta from region metadata in the filesystem, aka offline meta
> rebuild.-
> * - Fix assignment errors (undeployed regions, double assignments (yes,
> should not be possible), etc)
> * - Fix region holes, overlaps, and other errors in the region chain
> * - Fix failed split and merge transactions that have failed to roll back
>due to some bug (related to previous)
> * - Enumerate store files to determine file level corruption and sideline
> corrupt files-
> * - Fix hfile link problems (dangling / broken)-
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HBASE-21745) Make HBCK2 be able to fix issues other than region assignment
[
https://issues.apache.org/jira/browse/HBASE-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-21745:
--
Description:
This is what [~apurtell] posted on mailing-list, HBCK2 should support
{quote}
* - Rebuild meta from region metadata in the filesystem, aka offline meta
rebuild.-
* - Fix assignment errors (undeployed regions, double assignments (yes, should
not be possible), etc)
* - Fix region holes, overlaps, and other errors in the region chain
* - Fix failed split and merge transactions that have failed to roll back
due to some bug (related to previous)
* - Enumerate store files to determine file level corruption and sideline
corrupt files-
* - Fix hfile link problems (dangling / broken)-
was:
This is what [~apurtell] posted on mailing-list, HBCK2 should support
{quote}
- - Rebuild meta from region metadata in the filesystem, aka offline meta
rebuild.-
- Fix assignment errors (undeployed regions, double assignments (yes,
should not be possible), etc)
- Fix region holes, overlaps, and other errors in the region chain
- Fix failed split and merge transactions that have failed to roll back
due to some bug (related to previous)
- - Enumerate store files to determine file level corruption and sideline
corrupt files-
- - Fix hfile link problems (dangling / broken)
{quote}-
> Make HBCK2 be able to fix issues other than region assignment
> -
>
> Key: HBASE-21745
> URL: https://issues.apache.org/jira/browse/HBASE-21745
> Project: HBase
> Issue Type: Umbrella
> Components: hbase-operator-tools, hbck2
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
>
> This is what [~apurtell] posted on mailing-list, HBCK2 should support
> {quote}
> * - Rebuild meta from region metadata in the filesystem, aka offline meta
> rebuild.-
> * - Fix assignment errors (undeployed regions, double assignments (yes,
> should not be possible), etc)
> * - Fix region holes, overlaps, and other errors in the region chain
> * - Fix failed split and merge transactions that have failed to roll back
>due to some bug (related to previous)
> * - Enumerate store files to determine file level corruption and sideline
> corrupt files-
> * - Fix hfile link problems (dangling / broken)-
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HBASE-21745) Make HBCK2 be able to fix issues other than region assignment
[
https://issues.apache.org/jira/browse/HBASE-21745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-21745:
--
Description:
This is what [~apurtell] posted on mailing-list, HBCK2 should support
{quote}
- - Rebuild meta from region metadata in the filesystem, aka offline meta
rebuild.-
- Fix assignment errors (undeployed regions, double assignments (yes,
should not be possible), etc)
- Fix region holes, overlaps, and other errors in the region chain
- Fix failed split and merge transactions that have failed to roll back
due to some bug (related to previous)
- - Enumerate store files to determine file level corruption and sideline
corrupt files-
- - Fix hfile link problems (dangling / broken)
{quote}-
was:
This is what [~apurtell] posted on mailing-list, HBCK2 should support
{quote}
- Rebuild meta from region metadata in the filesystem, aka offline meta
rebuild.
- Fix assignment errors (undeployed regions, double assignments (yes,
should not be possible), etc)
- Fix region holes, overlaps, and other errors in the region chain
- Fix failed split and merge transactions that have failed to roll back
due to some bug (related to previous)
- Enumerate store files to determine file level corruption and sideline
corrupt files
- Fix hfile link problems (dangling / broken)
{quote}
> Make HBCK2 be able to fix issues other than region assignment
> -
>
> Key: HBASE-21745
> URL: https://issues.apache.org/jira/browse/HBASE-21745
> Project: HBase
> Issue Type: Umbrella
> Components: hbase-operator-tools, hbck2
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
>
> This is what [~apurtell] posted on mailing-list, HBCK2 should support
> {quote}
> - - Rebuild meta from region metadata in the filesystem, aka offline meta
>rebuild.-
>- Fix assignment errors (undeployed regions, double assignments (yes,
>should not be possible), etc)
>- Fix region holes, overlaps, and other errors in the region chain
>- Fix failed split and merge transactions that have failed to roll back
>due to some bug (related to previous)
> - - Enumerate store files to determine file level corruption and sideline
>corrupt files-
> - - Fix hfile link problems (dangling / broken)
> {quote}-
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] infraio merged pull request #389: HBASE-22704 Avoid NPE when access table.jsp and snapshot.jsp but mast…
infraio merged pull request #389: HBASE-22704 Avoid NPE when access table.jsp and snapshot.jsp but mast… URL: https://github.com/apache/hbase/pull/389 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-20368) Fix RIT stuck when a rsgroup has no online servers but AM's pendingAssginQueue is cleared
[
https://issues.apache.org/jira/browse/HBASE-20368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887501#comment-16887501
]
Hudson commented on HBASE-20368:
Results for branch branch-2
[build #2081 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2081/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2081//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2081//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2081//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 3. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2081//artifact/output-integration/hadoop-3.log].
(note that this means we didn't check the Hadoop 3 shaded client)
> Fix RIT stuck when a rsgroup has no online servers but AM's
> pendingAssginQueue is cleared
> -
>
> Key: HBASE-20368
> URL: https://issues.apache.org/jira/browse/HBASE-20368
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.0.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: HBASE-20368.branch-2.001.patch,
> HBASE-20368.branch-2.002.patch, HBASE-20368.branch-2.003.patch,
> HBASE-20368.branch-2.003.patch, HBASE-20368.branch-2.003.patch,
> HBASE-20368.branch-2.1.001.patch
>
>
> This error can be reproduced by shutting down all servers in a rsgroups and
> starting them soon afterwards.
> The regions on this rsgroup will be reassigned, but there is no available
> servers of this rsgroup.
> They will be added to AM's pendingAssginQueue, which AM will clear regardless
> of the result of assigning in this case.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22700) Incorrect timeout in recommended ZooKeeper configuration
[
https://issues.apache.org/jira/browse/HBASE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887502#comment-16887502
]
Hudson commented on HBASE-22700:
Results for branch branch-2
[build #2081 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2081/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2081//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2081//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2081//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 3. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2081//artifact/output-integration/hadoop-3.log].
(note that this means we didn't check the Hadoop 3 shaded client)
> Incorrect timeout in recommended ZooKeeper configuration
>
>
> Key: HBASE-22700
> URL: https://issues.apache.org/jira/browse/HBASE-22700
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Peter Somogyi
>Assignee: maoling
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
>
> The [Recommended
> configuration|https://hbase.apache.org/book.html#recommended_configurations.zk]
> section for ZooKeeper states that the default zookeeper.session.timeout is 3
> minutes, however, the [default
> configuration|https://github.com/apache/hbase/blob/master/hbase-common/src/main/resources/hbase-default.xml#L372-L373]
> is 90 seconds(
>
> {code:java}
> /** Default value for ZooKeeper session timeout */
> public static final int DEFAULT_ZK_SESSION_TIMEOUT = 90 * 1000;
> ).
> {code}
>
> This section in the documentation should be modified to reflect the default
> configuration.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase] saintstack commented on a change in pull request #373: HBASE-22527 [hbck2] Add a master web ui to show the problematic regions
saintstack commented on a change in pull request #373: HBASE-22527 [hbck2] Add a master web ui to show the problematic regions URL: https://github.com/apache/hbase/pull/373#discussion_r304687577 ## File path: hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/AssignmentManagerStatusTmpl.jamon ## @@ -17,27 +17,98 @@ See the License for the specific language governing permissions and limitations under the License. <%import> -org.apache.hadoop.hbase.master.assignment.AssignmentManager; -org.apache.hadoop.hbase.master.assignment.AssignmentManager.RegionInTransitionStat; -org.apache.hadoop.hbase.master.assignment.RegionStates.RegionFailedOpen; -org.apache.hadoop.hbase.master.RegionState; +java.util.Map; +java.util.Set; +java.util.SortedSet; +java.util.concurrent.atomic.AtomicInteger; +java.util.stream.Collectors; org.apache.hadoop.conf.Configuration; org.apache.hadoop.hbase.HBaseConfiguration; org.apache.hadoop.hbase.HConstants; +org.apache.hadoop.hbase.ServerName; +org.apache.hadoop.hbase.client.RegionInfo; org.apache.hadoop.hbase.client.RegionInfoDisplay; -java.util.HashSet; -java.util.SortedSet; -java.util.Map; -java.util.concurrent.atomic.AtomicInteger; +org.apache.hadoop.hbase.master.RegionState; +org.apache.hadoop.hbase.master.assignment.AssignmentManager; +org.apache.hadoop.hbase.master.assignment.AssignmentManager.RegionInTransitionStat; +org.apache.hadoop.hbase.master.assignment.RegionStates.RegionFailedOpen; +org.apache.hadoop.hbase.util.Pair; <%args> AssignmentManager assignmentManager; int limit = 100; -<%java SortedSet rit = assignmentManager - .getRegionStates().getRegionsInTransitionOrderedByTimestamp(); -%> +<%java> +SortedSet rit = assignmentManager.getRegionStates() +.getRegionsInTransitionOrderedByTimestamp(); +Map>> problematicRegions = assignmentManager +.getProblematicRegions(); + + +<%if !problematicRegions.isEmpty() %> +<%java> +int totalSize = problematicRegions.size(); +int sizePerPage = Math.min(10, totalSize); +int numOfPages = (int) Math.ceil(totalSize * 1.0 / sizePerPage); + + +Problematic Regions Review comment: Or, you say below what it is...* case 1. Master thought this region opened, but no regionserver reported it. * case 2. Master thought this region opened on Server1, but regionserver reported Server2 * case 3. More than one regionservers reported opened this region Can this be added in the UI maybe in small text? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #373: HBASE-22527 [hbck2] Add a master web ui to show the problematic regions
saintstack commented on a change in pull request #373: HBASE-22527 [hbck2] Add a master web ui to show the problematic regions URL: https://github.com/apache/hbase/pull/373#discussion_r304687345 ## File path: hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/AssignmentManagerStatusTmpl.jamon ## @@ -17,27 +17,98 @@ See the License for the specific language governing permissions and limitations under the License. <%import> -org.apache.hadoop.hbase.master.assignment.AssignmentManager; -org.apache.hadoop.hbase.master.assignment.AssignmentManager.RegionInTransitionStat; -org.apache.hadoop.hbase.master.assignment.RegionStates.RegionFailedOpen; -org.apache.hadoop.hbase.master.RegionState; +java.util.Map; +java.util.Set; +java.util.SortedSet; +java.util.concurrent.atomic.AtomicInteger; +java.util.stream.Collectors; org.apache.hadoop.conf.Configuration; org.apache.hadoop.hbase.HBaseConfiguration; org.apache.hadoop.hbase.HConstants; +org.apache.hadoop.hbase.ServerName; +org.apache.hadoop.hbase.client.RegionInfo; org.apache.hadoop.hbase.client.RegionInfoDisplay; -java.util.HashSet; -java.util.SortedSet; -java.util.Map; -java.util.concurrent.atomic.AtomicInteger; +org.apache.hadoop.hbase.master.RegionState; +org.apache.hadoop.hbase.master.assignment.AssignmentManager; +org.apache.hadoop.hbase.master.assignment.AssignmentManager.RegionInTransitionStat; +org.apache.hadoop.hbase.master.assignment.RegionStates.RegionFailedOpen; +org.apache.hadoop.hbase.util.Pair; <%args> AssignmentManager assignmentManager; int limit = 100; -<%java SortedSet rit = assignmentManager - .getRegionStates().getRegionsInTransitionOrderedByTimestamp(); -%> +<%java> +SortedSet rit = assignmentManager.getRegionStates() +.getRegionsInTransitionOrderedByTimestamp(); +Map>> problematicRegions = assignmentManager +.getProblematicRegions(); + + +<%if !problematicRegions.isEmpty() %> +<%java> +int totalSize = problematicRegions.size(); +int sizePerPage = Math.min(10, totalSize); +int numOfPages = (int) Math.ceil(totalSize * 1.0 / sizePerPage); + + +Problematic Regions Review comment: On commit, add a sentence that says what a problematic region is. It seems like its one that has a meta entry that does not agree w/ where it is actually deployed? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21393) Add an API ScheduleSCP() to HBCK2
[ https://issues.apache.org/jira/browse/HBASE-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887497#comment-16887497 ] stack commented on HBASE-21393: --- Looking for reviews of PR. Added a version check as part of this patch which should work going forward as we add actions to hbck2. > Add an API ScheduleSCP() to HBCK2 > -- > > Key: HBASE-21393 > URL: https://issues.apache.org/jira/browse/HBASE-21393 > Project: HBase > Issue Type: Sub-task > Components: hbase-operator-tools, hbck2 >Reporter: Jingyun Tian >Assignee: Jingyun Tian >Priority: Major > Attachments: 0001-HBASE-21393-Add-an-API-ScheduleSCP-to-HBCK2.patch, > 0002-HBASE-21393-Add-an-API-ScheduleSCP-to-HBCK2.patch > > > Add the API of ScheduleSCP() to hbase-operator-tools so that operators can > schedule ServerCrashProcedures of specified regionservers. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase-operator-tools] saintstack commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
saintstack commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#issuecomment-512609108 Ok. Going to stop messing w/ this checkstyle thing. Its my favorite, import order. Makes no sense. Reviews please! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#issuecomment-512608842 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/49/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#issuecomment-512606781 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/48/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#issuecomment-512602836 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/47/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#issuecomment-512599606 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/46/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887488#comment-16887488
]
Hudson commented on HBASE-7191:
---
Results for branch branch-2.1
[build #1377 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1377/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1377//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1377//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1377//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Sub-task
> Components: hbck
>Affects Versions: 0.94.1
>Reporter: Enis Soztutar
>Assignee: xufeng
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: 7191-2.patch, HBASE-7191.branch-2.001.patch,
> hbck1-1.4-v1.patch
>
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887478#comment-16887478
]
Hudson commented on HBASE-7191:
---
Results for branch branch-2.2
[build #440 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/440/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/440//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/440//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/440//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Sub-task
> Components: hbck
>Affects Versions: 0.94.1
>Reporter: Enis Soztutar
>Assignee: xufeng
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: 7191-2.patch, HBASE-7191.branch-2.001.patch,
> hbck1-1.4-v1.patch
>
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887466#comment-16887466
]
Hudson commented on HBASE-7191:
---
Results for branch branch-2
[build #2082 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2082/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2082//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2082//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2082//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Sub-task
> Components: hbck
>Affects Versions: 0.94.1
>Reporter: Enis Soztutar
>Assignee: xufeng
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: 7191-2.patch, HBASE-7191.branch-2.001.patch,
> hbck1-1.4-v1.patch
>
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22700) Incorrect timeout in recommended ZooKeeper configuration
[
https://issues.apache.org/jira/browse/HBASE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887462#comment-16887462
]
Hudson commented on HBASE-22700:
Results for branch branch-2.1
[build #1376 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1376/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1376//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1376//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/1376//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Incorrect timeout in recommended ZooKeeper configuration
>
>
> Key: HBASE-22700
> URL: https://issues.apache.org/jira/browse/HBASE-22700
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Peter Somogyi
>Assignee: maoling
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
>
> The [Recommended
> configuration|https://hbase.apache.org/book.html#recommended_configurations.zk]
> section for ZooKeeper states that the default zookeeper.session.timeout is 3
> minutes, however, the [default
> configuration|https://github.com/apache/hbase/blob/master/hbase-common/src/main/resources/hbase-default.xml#L372-L373]
> is 90 seconds(
>
> {code:java}
> /** Default value for ZooKeeper session timeout */
> public static final int DEFAULT_ZK_SESSION_TIMEOUT = 90 * 1000;
> ).
> {code}
>
> This section in the documentation should be modified to reflect the default
> configuration.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HBASE-22685) Add to migration doc that meta should be healthy before upgrade
[ https://issues.apache.org/jira/browse/HBASE-22685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-22685: -- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) I pushed this small doc change. > Add to migration doc that meta should be healthy before upgrade > --- > > Key: HBASE-22685 > URL: https://issues.apache.org/jira/browse/HBASE-22685 > Project: HBase > Issue Type: Sub-task > Components: documentation >Reporter: stack >Assignee: stack >Priority: Major > Labels: documentation > Fix For: 3.0.0 > > Attachments: HBASE-22685.master.001.patch > > > Add a note to doc that operator ensures healthy meta before upgrade. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase-operator-tools] asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#issuecomment-512582428 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/45/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#issuecomment-512570493 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/44/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#issuecomment-512566766 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/43/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-13672) Run tests in a Docker container under Jenkins
[ https://issues.apache.org/jira/browse/HBASE-13672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-13672. --- Resolution: Implemented The current pre-commit and nightly tests run in Docker containers via Yetus. > Run tests in a Docker container under Jenkins > - > > Key: HBASE-13672 > URL: https://issues.apache.org/jira/browse/HBASE-13672 > Project: HBase > Issue Type: Test >Reporter: Andrew Purtell >Priority: Major > > On HADOOP-11933, Hadoop is thinking about running their precommit builds in a > Docker container under Jenkins. If they can get this working, this is > definitely something to look at adopting here. Unquestionably it will improve > test repeatability and stability by reducing variability we see from test VM > to test VM. Could be useful for all builds, not just precommit. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (HBASE-13650) Tests failing for 0.94.x against hadoop 2.x
[ https://issues.apache.org/jira/browse/HBASE-13650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-13650. --- Resolution: Won't Fix > Tests failing for 0.94.x against hadoop 2.x > --- > > Key: HBASE-13650 > URL: https://issues.apache.org/jira/browse/HBASE-13650 > Project: HBase > Issue Type: Test >Affects Versions: 0.94.18 >Reporter: Kuldeep >Priority: Major > > After changing hadoop.version in hadoop-2.0 profile and regenrating protobuf > files as mentioned here > https://hbase.apache.org/book.html#hadoop2.hbase_0.94 > tests agains hadoop 2.4.0 fail with following expceptions > Results : > Tests in error: > > testFailingCreateTable(org.apache.hadoop.hbase.rest.client.TestRemoteAdminRetries): > This is supposed to be overridden by subclasses. > > testGetScanner(org.apache.hadoop.hbase.rest.client.TestRemoteHTableRetries): > This is supposed to be overridden by subclasses. > > testSingleRowPut(org.apache.hadoop.hbase.rest.client.TestRemoteHTableRetries): > This is supposed to be overridden by subclasses. > > testMultiRowPut(org.apache.hadoop.hbase.rest.client.TestRemoteHTableRetries): > This is supposed to be overridden by subclasses. > > testCheckAndPut(org.apache.hadoop.hbase.rest.client.TestRemoteHTableRetries): > This is supposed to be overridden by subclasses. > > testCheckAndDelete(org.apache.hadoop.hbase.rest.client.TestRemoteHTableRetries): > This is supposed to be overridden by subclasses. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (HBASE-9795) Backport the type support
[ https://issues.apache.org/jira/browse/HBASE-9795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-9795. -- Resolution: Won't Fix Stale. > Backport the type support > - > > Key: HBASE-9795 > URL: https://issues.apache.org/jira/browse/HBASE-9795 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Diego Masini >Priority: Major > > Backport type support from v0.95.2 to 0.94. See HBASE-8089 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-18273) hbase_rotate_log in hbase-daemon.sh script not working for some JDK
[ https://issues.apache.org/jira/browse/HBASE-18273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi updated HBASE-18273: -- Resolution: Won't Fix Status: Resolved (was: Patch Available) > hbase_rotate_log in hbase-daemon.sh script not working for some JDK > --- > > Key: HBASE-18273 > URL: https://issues.apache.org/jira/browse/HBASE-18273 > Project: HBase > Issue Type: Bug >Affects Versions: 1.2.6, 1.1.11, 2.0.0-alpha-1 >Reporter: Fangyuan Deng >Assignee: Fangyuan Deng >Priority: Major > Attachments: HBASE-18273.0.patch, HBASE-18273.1.patch, > HBASE-18273.2.patch > > > When restarting a hbase process, hbase_rotate_log $HBASE_LOGGC will rotate > GC logs. > the code looks like this, > if [ -f "$log" ]; then # rotate logs > while [ $num -gt 1 ]; do > prev=`expr $num - 1` > [ -f "$log.$prev" ] && mv -f "$log.$prev" "$log.$num" > num=$prev > done > But, some version JDK will add a suffix (.0) to the gc file, like > hbase-xxx.gc.0, rather than hbase-xxx.gc. > So I add a check before rotate, > if [ ! -f "$log" ]; then #for some jdk, gc log has a postfix 0 > if [ -f "$log.0" ]; then > mv -f "$log.0" "$log"; > fi > fi -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887384#comment-16887384
]
Hudson commented on HBASE-7191:
---
Results for branch branch-2.0
[build #1766 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1766/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1766//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1766//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1766//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Sub-task
> Components: hbck
>Affects Versions: 0.94.1
>Reporter: Enis Soztutar
>Assignee: xufeng
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: 7191-2.patch, HBASE-7191.branch-2.001.patch,
> hbck1-1.4-v1.patch
>
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22700) Incorrect timeout in recommended ZooKeeper configuration
[
https://issues.apache.org/jira/browse/HBASE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887383#comment-16887383
]
Hudson commented on HBASE-22700:
Results for branch branch-2.2
[build #439 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/439/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/439//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/439//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/439//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Incorrect timeout in recommended ZooKeeper configuration
>
>
> Key: HBASE-22700
> URL: https://issues.apache.org/jira/browse/HBASE-22700
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Peter Somogyi
>Assignee: maoling
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
>
> The [Recommended
> configuration|https://hbase.apache.org/book.html#recommended_configurations.zk]
> section for ZooKeeper states that the default zookeeper.session.timeout is 3
> minutes, however, the [default
> configuration|https://github.com/apache/hbase/blob/master/hbase-common/src/main/resources/hbase-default.xml#L372-L373]
> is 90 seconds(
>
> {code:java}
> /** Default value for ZooKeeper session timeout */
> public static final int DEFAULT_ZK_SESSION_TIMEOUT = 90 * 1000;
> ).
> {code}
>
> This section in the documentation should be modified to reflect the default
> configuration.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-20368) Fix RIT stuck when a rsgroup has no online servers but AM's pendingAssginQueue is cleared
[
https://issues.apache.org/jira/browse/HBASE-20368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887382#comment-16887382
]
Hudson commented on HBASE-20368:
Results for branch branch-2.2
[build #439 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/439/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/439//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/439//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/439//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix RIT stuck when a rsgroup has no online servers but AM's
> pendingAssginQueue is cleared
> -
>
> Key: HBASE-20368
> URL: https://issues.apache.org/jira/browse/HBASE-20368
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.0.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: HBASE-20368.branch-2.001.patch,
> HBASE-20368.branch-2.002.patch, HBASE-20368.branch-2.003.patch,
> HBASE-20368.branch-2.003.patch, HBASE-20368.branch-2.003.patch,
> HBASE-20368.branch-2.1.001.patch
>
>
> This error can be reproduced by shutting down all servers in a rsgroups and
> starting them soon afterwards.
> The regions on this rsgroup will be reassigned, but there is no available
> servers of this rsgroup.
> They will be added to AM's pendingAssginQueue, which AM will clear regardless
> of the result of assigning in this case.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-20368) Fix RIT stuck when a rsgroup has no online servers but AM's pendingAssginQueue is cleared
[
https://issues.apache.org/jira/browse/HBASE-20368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887374#comment-16887374
]
Hudson commented on HBASE-20368:
Results for branch master
[build #1232 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1232/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1232//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1232//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1232//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix RIT stuck when a rsgroup has no online servers but AM's
> pendingAssginQueue is cleared
> -
>
> Key: HBASE-20368
> URL: https://issues.apache.org/jira/browse/HBASE-20368
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 2.0.0
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: HBASE-20368.branch-2.001.patch,
> HBASE-20368.branch-2.002.patch, HBASE-20368.branch-2.003.patch,
> HBASE-20368.branch-2.003.patch, HBASE-20368.branch-2.003.patch,
> HBASE-20368.branch-2.1.001.patch
>
>
> This error can be reproduced by shutting down all servers in a rsgroups and
> starting them soon afterwards.
> The regions on this rsgroup will be reassigned, but there is no available
> servers of this rsgroup.
> They will be added to AM's pendingAssginQueue, which AM will clear regardless
> of the result of assigning in this case.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22700) Incorrect timeout in recommended ZooKeeper configuration
[
https://issues.apache.org/jira/browse/HBASE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887364#comment-16887364
]
Hudson commented on HBASE-22700:
Results for branch branch-2.0
[build #1765 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1765/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1765//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1765//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1765//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Incorrect timeout in recommended ZooKeeper configuration
>
>
> Key: HBASE-22700
> URL: https://issues.apache.org/jira/browse/HBASE-22700
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Peter Somogyi
>Assignee: maoling
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
>
> The [Recommended
> configuration|https://hbase.apache.org/book.html#recommended_configurations.zk]
> section for ZooKeeper states that the default zookeeper.session.timeout is 3
> minutes, however, the [default
> configuration|https://github.com/apache/hbase/blob/master/hbase-common/src/main/resources/hbase-default.xml#L372-L373]
> is 90 seconds(
>
> {code:java}
> /** Default value for ZooKeeper session timeout */
> public static final int DEFAULT_ZK_SESSION_TIMEOUT = 90 * 1000;
> ).
> {code}
>
> This section in the documentation should be modified to reflect the default
> configuration.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22700) Incorrect timeout in recommended ZooKeeper configuration
[
https://issues.apache.org/jira/browse/HBASE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887363#comment-16887363
]
Hudson commented on HBASE-22700:
Results for branch master
[build #1233 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1233/]: (/)
*{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1233//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1233//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1233//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Incorrect timeout in recommended ZooKeeper configuration
>
>
> Key: HBASE-22700
> URL: https://issues.apache.org/jira/browse/HBASE-22700
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Peter Somogyi
>Assignee: maoling
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
>
> The [Recommended
> configuration|https://hbase.apache.org/book.html#recommended_configurations.zk]
> section for ZooKeeper states that the default zookeeper.session.timeout is 3
> minutes, however, the [default
> configuration|https://github.com/apache/hbase/blob/master/hbase-common/src/main/resources/hbase-default.xml#L372-L373]
> is 90 seconds(
>
> {code:java}
> /** Default value for ZooKeeper session timeout */
> public static final int DEFAULT_ZK_SESSION_TIMEOUT = 90 * 1000;
> ).
> {code}
>
> This section in the documentation should be modified to reflect the default
> configuration.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-7191) HBCK - Add offline create/fix hbase.version and hbase.id
[
https://issues.apache.org/jira/browse/HBASE-7191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887361#comment-16887361
]
Hudson commented on HBASE-7191:
---
Results for branch master
[build #1234 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1234/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1234//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1234//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1234//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> HBCK - Add offline create/fix hbase.version and hbase.id
> -
>
> Key: HBASE-7191
> URL: https://issues.apache.org/jira/browse/HBASE-7191
> Project: HBase
> Issue Type: Sub-task
> Components: hbck
>Affects Versions: 0.94.1
>Reporter: Enis Soztutar
>Assignee: xufeng
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: 7191-2.patch, HBASE-7191.branch-2.001.patch,
> hbck1-1.4-v1.patch
>
>
> One of our clients run into a problem, in which they have the hbase.root.dir,
> and cluster data, but their hbase.id and hbase.version files are corrupted.
> HMaster creates those on start, but not if there is already existing data.
> We can add smt like --fixIdFile, and ability for HBCK to do some offline
> repairs for the version file.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase-operator-tools] asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
asf-ci commented on issue #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#issuecomment-512522967 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/42/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] saintstack commented on a change in pull request #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService
saintstack commented on a change in pull request #11: HBASE-21322 Add a scheduleServerCrashProcedure() API to HbckService URL: https://github.com/apache/hbase-operator-tools/pull/11#discussion_r304584934 ## File path: hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java ## @@ -88,6 +91,7 @@ private static final String FILESYSTEM = "filesystem"; private static final String VERSION = "version"; private static final String SET_REGION_STATE = "setRegionState"; + private static final String SCHEDULE_SCP = "serverCrashProcedures"; Review comment: Called it scheduleRecoveries (to match other commands which are plural to reflect they can do bulk) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] gjacoby126 commented on issue #390: HBASE-22623 - Add RegionObserver coprocessor hook for preWALAppend
gjacoby126 commented on issue #390: HBASE-22623 - Add RegionObserver coprocessor hook for preWALAppend URL: https://github.com/apache/hbase/pull/390#issuecomment-512515804 @apurtell - here's my implementation of HBASE-22623. In the end I did manage to get everything working through a single coprocessor hook, but coprocs will need something like Phoenix's BatchMutateContext (or eventually HBASE-18127) for the mutation context. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] gjacoby126 opened a new pull request #390: HBASE-22623 - Add RegionObserver coprocessor hook for preWALAppend
gjacoby126 opened a new pull request #390: HBASE-22623 - Add RegionObserver coprocessor hook for preWALAppend URL: https://github.com/apache/hbase/pull/390 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22567) HBCK2 addMissingRegionsToMeta
[
https://issues.apache.org/jira/browse/HBASE-22567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887254#comment-16887254
]
Wellington Chevreuil commented on HBASE-22567:
--
Right, had pushed new commits to this PR. [~stack], [~busbey], please let me
know if latest PR changes are ok. I had also opened HBASE-22707, to address the
need for reloading _RegionsStateStore_ with the new regions added to meta. Let
me know what do you guys think on that.
> HBCK2 addMissingRegionsToMeta
> -
>
> Key: HBASE-22567
> URL: https://issues.apache.org/jira/browse/HBASE-22567
> Project: HBase
> Issue Type: New Feature
> Components: hbck2
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> Following latest discussion on HBASE-21745, this proposes an hbck2 command
> that allows for inserting back regions missing in META that still have
> *regioninfo* available in HDFS. Although this is still an interactive and
> simpler version than the old _OfflineMetaRepair_, it still relies on hdfs
> state as the source of truth, and performs META updates mostly independently
> from Master (apart from requiring Meta table been online).
> For a more detailed explanation on this command behaviour, pasting _command
> usage_ text:
> {noformat}
> To be used for scenarios where some regions may be missing in META,
> but there's still a valid 'regioninfo' metadata file on HDFS.
> This is a lighter version of 'OfflineMetaRepair' tool commonly used for
> similar issues on 1.x release line.
> This command needs META to be online. For each table name passed as
> parameter, it performs a diff between regions available in META,
> against existing regions dirs on HDFS. Then, for region dirs with
> no matches in META, it reads regioninfo metadata file and
> re-creates given region in META. Regions are re-created in 'CLOSED'
> state at META table only, but not in Masters' cache, and are not
> assigned either. A rolling Masters restart, followed by a
> hbck2 'assigns' command with all re-inserted regions is required.
> This hbck2 'assigns' command is printed for user convenience.
> WARNING: To avoid potential region overlapping problems due to ongoing
> splits, this command disables given tables while re-inserting regions.
> An example adding missing regions for tables 'table_1' and 'table_2':
> $ HBCK2 addMissingRegionsInMeta table_1 table_2
> Returns hbck2 'assigns' command with all re-inserted regions.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HBASE-22628) Document the custom WAL directory (hbase.wal.dir) usage
[
https://issues.apache.org/jira/browse/HBASE-22628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887252#comment-16887252
]
HBase QA commented on HBASE-22628:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
55s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 11m
50s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 7m
52s{color} | {color:blue} patch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=18.09.7 Server=18.09.7 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/644/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22628 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12975071/HBASE-22628.patch |
| Optional Tests | dupname asflicense refguide |
| uname | Linux 1a9d2e3a18f7 4.4.0-137-generic #163-Ubuntu SMP Mon Sep 24
13:14:43 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 82ea04dbab |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/644/artifact/patchprocess/branch-site/book.html
|
| whitespace |
https://builds.apache.org/job/PreCommit-HBASE-Build/644/artifact/patchprocess/whitespace-eol.txt
|
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/644/artifact/patchprocess/patch-site/book.html
|
| Max. process+thread count | 95 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/644/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |
This message was automatically generated.
> Document the custom WAL directory (hbase.wal.dir) usage
> ---
>
> Key: HBASE-22628
> URL: https://issues.apache.org/jira/browse/HBASE-22628
> Project: HBase
> Issue Type: Bug
> Components: documentation, wal
>Reporter: Pankaj Kumar
>Assignee: Pankaj Kumar
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HBASE-22628.patch
>
>
> Custom WAL directory usage must be documented, otherwise it may lead to
> inconsistent data during migrating to new WAL dir path.
>
> You can consider below scenario while migrating to custom WAL directory.
> # Setup HBase cluster with the default setting (all WAL files are under the
> root directory ie. /hbase/WALs).
> # Create table 't1' and insert few records
> # Flush meta table (so that table region entries persist in FS)
> # Forcibly kill HBase processes (HM & RS).
> # Configure the hbase.wal.dir to outside the root dir (say /hbaseWAL)
> # Start the HBase servers
> # Scan 't1'
> Ideally HMaster should submit split task of old RS(s) WAL files (created
> under /hbase/WALs) and old data should be replayed. But currently, during HM
> startup we populate the previous dead servers from the current WAL dir (
> hbase.wal.dir -> /hbaseWAL).
>
> Since WAL dir path is new, so you need to copy RegionServer WAL directories
> manualy from old WAL dir to new path.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [hbase-operator-tools] wchevreuil commented on a change in pull request #3: Hbase 22567
wchevreuil commented on a change in pull request #3: Hbase 22567
URL: https://github.com/apache/hbase-operator-tools/pull/3#discussion_r304545722
##
File path: hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java
##
@@ -334,6 +469,51 @@ private static final String getCommandUsage() {
writer.println(" Returns \"0\" SUCCESS code if it informed region state
is changed, "
+ "\"1\" FAIL code otherwise.");
writer.println();
+writer.println(" " + ADD_MISSING_REGIONS_IN_META_FOR_TABLES + "
...");
+writer.println(" To be used in scenarios where some regions may be
missing in META,");
+writer.println(" but there's still a valid 'regioninfo metadata file on
HDFS. ");
+writer.println(" This is a lighter version of 'OfflineMetaRepair tool
commonly used for ");
+writer.println(" similar issues on 1.x release line. ");
+writer.println(" This command needs META to be online. For each table
name passed as");
+writer.println(" parameter, it performs a diff between regions available
in META, ");
+writer.println(" against existing regions dirs on HDFS. Then, for region
dirs with ");
+writer.println(" no matches in META, it reads regioninfo metadata file
and ");
+writer.println(" re-creates given region in META. Regions are re-created
in 'CLOSED' ");
+writer.println(" state at META table only, but not in Masters' cache,
and are not ");
+writer.println(" assigned either. A rolling Masters restart, followed by
a ");
Review comment:
Opened HBASE-22707 for this. Decided to embed it on current
_MasterRpcServices.assigns_ command, instead of defining a whole new RPC method
for meta reload.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (HBASE-22707) [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore
[ https://issues.apache.org/jira/browse/HBASE-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-22707: - Status: Patch Available (was: Open) > [HBCK2] MasterRpcServices assigns method should try to reload regions from > meta if the passed regions isn't found under AssignmentManager > RegionsStateStore > --- > > Key: HBASE-22707 > URL: https://issues.apache.org/jira/browse/HBASE-22707 > Project: HBase > Issue Type: Task > Components: hbck2, master >Affects Versions: 2.1.5, 2.2.0, 2.3.0 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 2.3.0 > > Attachments: HBASE-22707.branch-2.001.patch > > > Although HBCK2 related, this is a master side improvement. On situations > where regions are missing in META, any online fix tool such as the one being > implemented in HBASE-22567 would require a further master restart to get > RegionsStateStore reloaded from META, so that master can be aware of the > newly re-added regions. > After regions are re-added to meta in CLOSED state, it should be possible to > bring those by simply invoking hbck2 _assigns_ command. But before > _MasterRpcServices.assigns_ submits an _Assign_ procedure, it validates first > if the given region is available on _AssignmentManager.RegionsStateStore_. > The current patch reloads meta on _MasterRpcServices.assigns_ if the given > region is not found on the first lookup, then try a new lookup again before > giving-up on region assignment. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22707) [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore
[ https://issues.apache.org/jira/browse/HBASE-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-22707: - Attachment: HBASE-22707.branch-2.001.patch > [HBCK2] MasterRpcServices assigns method should try to reload regions from > meta if the passed regions isn't found under AssignmentManager > RegionsStateStore > --- > > Key: HBASE-22707 > URL: https://issues.apache.org/jira/browse/HBASE-22707 > Project: HBase > Issue Type: Task > Components: hbck2, master >Affects Versions: 2.2.0, 2.3.0, 2.1.5 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 2.3.0 > > Attachments: HBASE-22707.branch-2.001.patch > > > Although HBCK2 related, this is a master side improvement. On situations > where regions are missing in META, any online fix tool such as the one being > implemented in HBASE-22567 would require a further master restart to get > RegionsStateStore reloaded from META, so that master can be aware of the > newly re-added regions. > After regions are re-added to meta in CLOSED state, it should be possible to > bring those by simply invoking hbck2 _assigns_ command. But before > _MasterRpcServices.assigns_ submits an _Assign_ procedure, it validates first > if the given region is available on _AssignmentManager.RegionsStateStore_. > The current patch reloads meta on _MasterRpcServices.assigns_ if the given > region is not found on the first lookup, then try a new lookup again before > giving-up on region assignment. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22707) [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore
[ https://issues.apache.org/jira/browse/HBASE-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-22707: - Attachment: HBASE-22707.branch-2.001 > [HBCK2] MasterRpcServices assigns method should try to reload regions from > meta if the passed regions isn't found under AssignmentManager > RegionsStateStore > --- > > Key: HBASE-22707 > URL: https://issues.apache.org/jira/browse/HBASE-22707 > Project: HBase > Issue Type: Task > Components: hbck2, master >Affects Versions: 2.2.0, 2.3.0, 2.1.5 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 2.3.0 > > > Although HBCK2 related, this is a master side improvement. On situations > where regions are missing in META, any online fix tool such as the one being > implemented in HBASE-22567 would require a further master restart to get > RegionsStateStore reloaded from META, so that master can be aware of the > newly re-added regions. > After regions are re-added to meta in CLOSED state, it should be possible to > bring those by simply invoking hbck2 _assigns_ command. But before > _MasterRpcServices.assigns_ submits an _Assign_ procedure, it validates first > if the given region is available on _AssignmentManager.RegionsStateStore_. > The current patch reloads meta on _MasterRpcServices.assigns_ if the given > region is not found on the first lookup, then try a new lookup again before > giving-up on region assignment. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22707) [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore
[ https://issues.apache.org/jira/browse/HBASE-22707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-22707: - Attachment: (was: HBASE-22707.branch-2.001) > [HBCK2] MasterRpcServices assigns method should try to reload regions from > meta if the passed regions isn't found under AssignmentManager > RegionsStateStore > --- > > Key: HBASE-22707 > URL: https://issues.apache.org/jira/browse/HBASE-22707 > Project: HBase > Issue Type: Task > Components: hbck2, master >Affects Versions: 2.2.0, 2.3.0, 2.1.5 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 2.3.0 > > > Although HBCK2 related, this is a master side improvement. On situations > where regions are missing in META, any online fix tool such as the one being > implemented in HBASE-22567 would require a further master restart to get > RegionsStateStore reloaded from META, so that master can be aware of the > newly re-added regions. > After regions are re-added to meta in CLOSED state, it should be possible to > bring those by simply invoking hbck2 _assigns_ command. But before > _MasterRpcServices.assigns_ submits an _Assign_ procedure, it validates first > if the given region is available on _AssignmentManager.RegionsStateStore_. > The current patch reloads meta on _MasterRpcServices.assigns_ if the given > region is not found on the first lookup, then try a new lookup again before > giving-up on region assignment. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (HBASE-22707) [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore
Wellington Chevreuil created HBASE-22707: Summary: [HBCK2] MasterRpcServices assigns method should try to reload regions from meta if the passed regions isn't found under AssignmentManager RegionsStateStore Key: HBASE-22707 URL: https://issues.apache.org/jira/browse/HBASE-22707 Project: HBase Issue Type: Task Components: hbck2, master Affects Versions: 2.1.5, 2.2.0, 2.3.0 Reporter: Wellington Chevreuil Assignee: Wellington Chevreuil Fix For: 2.3.0 Although HBCK2 related, this is a master side improvement. On situations where regions are missing in META, any online fix tool such as the one being implemented in HBASE-22567 would require a further master restart to get RegionsStateStore reloaded from META, so that master can be aware of the newly re-added regions. After regions are re-added to meta in CLOSED state, it should be possible to bring those by simply invoking hbck2 _assigns_ command. But before _MasterRpcServices.assigns_ submits an _Assign_ procedure, it validates first if the given region is available on _AssignmentManager.RegionsStateStore_. The current patch reloads meta on _MasterRpcServices.assigns_ if the given region is not found on the first lookup, then try a new lookup again before giving-up on region assignment. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HBASE-22648) Snapshot TTL
[ https://issues.apache.org/jira/browse/HBASE-22648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-22648: - Attachment: HBASE-22648-master-v6.patch > Snapshot TTL > > > Key: HBASE-22648 > URL: https://issues.apache.org/jira/browse/HBASE-22648 > Project: HBase > Issue Type: New Feature > Components: snapshots >Affects Versions: 3.0.0 >Reporter: Andrew Purtell >Assignee: Viraj Jasani >Priority: Minor > Fix For: 3.0.0 > > Attachments: HBASE-22648-master-v2.patch, > HBASE-22648-master-v3.patch, HBASE-22648-master-v4.patch, > HBASE-22648-master-v5.patch, HBASE-22648-master-v6.patch, > HBASE-22648-master.patch, Screen Shot 2019-07-10 at 8.49.13 PM.png, Screen > Shot 2019-07-10 at 8.52.30 PM.png, Screen Shot 2019-07-10 at 9.06.36 PM.png, > Screen Shot 2019-07-16 at 11.06.03 AM.png > > > Snapshots have a lifecycle that is independent from the table from which they > are created. Although data in a table may be stored with TTL the data files > containing them become frozen by the snapshot. Space consumed by expired > cells will not be reclaimed by normal table housekeeping like compaction. > While this is expected it can be inconvenient at scale. When many snapshots > are under management and the data in various tables is expired by TTL some > notion of optional TTL (and optional default TTL) for snapshots could be > useful. It will help prevent the accumulation of junk files by automatically > dropping the snapshot after the assigned TTL, making their data files > eligible for cleaning. More comprehensive snapshot lifecycle management may > be considered in the future but this one case is expected to be immediately > useful given TTls on data are commonly applied for similar convenience. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase] virajjasani commented on issue #371: HBASE-22648 : Introducing Snapshot TTL
virajjasani commented on issue #371: HBASE-22648 : Introducing Snapshot TTL URL: https://github.com/apache/hbase/pull/371#issuecomment-512359437 > LGTM, and I think Andrew's comments are good and should be followed. > And please make sure the QA failed tests warning in JIRA are unrelated, because i can still see a snapshot related failed test, `TestSnapshotTemporaryDirectory` Thanks @Reidd The test is working fine locally and unit test is also +1 in recent commit summary This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22628) Document the custom WAL directory (hbase.wal.dir) usage
[ https://issues.apache.org/jira/browse/HBASE-22628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-22628: - Fix Version/s: 3.0.0 Status: Patch Available (was: Open) > Document the custom WAL directory (hbase.wal.dir) usage > --- > > Key: HBASE-22628 > URL: https://issues.apache.org/jira/browse/HBASE-22628 > Project: HBase > Issue Type: Bug > Components: documentation, wal >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Fix For: 3.0.0 > > Attachments: HBASE-22628.patch > > > Custom WAL directory usage must be documented, otherwise it may lead to > inconsistent data during migrating to new WAL dir path. > > You can consider below scenario while migrating to custom WAL directory. > # Setup HBase cluster with the default setting (all WAL files are under the > root directory ie. /hbase/WALs). > # Create table 't1' and insert few records > # Flush meta table (so that table region entries persist in FS) > # Forcibly kill HBase processes (HM & RS). > # Configure the hbase.wal.dir to outside the root dir (say /hbaseWAL) > # Start the HBase servers > # Scan 't1' > Ideally HMaster should submit split task of old RS(s) WAL files (created > under /hbase/WALs) and old data should be replayed. But currently, during HM > startup we populate the previous dead servers from the current WAL dir ( > hbase.wal.dir -> /hbaseWAL). > > Since WAL dir path is new, so you need to copy RegionServer WAL directories > manualy from old WAL dir to new path. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HBASE-21879) Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
[
https://issues.apache.org/jira/browse/HBASE-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887205#comment-16887205
]
Hudson commented on HBASE-21879:
Results for branch HBASE-21879
[build #180 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/180/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/180//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/180//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/180//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Read HFile's block to ByteBuffer directly instead of to byte for reducing
> young gc purpose
> --
>
> Key: HBASE-21879
> URL: https://issues.apache.org/jira/browse/HBASE-21879
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21879.v1.patch, HBASE-21879.v1.patch,
> QPS-latencies-before-HBASE-21879.png, gc-data-before-HBASE-21879.png
>
>
> In HFileBlock#readBlockDataInternal, we have the following:
> {code}
> @VisibleForTesting
> protected HFileBlock readBlockDataInternal(FSDataInputStream is, long offset,
> long onDiskSizeWithHeaderL, boolean pread, boolean verifyChecksum,
> boolean updateMetrics)
> throws IOException {
> // .
> // TODO: Make this ByteBuffer-based. Will make it easier to go to HDFS with
> BBPool (offheap).
> byte [] onDiskBlock = new byte[onDiskSizeWithHeader + hdrSize];
> int nextBlockOnDiskSize = readAtOffset(is, onDiskBlock, preReadHeaderSize,
> onDiskSizeWithHeader - preReadHeaderSize, true, offset +
> preReadHeaderSize, pread);
> if (headerBuf != null) {
> // ...
> }
> // ...
> }
> {code}
> In the read path, we still read the block from hfile to on-heap byte[], then
> copy the on-heap byte[] to offheap bucket cache asynchronously, and in my
> 100% get performance test, I also observed some frequent young gc, The
> largest memory footprint in the young gen should be the on-heap block byte[].
> In fact, we can read HFile's block to ByteBuffer directly instead of to
> byte[] for reducing young gc purpose. we did not implement this before,
> because no ByteBuffer reading interface in the older HDFS client, but 2.7+
> has supported this now, so we can fix this now. I think.
> Will provide an patch and some perf-comparison for this.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HBASE-22628) Document the custom WAL directory (hbase.wal.dir) usage
[ https://issues.apache.org/jira/browse/HBASE-22628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-22628: - Attachment: HBASE-22628.patch > Document the custom WAL directory (hbase.wal.dir) usage > --- > > Key: HBASE-22628 > URL: https://issues.apache.org/jira/browse/HBASE-22628 > Project: HBase > Issue Type: Bug > Components: documentation, wal >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Critical > Attachments: HBASE-22628.patch > > > Custom WAL directory usage must be documented, otherwise it may lead to > inconsistent data during migrating to new WAL dir path. > > You can consider below scenario while migrating to custom WAL directory. > # Setup HBase cluster with the default setting (all WAL files are under the > root directory ie. /hbase/WALs). > # Create table 't1' and insert few records > # Flush meta table (so that table region entries persist in FS) > # Forcibly kill HBase processes (HM & RS). > # Configure the hbase.wal.dir to outside the root dir (say /hbaseWAL) > # Start the HBase servers > # Scan 't1' > Ideally HMaster should submit split task of old RS(s) WAL files (created > under /hbase/WALs) and old data should be replayed. But currently, during HM > startup we populate the previous dead servers from the current WAL dir ( > hbase.wal.dir -> /hbaseWAL). > > Since WAL dir path is new, so you need to copy RegionServer WAL directories > manualy from old WAL dir to new path. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [hbase-operator-tools] asf-ci commented on issue #3: Hbase 22567
asf-ci commented on issue #3: Hbase 22567 URL: https://github.com/apache/hbase-operator-tools/pull/3#issuecomment-512348670 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/41/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
