[jira] [Created] (HBASE-22016) Rewrite the block reading methods by using hbase.nio.ByteBuff
Zheng Hu created HBASE-22016: Summary: Rewrite the block reading methods by using hbase.nio.ByteBuff Key: HBASE-22016 URL: https://issues.apache.org/jira/browse/HBASE-22016 Project: HBase Issue Type: Sub-task Reporter: Zheng Hu Assignee: Zheng Hu We've some useful discussion in HBASE-22005, so open an new JIRA for the ByteBuffer block reading. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22015) UserPermission should be annotated as InterfaceAudience.Public
Yi Mei created HBASE-22015:
--
Summary: UserPermission should be annotated as
InterfaceAudience.Public
Key: HBASE-22015
URL: https://issues.apache.org/jira/browse/HBASE-22015
Project: HBase
Issue Type: Sub-task
Reporter: Yi Mei
HBASE-11318 mark UserPermission as InterfaceAudience.Private.
HBASE-11452 instroduce AccessControlClient#getUserPermissions and return
UserPermission list but the UserPermission class is Private.
I also encounter the same problem when I want to move getUserPermissions method
as a admin api in HBASE-21911, otherwise the api of getUserPermissions may be
{code:java}
Map> getUserPermissions{code}
So shall we mark the UserPermission as Public? discussions are welcomed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[ https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787636#comment-16787636 ] Zheng Hu commented on HBASE-22005: -- Seems it will be better for others reviewing , if I divide this big patch into two seperate patches: 1. Rewrite the block reading methods by using hbase.nio.ByteBuff. Most of the work will be done in HFileBlock and the newly created BlockIOUtils. 2. Use ByteBuff's refcnt to track life cycle of data block in rpc path, we should consider each case in RPC path, whether the ByteBuff can release or not. Let me file an new JIRA. > Use ByteBuff's refcnt to track the life cycle of data block > --- > > Key: HBASE-22005 > URL: https://issues.apache.org/jira/browse/HBASE-22005 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22005.HBASE-21879.v1.patch, > HBASE-22005.HBASE-21879.v2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21977) Skip replay WAL and update seqid when open regions restored from snapshot
[
https://issues.apache.org/jira/browse/HBASE-21977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787632#comment-16787632
]
Hadoop QA commented on HBASE-21977:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
44s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
12s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
12m 13s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.4 or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}250m 36s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}303m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestFromClientSideWithCoprocessor |
| | hadoop.hbase.client.TestAdmin1 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21977 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961651/HBASE-21977.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c571a58fd607 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a7bbff170a |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/16298/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16298/testReport/ |
| Max. process+thread count | 4815 (vs. ulimit o
[jira] [Commented] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[ https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787628#comment-16787628 ] Zheng Hu commented on HBASE-22005: -- Uploaded patch.v2: add an check for FSDataInputStream, if no ByteBufferReable supported, then just read to heap and copy to hbase.nio.ByteBuff. > Use ByteBuff's refcnt to track the life cycle of data block > --- > > Key: HBASE-22005 > URL: https://issues.apache.org/jira/browse/HBASE-22005 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22005.HBASE-21879.v1.patch, > HBASE-22005.HBASE-21879.v2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[ https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-22005: - Attachment: HBASE-22005.HBASE-21879.v2.patch > Use ByteBuff's refcnt to track the life cycle of data block > --- > > Key: HBASE-22005 > URL: https://issues.apache.org/jira/browse/HBASE-22005 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22005.HBASE-21879.v1.patch, > HBASE-22005.HBASE-21879.v2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21736) TestHbck is flakey
[ https://issues.apache.org/jira/browse/HBASE-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21736: -- Assignee: Duo Zhang (was: Peter Somogyi) Status: Patch Available (was: Open) > TestHbck is flakey > -- > > Key: HBASE-21736 > URL: https://issues.apache.org/jira/browse/HBASE-21736 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Attachments: HBASE-21736.patch > > > It will hang on stopping region server > https://builds.apache.org/job/HBase-Flaky-Tests/job/master/2348/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestHbck-output.txt/*view*/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[ https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-22005: - Attachment: (was: HBASE-22005.HBASE-21879.v2.patch) > Use ByteBuff's refcnt to track the life cycle of data block > --- > > Key: HBASE-22005 > URL: https://issues.apache.org/jira/browse/HBASE-22005 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22005.HBASE-21879.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[ https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zheng Hu updated HBASE-22005: - Attachment: HBASE-22005.HBASE-21879.v2.patch > Use ByteBuff's refcnt to track the life cycle of data block > --- > > Key: HBASE-22005 > URL: https://issues.apache.org/jira/browse/HBASE-22005 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22005.HBASE-21879.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21736) TestHbck is flakey
[ https://issues.apache.org/jira/browse/HBASE-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21736: -- Attachment: HBASE-21736.patch > TestHbck is flakey > -- > > Key: HBASE-21736 > URL: https://issues.apache.org/jira/browse/HBASE-21736 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Peter Somogyi >Priority: Major > Attachments: HBASE-21736.patch > > > It will hang on stopping region server > https://builds.apache.org/job/HBase-Flaky-Tests/job/master/2348/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestHbck-output.txt/*view*/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21736) TestHbck is flakey
[ https://issues.apache.org/jira/browse/HBASE-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787617#comment-16787617 ] Duo Zhang commented on HBASE-21736: --- OK, we are assigning the regions to the dead server. Let me dig. > TestHbck is flakey > -- > > Key: HBASE-21736 > URL: https://issues.apache.org/jira/browse/HBASE-21736 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Peter Somogyi >Priority: Major > > It will hang on stopping region server > https://builds.apache.org/job/HBase-Flaky-Tests/job/master/2348/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestHbck-output.txt/*view*/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21736) TestHbck is flakey
[ https://issues.apache.org/jira/browse/HBASE-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787616#comment-16787616 ] Duo Zhang commented on HBASE-21736: --- OK, seems the SCP may hang there for a very long time. Let me see. > TestHbck is flakey > -- > > Key: HBASE-21736 > URL: https://issues.apache.org/jira/browse/HBASE-21736 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Peter Somogyi >Priority: Major > > It will hang on stopping region server > https://builds.apache.org/job/HBase-Flaky-Tests/job/master/2348/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestHbck-output.txt/*view*/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21990) puppycrawl checkstyle dtds 404... moved to sourceforge
[
https://issues.apache.org/jira/browse/HBASE-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787599#comment-16787599
]
stack commented on HBASE-21990:
---
I left off a closing '>'. Pushed ANOTHER addendum on branch-2.0 only to see how
it does.
{code}
diff --git
a/hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml
b/hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml
index b66b4d5972..ec805405eb 100644
--- a/hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml
+++ b/hbase-checkstyle/src/main/resources/hbase/checkstyle-suppressions.xml
@@ -1,7 +1,7 @@
https://checkstyle.org/dtds/suppressions_1_2.dtd";
+ "https://checkstyle.org/dtds/suppressions_1_2.dtd"; >
> Key: HBASE-21990
> URL: https://issues.apache.org/jira/browse/HBASE-21990
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.5, 1.3.4, 2.1.4, 1.5.1, 1.2.12
>
> Attachments: HBASE-21990.branch-2.0.001.patch,
> HBASE-21990.branch-2.0.002.patch
>
>
> Yetus flagging two badly formatted xml files. Seemingly related are the two
> filenotfoundexceptions that show in the xml.txt build product.
> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2.0/1399/artifact/output-general/xml.txt
> The complaint is that the xml dtds are not found.
> Indeed they were moved long time ago:
> https://github.com/checkstyle/checkstyle/issues/1571
> They were supposed to stick around at their old location but that seems to
> have changed recently.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21736) TestHbck is flakey
[ https://issues.apache.org/jira/browse/HBASE-21736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787601#comment-16787601 ] Duo Zhang commented on HBASE-21736: --- OK, I think the problem is that, we kill a region server by scheduling a SCP, and then without waiting for it actually down, we start the next test, where will schedule TRSP for the regions on the dead server, and for hbck, we will skip the normal fencing code, then it will cause strange problems... Let me provide a patch here to see if it helps. > TestHbck is flakey > -- > > Key: HBASE-21736 > URL: https://issues.apache.org/jira/browse/HBASE-21736 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Peter Somogyi >Priority: Major > > It will hang on stopping region server > https://builds.apache.org/job/HBase-Flaky-Tests/job/master/2348/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestHbck-output.txt/*view*/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[
https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787596#comment-16787596
]
Hudson commented on HBASE-21999:
Results for branch branch-2.1
[build #933 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/933/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/933//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/933//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/933//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/933//console].
> [DEBUG] Exit if git returns empty revision!
> ---
>
> Key: HBASE-21999
> URL: https://issues.apache.org/jira/browse/HBASE-21999
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1
>
> Attachments: HBASE-21999-addendum-v0.patch,
> HBASE-21999-addendum-v1.patch
>
>
> Let me commit a bit of debug on branch-2.0. Can't figure out how we are
> getting empty git revision string. Have the build fail if empty
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[
https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787581#comment-16787581
]
Hudson commented on HBASE-21999:
Results for branch master
[build #847 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/847/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/847//console].
> [DEBUG] Exit if git returns empty revision!
> ---
>
> Key: HBASE-21999
> URL: https://issues.apache.org/jira/browse/HBASE-21999
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1
>
> Attachments: HBASE-21999-addendum-v0.patch,
> HBASE-21999-addendum-v1.patch
>
>
> Let me commit a bit of debug on branch-2.0. Can't figure out how we are
> getting empty git revision string. Have the build fail if empty
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[
https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787591#comment-16787591
]
Hudson commented on HBASE-21999:
Results for branch branch-2.2
[build #93 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/93/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/93//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/93//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/93//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/93//console].
> [DEBUG] Exit if git returns empty revision!
> ---
>
> Key: HBASE-21999
> URL: https://issues.apache.org/jira/browse/HBASE-21999
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1
>
> Attachments: HBASE-21999-addendum-v0.patch,
> HBASE-21999-addendum-v1.patch
>
>
> Let me commit a bit of debug on branch-2.0. Can't figure out how we are
> getting empty git revision string. Have the build fail if empty
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22007) Add restoreSnapshot and cloneSnapshot with acl methods in AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-22007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787589#comment-16787589 ] Zheng Hu commented on HBASE-22007: -- +1, The abstraction of SnapshotWithAclTestBase looks good, so that we can test both the sync api and async api. > Add restoreSnapshot and cloneSnapshot with acl methods in AsyncAdmin > > > Key: HBASE-22007 > URL: https://issues.apache.org/jira/browse/HBASE-22007 > Project: HBase > Issue Type: Task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-22007-v1.patch, HBASE-22007-v1.patch, > HBASE-22007.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22006) Fix branch-2.1 findbugs warning; causes nightly show as failed.
[
https://issues.apache.org/jira/browse/HBASE-22006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787579#comment-16787579
]
Hudson commented on HBASE-22006:
Results for branch master
[build #847 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/847/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/847//console].
> Fix branch-2.1 findbugs warning; causes nightly show as failed.
> ---
>
> Key: HBASE-22006
> URL: https://issues.apache.org/jira/browse/HBASE-22006
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.0.5, 2.1.5, 2.2.1
>
>
> Seems like an old bit of code but its causing warning:
> Dodgy code Warnings
> Code Warning
> DLS Dead store to $L8 in
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeMemStoreChunkCreator()
> Bug type DLS_DEAD_LOCAL_STORE (click for details)
> In class org.apache.hadoop.hbase.regionserver.HRegionServer
> In method
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeMemStoreChunkCreator()
> Local variable stored in JVM register 8
> At HRegionServer.java:[line 1593]
> This seems to be only thing between 2.1 being a blue run in last few
> instances. Let me fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-20918) Re-enable TestRpcHandlerException
[
https://issues.apache.org/jira/browse/HBASE-20918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787580#comment-16787580
]
Hudson commented on HBASE-20918:
Results for branch master
[build #847 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/847/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/847//console].
> Re-enable TestRpcHandlerException
> --
>
> Key: HBASE-20918
> URL: https://issues.apache.org/jira/browse/HBASE-20918
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Jack Bearden
>Assignee: Jack Bearden
>Priority: Minor
> Attachments: HBASE-20918.001.patch
>
>
> Work done to re-enable this test as part of the HBASE-20266 review.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22001) Polish the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-22001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787582#comment-16787582 ] Duo Zhang commented on HBASE-22001: --- Any other concerns? [~stack] Thanks. > Polish the Admin interface > -- > > Key: HBASE-22001 > URL: https://issues.apache.org/jira/browse/HBASE-22001 > Project: HBase > Issue Type: Task > Components: Admin, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-22001-v1.patch, HBASE-22001-v2.patch, > HBASE-22001-v3.patch, HBASE-22001-v3.patch, HBASE-22001-v4.patch, > HBASE-22001.patch > > > The snapshot related methods are not well declared, we missed several methods > which has the restoreAcl parameter. And also, the snapshotAsync method > returns nothing, which is bit strange. > And we can use default methods to reduce the code in HBaseAdmin and also the > new Admin implementation in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[
https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787578#comment-16787578
]
Hudson commented on HBASE-21874:
Results for branch master
[build #847 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/847/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/847//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/master/847//console].
> Bucket cache on Persistent memory
> -
>
> Key: HBASE-21874
> URL: https://issues.apache.org/jira/browse/HBASE-21874
> Project: HBase
> Issue Type: New Feature
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21874.patch, HBASE-21874.patch,
> HBASE-21874_V2.patch, HBASE-21874_V4.patch, HBASE-21874_V5.patch,
> HBASE-21874_V6.patch, Pmem_BC.png
>
>
> Non volatile persistent memory devices are byte addressable like DRAM (for
> eg. Intel DCPMM). Bucket cache implementation can take advantage of this new
> memory type and can make use of the existing offheap data structures to serve
> data directly from this memory area without having to bring the data to
> onheap.
> The patch is a new IOEngine implementation that works with the persistent
> memory.
> Note : Here we don't make use of the persistence nature of the device and
> just make use of the big memory it provides.
> Performance numbers to follow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22007) Add restoreSnapshot and cloneSnapshot with acl methods in AsyncAdmin
[
https://issues.apache.org/jira/browse/HBASE-22007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787573#comment-16787573
]
Hadoop QA commented on HBASE-22007:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
15s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} hbase-client: The patch generated 0 new + 26
unchanged - 2 fixed = 26 total (was 28) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} The patch passed checkstyle in hbase-server {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
10s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 4s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
20s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}131m
8s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
47s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}178m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22007 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961654/HBASE-22007-v1.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux cdadfe42e292 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-
[jira] [Commented] (HBASE-22010) docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
[
https://issues.apache.org/jira/browse/HBASE-22010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787572#comment-16787572
]
Guanghao Zhang commented on HBASE-22010:
+1
> docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
> ---
>
> Key: HBASE-22010
> URL: https://issues.apache.org/jira/browse/HBASE-22010
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-22010.0.patch
>
>
> heading doesn't work, which also means no linking to it.
> the rendered page has an unrendered bit, e.g.
> {code}
> [[upgrade 2.2]] === Upgrade from 2.0 or 2.1 to 2.2+
> HBase 2.2+ uses a new Procedure form assiging/unassigning/moving Regions. It
> does not process HBase 2.1 and 2.0’s Unassign/Assign Procedure types. Upgrade
> requires that we first drain the Master Procedure Store of old style
> Procedures before starting the new 2.2 Master. So you need to make sure that
> before you kill the old version (2.0 or 2.1) Master, there is no region in
> transition. And once the new version (2.2+) Master is up, you can rolling
> upgrade RegionServers one by one.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-22014) Space Quota: If some reasons are offline then quota usage calculation is not happening.
Ajeet Rai created HBASE-22014: - Summary: Space Quota: If some reasons are offline then quota usage calculation is not happening. Key: HBASE-22014 URL: https://issues.apache.org/jira/browse/HBASE-22014 Project: HBase Issue Type: Bug Reporter: Ajeet Rai Space Quota: If some reasons are offline then quota usage calculation is not happening. Steps: Scenario1: 1: create 'testmultireason','info0',SPLITS => ['100','200','300','400','500','600','700','800','900','1000'] 2: unassign '3f356ccac99fba4e12741a57f79a18a7' // close one of the region 3: set_quota TYPE => SPACE, TABLE => 'testmultireason', LIMIT => '3M', POLICY => NO_INSERTS 4: put some data 5: observe that after some time data usage and state is also shown but usage is 0 only even if data has crossed quota limit Scenario2: 1: set_quota TYPE => SPACE, TABLE => 'testmultireason2', LIMIT => '3M', POLICY => NO_INSERTS 2: create 'testmultireason2','info0',SPLITS => ['100','200','300','400','500','600','700','800','900','1000'] 3: unassign '3b6e28e0060596e35814673ef4941d4c' //close one of the region 4: put 4 mb data and observe that policy is in violation 5: Observe after some time policy is changed from violation to In Observance -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22013) Space Quota: Space Quota Issue: If a table is created with region replica then quota calculation is not happening
Ajeet Rai created HBASE-22013: - Summary: Space Quota: Space Quota Issue: If a table is created with region replica then quota calculation is not happening Key: HBASE-22013 URL: https://issues.apache.org/jira/browse/HBASE-22013 Project: HBase Issue Type: Bug Reporter: Ajeet Rai Space Quota: Space Quota Issue: If a table is created with region replica then quota calculation is not happening Steps: 1: Create a table with 100 regions with region replica 3 2: Observe that 'hbase:quota' table doesn't have entry of usage for this table So In UI only policy Limit and Policy is shown but not Usage and State. Reason: It looks like File system utilization core is sending data of 100 reasons but not the size of region replicas. But in quota observer chore, it is considering total region(actual regions+ replica reasons) So the ratio of reported regions is less then configured percentRegionsReportedThreshold. SO quota calculation is not happening -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-22012) Space Quota: Policy state is getting changed from disable to Observance after sometime automatically.
Ajeet Rai created HBASE-22012: - Summary: Space Quota: Policy state is getting changed from disable to Observance after sometime automatically. Key: HBASE-22012 URL: https://issues.apache.org/jira/browse/HBASE-22012 Project: HBase Issue Type: Bug Reporter: Ajeet Rai pace Quota: Policy state is getting changed from disable to Observance after sometime automatically. Steps: 1: Create a table with space quota policy as Disable 2: Put some data so that table state is in space quota violation 3: So observe that table state is in violation 4: Now wait for some time 5: Observe that after some time table state is changing to to Observance however table is still disabled -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21990) puppycrawl checkstyle dtds 404... moved to sourceforge
[
https://issues.apache.org/jira/browse/HBASE-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787522#comment-16787522
]
Hudson commented on HBASE-21990:
Results for branch branch-2.0
[build #1417 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1417/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1417//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1417//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1417//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> puppycrawl checkstyle dtds 404... moved to sourceforge
> --
>
> Key: HBASE-21990
> URL: https://issues.apache.org/jira/browse/HBASE-21990
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.5, 1.3.4, 2.1.4, 1.5.1, 1.2.12
>
> Attachments: HBASE-21990.branch-2.0.001.patch,
> HBASE-21990.branch-2.0.002.patch
>
>
> Yetus flagging two badly formatted xml files. Seemingly related are the two
> filenotfoundexceptions that show in the xml.txt build product.
> https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2.0/1399/artifact/output-general/xml.txt
> The complaint is that the xml dtds are not found.
> Indeed they were moved long time ago:
> https://github.com/checkstyle/checkstyle/issues/1571
> They were supposed to stick around at their old location but that seems to
> have changed recently.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[
https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787521#comment-16787521
]
Hudson commented on HBASE-21999:
Results for branch branch-2.0
[build #1417 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1417/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1417//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1417//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1417//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> [DEBUG] Exit if git returns empty revision!
> ---
>
> Key: HBASE-21999
> URL: https://issues.apache.org/jira/browse/HBASE-21999
> Project: HBase
> Issue Type: Sub-task
> Components: build
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1
>
> Attachments: HBASE-21999-addendum-v0.patch,
> HBASE-21999-addendum-v1.patch
>
>
> Let me commit a bit of debug on branch-2.0. Can't figure out how we are
> getting empty git revision string. Have the build fail if empty
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21991) Fix MetaMetrics issues - [Race condition, Faulty remove logic], few improvements
[ https://issues.apache.org/jira/browse/HBASE-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sakthi updated HBASE-21991: --- Attachment: hbase-21991.master.002.patch > Fix MetaMetrics issues - [Race condition, Faulty remove logic], few > improvements > > > Key: HBASE-21991 > URL: https://issues.apache.org/jira/browse/HBASE-21991 > Project: HBase > Issue Type: Bug > Components: Coprocessors, metrics >Reporter: Sakthi >Assignee: Sakthi >Priority: Major > Attachments: hbase-21991.master.001.patch, > hbase-21991.master.002.patch > > > Here is a list of the issues related to the MetaMetrics implementation: > +*Bugs*+: > # [_Lossy counting for top-k_] *Faulty remove logic of non-eligible meters*: > Under certain conditions, we might end up storing/exposing all the meters > rather than top-k-ish > # MetaMetrics can throw NPE resulting in aborting of the RS because of a > *Race Condition*. > +*Improvements*+: > # With high number of regions in the cluster, exposure of metrics for each > region blows up the JMX from ~140 Kbs to 100+ Mbs depending on the number of > regions. It's better to use *lossy counting to maintain top-k for region > metrics* as well. > # As the lossy meters do not represent actual counts, I think, it'll be > better to *rename the meters to include "lossy" in the name*. It would be > more informative while monitoring the metrics and there would be less > confusion regarding actual counts to lossy counts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21991) Fix MetaMetrics issues - [Race condition, Faulty remove logic], few improvements
[ https://issues.apache.org/jira/browse/HBASE-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787510#comment-16787510 ] Sakthi commented on HBASE-21991: Have updated the patch with your suggestions and few modifications. > Fix MetaMetrics issues - [Race condition, Faulty remove logic], few > improvements > > > Key: HBASE-21991 > URL: https://issues.apache.org/jira/browse/HBASE-21991 > Project: HBase > Issue Type: Bug > Components: Coprocessors, metrics >Reporter: Sakthi >Assignee: Sakthi >Priority: Major > Attachments: hbase-21991.master.001.patch, > hbase-21991.master.002.patch > > > Here is a list of the issues related to the MetaMetrics implementation: > +*Bugs*+: > # [_Lossy counting for top-k_] *Faulty remove logic of non-eligible meters*: > Under certain conditions, we might end up storing/exposing all the meters > rather than top-k-ish > # MetaMetrics can throw NPE resulting in aborting of the RS because of a > *Race Condition*. > +*Improvements*+: > # With high number of regions in the cluster, exposure of metrics for each > region blows up the JMX from ~140 Kbs to 100+ Mbs depending on the number of > regions. It's better to use *lossy counting to maintain top-k for region > metrics* as well. > # As the lossy meters do not represent actual counts, I think, it'll be > better to *rename the meters to include "lossy" in the name*. It would be > more informative while monitoring the metrics and there would be less > confusion regarding actual counts to lossy counts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21994) Expose TableDescriptors to master coprocessor
[ https://issues.apache.org/jira/browse/HBASE-21994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-21994: -- Attachment: HBASE-21994-v1.patch > Expose TableDescriptors to master coprocessor > - > > Key: HBASE-21994 > URL: https://issues.apache.org/jira/browse/HBASE-21994 > Project: HBase > Issue Type: Improvement >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-21994-v1.patch, HBASE-21994.patch > > > When implementing HBASE-21717, I found that AccessController will use the > Admin interface to get table descriptors on master. We can expose the read > only methods in TableDescritors to CPs so user can get the table descriptor > directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22011) ThriftUtilities.getFromThrift should set filter when not set columns
[
https://issues.apache.org/jira/browse/HBASE-22011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allan Yang updated HBASE-22011:
---
Affects Version/s: 2.2.0
> ThriftUtilities.getFromThrift should set filter when not set columns
>
>
> Key: HBASE-22011
> URL: https://issues.apache.org/jira/browse/HBASE-22011
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Bing Xiao
>Priority: Major
> Fix For: 2.2.1
>
>
> ThriftUtilities.getFromThrift, if TGet wihtout columns the filter is ignore.
> {code:java}
> if (!in.isSetColumns()) {
> return out;
> }
> for (TColumn column : in.getColumns()) {
> if (column.isSetQualifier()) {
> out.addColumn(column.getFamily(), column.getQualifier());
> } else {
> out.addFamily(column.getFamily());
> }
> }
> if (in.isSetFilterBytes()) {
> out.setFilter(filterFromThrift(in.getFilterBytes()));
> }{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22010) docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
[
https://issues.apache.org/jira/browse/HBASE-22010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787496#comment-16787496
]
Hadoop QA commented on HBASE-22010:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
51s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
24s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 15m
52s{color} | {color:blue} branch has no errors when building the reference
guide. See footer for rendered docs, which you should manually inspect. {color}
|
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:blue}0{color} | {color:blue} refguide {color} | {color:blue} 5m
0s{color} | {color:blue} patch has no errors when building the reference guide.
See footer for rendered docs, which you should manually inspect. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22010 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961655/HBASE-22010.0.patch |
| Optional Tests | dupname asflicense refguide |
| uname | Linux c959b371b643 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a7bbff170a |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/16300/artifact/patchprocess/branch-site/book.html
|
| refguide |
https://builds.apache.org/job/PreCommit-HBASE-Build/16300/artifact/patchprocess/patch-site/book.html
|
| Max. process+thread count | 97 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16300/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
> ---
>
> Key: HBASE-22010
> URL: https://issues.apache.org/jira/browse/HBASE-22010
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-22010.0.patch
>
>
> heading doesn't work, which also means no linking to it.
> the rendered page has an unrendered bit, e.g.
> {code}
> [[upgrade 2.2]] === Upgrade from 2.0 or 2.1 to 2.2+
> HBase 2.2+ uses a new Procedure form assiging/unassigning/moving Regions. It
> does not process HBase 2.1 and 2.0’s Unassign/Assign Procedure types. Upgrade
> requires that we first drain the Master Procedure Store of old style
> Procedures before starting the new 2.2 Master. So you need to make sure that
> before you kill the old version (2.0 or 2.1) Master, there is no region in
> transition. And once the new version (2.2+) Master is up, you can rolling
> upgrade RegionServers one by one.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22010) docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
[
https://issues.apache.org/jira/browse/HBASE-22010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-22010:
Attachment: HBASE-22010.0.patch
> docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
> ---
>
> Key: HBASE-22010
> URL: https://issues.apache.org/jira/browse/HBASE-22010
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-22010.0.patch
>
>
> heading doesn't work, which also means no linking to it.
> the rendered page has an unrendered bit, e.g.
> {code}
> [[upgrade 2.2]] === Upgrade from 2.0 or 2.1 to 2.2+
> HBase 2.2+ uses a new Procedure form assiging/unassigning/moving Regions. It
> does not process HBase 2.1 and 2.0’s Unassign/Assign Procedure types. Upgrade
> requires that we first drain the Master Procedure Store of old style
> Procedures before starting the new 2.2 Master. So you need to make sure that
> before you kill the old version (2.0 or 2.1) Master, there is no region in
> transition. And once the new version (2.2+) Master is up, you can rolling
> upgrade RegionServers one by one.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-22011) ThriftUtilities.getFromThrift should set filter when not set columns
Bing Xiao created HBASE-22011:
-
Summary: ThriftUtilities.getFromThrift should set filter when not
set columns
Key: HBASE-22011
URL: https://issues.apache.org/jira/browse/HBASE-22011
Project: HBase
Issue Type: Bug
Reporter: Bing Xiao
Fix For: 2.2.1
ThriftUtilities.getFromThrift, if TGet wihtout columns the filter is ignore.
{code:java}
if (!in.isSetColumns()) {
return out;
}
for (TColumn column : in.getColumns()) {
if (column.isSetQualifier()) {
out.addColumn(column.getFamily(), column.getQualifier());
} else {
out.addFamily(column.getFamily());
}
}
if (in.isSetFilterBytes()) {
out.setFilter(filterFromThrift(in.getFilterBytes()));
}{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22010) docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
[
https://issues.apache.org/jira/browse/HBASE-22010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-22010:
Status: Patch Available (was: In Progress)
v0
- remove the space from the heading link for upgrade to 2.2.
> docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
> ---
>
> Key: HBASE-22010
> URL: https://issues.apache.org/jira/browse/HBASE-22010
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
> Attachments: HBASE-22010.0.patch
>
>
> heading doesn't work, which also means no linking to it.
> the rendered page has an unrendered bit, e.g.
> {code}
> [[upgrade 2.2]] === Upgrade from 2.0 or 2.1 to 2.2+
> HBase 2.2+ uses a new Procedure form assiging/unassigning/moving Regions. It
> does not process HBase 2.1 and 2.0’s Unassign/Assign Procedure types. Upgrade
> requires that we first drain the Master Procedure Store of old style
> Procedures before starting the new 2.2 Master. So you need to make sure that
> before you kill the old version (2.0 or 2.1) Master, there is no region in
> transition. And once the new version (2.2+) Master is up, you can rolling
> upgrade RegionServers one by one.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22007) Add restoreSnapshot and cloneSnapshot with acl methods in AsyncAdmin
[ https://issues.apache.org/jira/browse/HBASE-22007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22007: -- Attachment: HBASE-22007-v1.patch > Add restoreSnapshot and cloneSnapshot with acl methods in AsyncAdmin > > > Key: HBASE-22007 > URL: https://issues.apache.org/jira/browse/HBASE-22007 > Project: HBase > Issue Type: Task > Components: Admin, asyncclient, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-22007-v1.patch, HBASE-22007-v1.patch, > HBASE-22007.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21977) Skip replay WAL and update seqid when open regions restored from snapshot
[ https://issues.apache.org/jira/browse/HBASE-21977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21977: --- Attachment: HBASE-21977.master.003.patch > Skip replay WAL and update seqid when open regions restored from snapshot > - > > Key: HBASE-21977 > URL: https://issues.apache.org/jira/browse/HBASE-21977 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21977.master.001.patch, > HBASE-21977.master.002.patch, HBASE-21977.master.002.patch, > HBASE-21977.master.003.patch > > > TableSnapshotScanner restore a snapshot and then open the restored regions. > When open these regions, we can skip replay WAL and update seqid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22001) Polish the Admin interface
[ https://issues.apache.org/jira/browse/HBASE-22001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang updated HBASE-22001: -- Attachment: HBASE-22001-v4.patch > Polish the Admin interface > -- > > Key: HBASE-22001 > URL: https://issues.apache.org/jira/browse/HBASE-22001 > Project: HBase > Issue Type: Task > Components: Admin, Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: HBASE-22001-v1.patch, HBASE-22001-v2.patch, > HBASE-22001-v3.patch, HBASE-22001-v3.patch, HBASE-22001-v4.patch, > HBASE-22001.patch > > > The snapshot related methods are not well declared, we missed several methods > which has the restoreAcl parameter. And also, the snapshotAsync method > returns nothing, which is bit strange. > And we can use default methods to reduce the code in HBaseAdmin and also the > new Admin implementation in the future. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[ https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787473#comment-16787473 ] Duo Zhang commented on HBASE-22005: --- And [~ste...@apache.org], the point here is not about streaming API, we just want to use (Direct)ByteBuffer instead of byte[] so we can avoid copying data on heap. And for the high speed new hardware(NVMe, 3DXPoint), it is mmapped, and usually we will expose a MappedByteBuffer in the java API, which is already enough for us here I believe? And for high latency object storage, such as S3, I do not see any difference between passing a and ? Anyway, we need the StreamCapabilities API to query whether a given stream has the ability, sadly it is only provided in hadoop-2.9+ and we still need to support 2.7... > Use ByteBuff's refcnt to track the life cycle of data block > --- > > Key: HBASE-22005 > URL: https://issues.apache.org/jira/browse/HBASE-22005 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22005.HBASE-21879.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21991) Fix MetaMetrics issues - [Race condition, Faulty remove logic], few improvements
[
https://issues.apache.org/jira/browse/HBASE-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787455#comment-16787455
]
Sakthi edited comment on HBASE-21991 at 3/8/19 2:27 AM:
[~xucang], Please ignore my above comment. I analyzed the possibilities a
little more. Here's my take.
There are 2 possible code flows where we are modifying the state of the class.
1:
{code:java}
// This is for the non top-k metrics
private void registerAndMarkMeterIfNotPresent(String name) {
registerMeterIfNotPresent(name);
markMeterIfPresent(name);
}
{code}
Here registerMeterIfNotPresent() does only puts. Registers in the registry and
accordingly updates in the map. As the 'registry' takes care of multithreaded
access by using 'computeIfAbsent' to put new meter, we are good here. So no
duplicates in the registry or the map as we use registry.get to put in the map.
And, markMeterIfPresent() does only get & update. As, the metric.mark() looks
like an atomic operation (uses Atomiclong inside), we look safe here from
loosing updates.
2:
{code:java}
// This is for the top-k metrics
private void "topK"MetricRegisterAndMark()(String name) {
registerLossyCountingMeterIfNotPresent(meter, LossyCounting);
markMeterIfPresent(meter);
}
{code}
Here, we always need to maintain that the internal dataMap of the LossyCounting
class exactly is equivalent to what we have in our requestsMap. So all the
access to a particular lossyCounter and it's subsequent effect on the
requestsMap & registry must be synchronized. Or else, we can end up having a
metric in the registry that actually isn't in the internal dataMap of the
lossyCounter for example like below:
Lossy Counter referred below as *LC.*
*Thread-1*: Add meter 'm'
*LC* [by Thread-1]: Added 'm' to the internal 'dataMap'. Add it to the registry
*Thread-2*: Add meter 'm'
*LC* [by Thread-2]: Increment 'm' count in the 'dataMap'. Swept off. Removed
'm' from the 'dataMap'. So, remove it from the registry.
*Thread-2*: A new meter, being removed. So don't add it in the registry and
remove the remaining meters from the list given by *LC*;
*Thread-1*: Add the meter to the registry as directed by the *LC*
*Voila!* LC doesn't have 'm'. But registry has the 'm'.
*Suggestion:* We don't need synchronization anywhere else in the class except
in the below function:
{code:java}
private void registerLossyCountingMeterIfNotPresent(String requestMeter,
LossyCounting lossyCounting) {
...
synchronized(lossyCounting) {
...
}
}
{code}
This way, even addition of a new lossyCounter like for the region-metrics too,
wouldn't be an issue. Your thoughts? :)
was (Author: jatsakthi):
[~xucang], Please ignore my above comment. I analyzed the possibilities a
little more. Here's my take.
There are 2 possible code flows where we are modifying the state of the class.
1:
{code:java}
// This is for the non top-k metrics
private void registerAndMarkMeterIfNotPresent(String name) {
registerMeterIfNotPresent(name);
markMeterIfPresent(name);
}
{code}
Here registerMeterIfNotPresent() does only puts. Registers in the registry and
accordingly updates in the map. As the 'registry' takes care of multithreaded
access by using 'computeIfAbsent' to put new meter, we are good here. So no
duplicates in the registry or the map as we use registry.get to put in the map.
And, markMeterIfPresent() does only get & update. As, the metric.mark() looks
like an atomic operation (uses Atomiclong inside), we look safe here from
loosing updates.
2:
{code:java}
// This is for the top-k metrics
private void "topK"MetricRegisterAndMark()(String name) {
registerLossyCountingMeterIfNotPresent(meter, LossyCounting);
markMeterIfPresent(meter);
}
{code}
Here, we always need to maintain that the internal dataMap of the LossyCounting
class exactly is equivalent to what we have in our requestsMap. So all the
access to a particular lossyCounter and it's subsequent effect on the
requestsMap & registry must be synchronized. Or else, we can end up having a
metric in the registry that actually isn't in the internal dataMap of the
lossyCounter for example like below:
Lossy Counter referred below as *LC.*
*Thread-1*: Add meter 'm'
*LC* [by Thread-2]**: Added 'm' to the internal 'dataMap'. Add it to the
registry
*Thread-2*: Add meter 'm'
*LC* [by Thread-2]: Increment 'm' count in the 'dataMap'. Swept off. Removed
'm' from the 'dataMap'. So, remove it from the registry.
*Thread-2*: A new meter, being removed. So don't add it in the registry and
remove the remaining meters from the list given by *LC*;
*Thread-1*: Add the meter to the registry as directed by the *LC*
*Voila!* LC doesn't have 'm'. But registry has the 'm'.
*Suggestion:* We don't need synchronization anywhere else in the class except
in the below function:
{cod
[jira] [Comment Edited] (HBASE-21991) Fix MetaMetrics issues - [Race condition, Faulty remove logic], few improvements
[
https://issues.apache.org/jira/browse/HBASE-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787455#comment-16787455
]
Sakthi edited comment on HBASE-21991 at 3/8/19 2:28 AM:
[~xucang], Please ignore my above comment. I analyzed the possibilities a
little more. Here's my take.
There are 2 possible code flows where we are modifying the state of the class.
1:
{code:java}
// This is for the non top-k metrics
private void registerAndMarkMeterIfNotPresent(String name) {
registerMeterIfNotPresent(name);
markMeterIfPresent(name);
}
{code}
Here registerMeterIfNotPresent() does only puts. Registers in the registry and
accordingly updates in the map. As the 'registry' takes care of multithreaded
access by using 'computeIfAbsent' to put new meter, we are good here. So no
duplicates in the registry or the map as we use registry.get to put in the map.
And, markMeterIfPresent() does only get & update. As, the metric.mark() looks
like an atomic operation (uses Atomiclong inside), we look safe here from
loosing updates.
2:
{code:java}
// This is for the top-k metrics
private void "topK"MetricRegisterAndMark()(String name) {
registerLossyCountingMeterIfNotPresent(meter, LossyCounting);
markMeterIfPresent(meter);
}
{code}
Here, we always need to maintain that the internal dataMap of the LossyCounting
class exactly is equivalent to what we have in our requestsMap. So all the
access to a particular lossyCounter and it's subsequent effect on the
requestsMap & registry must be synchronized. Or else, we can end up having a
metric in the registry that actually isn't in the internal dataMap of the
lossyCounter for example like below:
Lossy Counter referred below as *LC.*
*Thread-1*: Add meter 'm'
*LC* [by Thread-1]: Added 'm' to the internal 'dataMap'. No data swept off. So,
add it to the registry
*Thread-2*: Add meter 'm'
*LC* [by Thread-2]: Increment 'm' count in the 'dataMap'. Swept off. Removed
'm' from the 'dataMap'. So, remove it from the registry.
*Thread-2*: A new meter, being removed. So don't add it in the registry and
remove the remaining meters from the list given by *LC*;
*Thread-1*: Add the meter to the registry as directed by the *LC*
*Voila!* LC doesn't have 'm'. But registry has the 'm'.
*Suggestion:* We don't need synchronization anywhere else in the class except
in the below function:
{code:java}
private void registerLossyCountingMeterIfNotPresent(String requestMeter,
LossyCounting lossyCounting) {
...
synchronized(lossyCounting) {
...
}
}
{code}
This way, even addition of a new lossyCounter like for the region-metrics too,
wouldn't be an issue. Your thoughts? :)
was (Author: jatsakthi):
[~xucang], Please ignore my above comment. I analyzed the possibilities a
little more. Here's my take.
There are 2 possible code flows where we are modifying the state of the class.
1:
{code:java}
// This is for the non top-k metrics
private void registerAndMarkMeterIfNotPresent(String name) {
registerMeterIfNotPresent(name);
markMeterIfPresent(name);
}
{code}
Here registerMeterIfNotPresent() does only puts. Registers in the registry and
accordingly updates in the map. As the 'registry' takes care of multithreaded
access by using 'computeIfAbsent' to put new meter, we are good here. So no
duplicates in the registry or the map as we use registry.get to put in the map.
And, markMeterIfPresent() does only get & update. As, the metric.mark() looks
like an atomic operation (uses Atomiclong inside), we look safe here from
loosing updates.
2:
{code:java}
// This is for the top-k metrics
private void "topK"MetricRegisterAndMark()(String name) {
registerLossyCountingMeterIfNotPresent(meter, LossyCounting);
markMeterIfPresent(meter);
}
{code}
Here, we always need to maintain that the internal dataMap of the LossyCounting
class exactly is equivalent to what we have in our requestsMap. So all the
access to a particular lossyCounter and it's subsequent effect on the
requestsMap & registry must be synchronized. Or else, we can end up having a
metric in the registry that actually isn't in the internal dataMap of the
lossyCounter for example like below:
Lossy Counter referred below as *LC.*
*Thread-1*: Add meter 'm'
*LC* [by Thread-1]: Added 'm' to the internal 'dataMap'. Add it to the registry
*Thread-2*: Add meter 'm'
*LC* [by Thread-2]: Increment 'm' count in the 'dataMap'. Swept off. Removed
'm' from the 'dataMap'. So, remove it from the registry.
*Thread-2*: A new meter, being removed. So don't add it in the registry and
remove the remaining meters from the list given by *LC*;
*Thread-1*: Add the meter to the registry as directed by the *LC*
*Voila!* LC doesn't have 'm'. But registry has the 'm'.
*Suggestion:* We don't need synchronization anywhere else in the class except
in the
[jira] [Commented] (HBASE-22010) docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
[
https://issues.apache.org/jira/browse/HBASE-22010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787459#comment-16787459
]
Sean Busbey commented on HBASE-22010:
-
no worries, it happens. the precommit bot will render it for you as well.
> docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
> ---
>
> Key: HBASE-22010
> URL: https://issues.apache.org/jira/browse/HBASE-22010
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
>
> heading doesn't work, which also means no linking to it.
> the rendered page has an unrendered bit, e.g.
> {code}
> [[upgrade 2.2]] === Upgrade from 2.0 or 2.1 to 2.2+
> HBase 2.2+ uses a new Procedure form assiging/unassigning/moving Regions. It
> does not process HBase 2.1 and 2.0’s Unassign/Assign Procedure types. Upgrade
> requires that we first drain the Master Procedure Store of old style
> Procedures before starting the new 2.2 Master. So you need to make sure that
> before you kill the old version (2.0 or 2.1) Master, there is no region in
> transition. And once the new version (2.2+) Master is up, you can rolling
> upgrade RegionServers one by one.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-21977) Skip replay WAL and update seqid when open regions restored from snapshot
[ https://issues.apache.org/jira/browse/HBASE-21977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Mei updated HBASE-21977: --- Attachment: (was: HBASE-21977.master.003.patch) > Skip replay WAL and update seqid when open regions restored from snapshot > - > > Key: HBASE-21977 > URL: https://issues.apache.org/jira/browse/HBASE-21977 > Project: HBase > Issue Type: Sub-task >Reporter: Yi Mei >Assignee: Yi Mei >Priority: Major > Attachments: HBASE-21977.master.001.patch, > HBASE-21977.master.002.patch, HBASE-21977.master.002.patch, > HBASE-21977.master.003.patch > > > TableSnapshotScanner restore a snapshot and then open the restored regions. > When open these regions, we can skip replay WAL and update seqid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[ https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787457#comment-16787457 ] Duo Zhang commented on HBASE-22005: --- [~openinx] I think we could introduce the method in HBase, in the CommonFSUtils. We have done a lot of dirty works there. > Use ByteBuff's refcnt to track the life cycle of data block > --- > > Key: HBASE-22005 > URL: https://issues.apache.org/jira/browse/HBASE-22005 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22005.HBASE-21879.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21991) Fix MetaMetrics issues - [Race condition, Faulty remove logic], few improvements
[
https://issues.apache.org/jira/browse/HBASE-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787455#comment-16787455
]
Sakthi commented on HBASE-21991:
[~xucang], Please ignore my above comment. I analyzed the possibilities a
little more. Here's my take.
There are 2 possible code flows where we are modifying the state of the class.
1:
{code:java}
// This is for the non top-k metrics
private void registerAndMarkMeterIfNotPresent(String name) {
registerMeterIfNotPresent(name);
markMeterIfPresent(name);
}
{code}
Here registerMeterIfNotPresent() does only puts. Registers in the registry and
accordingly updates in the map. As the 'registry' takes care of multithreaded
access by using 'computeIfAbsent' to put new meter, we are good here. So no
duplicates in the registry or the map as we use registry.get to put in the map.
And, markMeterIfPresent() does only get & update. As, the metric.mark() looks
like an atomic operation (uses Atomiclong inside), we look safe here from
loosing updates.
2:
{code:java}
// This is for the top-k metrics
private void "topK"MetricRegisterAndMark()(String name) {
registerLossyCountingMeterIfNotPresent(meter, LossyCounting);
markMeterIfPresent(meter);
}
{code}
Here, we always need to maintain that the internal dataMap of the LossyCounting
class exactly is equivalent to what we have in our requestsMap. So all the
access to a particular lossyCounter and it's subsequent effect on the
requestsMap & registry must be synchronized. Or else, we can end up having a
metric in the registry that actually isn't in the internal dataMap of the
lossyCounter for example like below:
Lossy Counter referred below as *LC.*
*Thread-1*: Add meter 'm'
*LC* [by Thread-2]**: Added 'm' to the internal 'dataMap'. Add it to the
registry
*Thread-2*: Add meter 'm'
*LC* [by Thread-2]: Increment 'm' count in the 'dataMap'. Swept off. Removed
'm' from the 'dataMap'. So, remove it from the registry.
*Thread-2*: A new meter, being removed. So don't add it in the registry and
remove the remaining meters from the list given by *LC*;
*Thread-1*: Add the meter to the registry as directed by the *LC*
*Voila!* LC doesn't have 'm'. But registry has the 'm'.
*Suggestion:* We don't need synchronization anywhere else in the class except
in the below function:
{code:java}
private void registerLossyCountingMeterIfNotPresent(String requestMeter,
LossyCounting lossyCounting) {
...
synchronized(lossyCounting) {
...
}
}
{code}
This way, even addition of a new lossyCounter like for the region-metrics too,
wouldn't be an issue. Your thoughts? :)
> Fix MetaMetrics issues - [Race condition, Faulty remove logic], few
> improvements
>
>
> Key: HBASE-21991
> URL: https://issues.apache.org/jira/browse/HBASE-21991
> Project: HBase
> Issue Type: Bug
> Components: Coprocessors, metrics
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Attachments: hbase-21991.master.001.patch
>
>
> Here is a list of the issues related to the MetaMetrics implementation:
> +*Bugs*+:
> # [_Lossy counting for top-k_] *Faulty remove logic of non-eligible meters*:
> Under certain conditions, we might end up storing/exposing all the meters
> rather than top-k-ish
> # MetaMetrics can throw NPE resulting in aborting of the RS because of a
> *Race Condition*.
> +*Improvements*+:
> # With high number of regions in the cluster, exposure of metrics for each
> region blows up the JMX from ~140 Kbs to 100+ Mbs depending on the number of
> regions. It's better to use *lossy counting to maintain top-k for region
> metrics* as well.
> # As the lossy meters do not represent actual counts, I think, it'll be
> better to *rename the meters to include "lossy" in the name*. It would be
> more informative while monitoring the metrics and there would be less
> confusion regarding actual counts to lossy counts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[
https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787452#comment-16787452
]
Zheng Hu commented on HBASE-22005:
--
OK, [~ste...@apache.org], Got your mind. Then I think we need an method in
FSDataInputStream to known whether it support the ByteBufferReadable or not.
Now I can only write an method like this to check, while I don't think it's
efficient or good desgin.
{code}
static boolean isByteBufferReadable(FSDataInputStream is) {
InputStream cur = is.getWrappedStream();
for (;;) {
if ((cur instanceof FSDataInputStream)) {
cur = ((FSDataInputStream) cur).getWrappedStream();
} else {
break;
}
}
return cur instanceof ByteBufferReadable;
}
{code}
> Use ByteBuff's refcnt to track the life cycle of data block
> ---
>
> Key: HBASE-22005
> URL: https://issues.apache.org/jira/browse/HBASE-22005
> Project: HBase
> Issue Type: Sub-task
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Attachments: HBASE-22005.HBASE-21879.v1.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22010) docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
[
https://issues.apache.org/jira/browse/HBASE-22010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787436#comment-16787436
]
Guanghao Zhang commented on HBASE-22010:
Sorry. Forget to mvn site and check if it work right.:(
> docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
> ---
>
> Key: HBASE-22010
> URL: https://issues.apache.org/jira/browse/HBASE-22010
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
>
> heading doesn't work, which also means no linking to it.
> the rendered page has an unrendered bit, e.g.
> {code}
> [[upgrade 2.2]] === Upgrade from 2.0 or 2.1 to 2.2+
> HBase 2.2+ uses a new Procedure form assiging/unassigning/moving Regions. It
> does not process HBase 2.1 and 2.0’s Unassign/Assign Procedure types. Upgrade
> requires that we first drain the Master Procedure Store of old style
> Procedures before starting the new 2.2 Master. So you need to make sure that
> before you kill the old version (2.0 or 2.1) Master, there is no region in
> transition. And once the new version (2.2+) Master is up, you can rolling
> upgrade RegionServers one by one.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22006) Fix branch-2.1 findbugs warning; causes nightly show as failed.
[
https://issues.apache.org/jira/browse/HBASE-22006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787387#comment-16787387
]
Hudson commented on HBASE-22006:
Results for branch branch-2
[build #1736 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1736/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1736//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1736//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1736//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1736//console].
> Fix branch-2.1 findbugs warning; causes nightly show as failed.
> ---
>
> Key: HBASE-22006
> URL: https://issues.apache.org/jira/browse/HBASE-22006
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.0.5, 2.1.5, 2.2.1
>
>
> Seems like an old bit of code but its causing warning:
> Dodgy code Warnings
> Code Warning
> DLS Dead store to $L8 in
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeMemStoreChunkCreator()
> Bug type DLS_DEAD_LOCAL_STORE (click for details)
> In class org.apache.hadoop.hbase.regionserver.HRegionServer
> In method
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeMemStoreChunkCreator()
> Local variable stored in JVM register 8
> At HRegionServer.java:[line 1593]
> This seems to be only thing between 2.1 being a blue run in last few
> instances. Let me fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[
https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787386#comment-16787386
]
Hudson commented on HBASE-21874:
Results for branch branch-2
[build #1736 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1736/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1736//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1736//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1736//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1736//console].
> Bucket cache on Persistent memory
> -
>
> Key: HBASE-21874
> URL: https://issues.apache.org/jira/browse/HBASE-21874
> Project: HBase
> Issue Type: New Feature
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21874.patch, HBASE-21874.patch,
> HBASE-21874_V2.patch, HBASE-21874_V4.patch, HBASE-21874_V5.patch,
> HBASE-21874_V6.patch, Pmem_BC.png
>
>
> Non volatile persistent memory devices are byte addressable like DRAM (for
> eg. Intel DCPMM). Bucket cache implementation can take advantage of this new
> memory type and can make use of the existing offheap data structures to serve
> data directly from this memory area without having to bring the data to
> onheap.
> The patch is a new IOEngine implementation that works with the persistent
> memory.
> Note : Here we don't make use of the persistence nature of the device and
> just make use of the big memory it provides.
> Performance numbers to follow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21990) puppycrawl checkstyle dtds 404... moved to sourceforge
[ https://issues.apache.org/jira/browse/HBASE-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787296#comment-16787296 ] stack commented on HBASE-21990: --- .002 is amendment that changes dtd urls. I pushed it to branch-2.0 so can see how it does. > puppycrawl checkstyle dtds 404... moved to sourceforge > -- > > Key: HBASE-21990 > URL: https://issues.apache.org/jira/browse/HBASE-21990 > Project: HBase > Issue Type: Bug > Components: build >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.0.5, 1.3.4, 2.1.4, 1.5.1, 1.2.12 > > Attachments: HBASE-21990.branch-2.0.001.patch, > HBASE-21990.branch-2.0.002.patch > > > Yetus flagging two badly formatted xml files. Seemingly related are the two > filenotfoundexceptions that show in the xml.txt build product. > https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2.0/1399/artifact/output-general/xml.txt > The complaint is that the xml dtds are not found. > Indeed they were moved long time ago: > https://github.com/checkstyle/checkstyle/issues/1571 > They were supposed to stick around at their old location but that seems to > have changed recently. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21990) puppycrawl checkstyle dtds 404... moved to sourceforge
[ https://issues.apache.org/jira/browse/HBASE-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-21990: -- Attachment: HBASE-21990.branch-2.0.002.patch > puppycrawl checkstyle dtds 404... moved to sourceforge > -- > > Key: HBASE-21990 > URL: https://issues.apache.org/jira/browse/HBASE-21990 > Project: HBase > Issue Type: Bug > Components: build >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.2.0, 2.0.5, 1.3.4, 2.1.4, 1.5.1, 1.2.12 > > Attachments: HBASE-21990.branch-2.0.001.patch, > HBASE-21990.branch-2.0.002.patch > > > Yetus flagging two badly formatted xml files. Seemingly related are the two > filenotfoundexceptions that show in the xml.txt build product. > https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2.0/1399/artifact/output-general/xml.txt > The complaint is that the xml dtds are not found. > Indeed they were moved long time ago: > https://github.com/checkstyle/checkstyle/issues/1571 > They were supposed to stick around at their old location but that seems to > have changed recently. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-21990) puppycrawl checkstyle dtds 404... moved to sourceforge
[
https://issues.apache.org/jira/browse/HBASE-21990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack reopened HBASE-21990:
---
Reopening. Still seeing xml fails.
see that branch-2.2 builds would be blue except they have a bad xml complaint
still. See below from
https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2.2/92/artifact/output-general/xml.txt
Looks like reference should be:
https://checkstyle.org/dtds/configuration_1_3.dtd
rather than...
http://checkstyle.sourceforge.net/dtds/configuration_1_3.dtd
I see other branches have failing xml files now too.
Seems like the dtds got removed a few days ago:
https://github.com/checkstyle/checkstyle/issues/6478
Caches are only catching up now?
Let me fix these.
{code}
java.lang.RuntimeException: java.io.FileNotFoundException:
http://checkstyle.sourceforge.net/dtds/configuration_1_3.dtd
at
jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:397)
at
jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:449)
at
jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:406)
at
jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:402)
at
jdk.nashorn.api.scripting.NashornScriptEngine.eval(NashornScriptEngine.java:155)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
at com.sun.tools.script.shell.Main.evaluateString(Main.java:298)
at com.sun.tools.script.shell.Main.evaluateString(Main.java:319)
at com.sun.tools.script.shell.Main.access$300(Main.java:37)
at com.sun.tools.script.shell.Main$3.run(Main.java:217)
at com.sun.tools.script.shell.Main.main(Main.java:48)
Caused by: java.io.FileNotFoundException:
http://checkstyle.sourceforge.net/dtds/configuration_1_3.dtd
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1890)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1492)
at
com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:647)
at
com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startEntity(XMLEntityManager.java:1304)
at
com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startDTDEntity(XMLEntityManager.java:1270)
at
com.sun.org.apache.xerces.internal.impl.XMLDTDScannerImpl.setInputSource(XMLDTDScannerImpl.java:264)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$DTDDriver.dispatch(XMLDocumentScannerImpl.java:1161)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$DTDDriver.next(XMLDocumentScannerImpl.java:1045)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:959)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
at
com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
at
com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at
com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
at
com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:205)
at
jdk.nashorn.internal.scripts.Script$Recompilation$2$19313A$\^system_init\_.XMLDocument(:747)
at jdk.nashorn.internal.scripts.Script$1$\^string\_.:program(:1)
at
jdk.nashorn.internal.runtime.ScriptFunctionData.invoke(ScriptFunctionData.java:637)
at
jdk.nashorn.internal.runtime.ScriptFunction.invoke(ScriptFunction.java:494)
at
jdk.nashorn.internal.runtime.ScriptRuntime.apply(ScriptRuntime.java:393)
... 10 more
{code}
> puppycrawl checkstyle dtds 404... moved to sourceforge
> --
>
> Key: HBASE-21990
> URL: https://issues.apache.org/jira/browse/HBASE-21990
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.2.0, 2.0.5, 1.3.4, 2.1.4, 1.5.1, 1.2.12
>
> Attachments: HBASE-21990.branch-2.0.001.patch
>
>
> Yetus flagging two badly formatted xml files. Seemingly related are the two
> filenotfoundexceptions that show in the xml.txt build product.
> https://builds.apache.org/view/H-L/view/HBase
[jira] [Commented] (HBASE-22006) Fix branch-2.1 findbugs warning; causes nightly show as failed.
[
https://issues.apache.org/jira/browse/HBASE-22006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787252#comment-16787252
]
Hudson commented on HBASE-22006:
Results for branch branch-2.0
[build #1416 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1416/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1416//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1416//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.0/1416//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Fix branch-2.1 findbugs warning; causes nightly show as failed.
> ---
>
> Key: HBASE-22006
> URL: https://issues.apache.org/jira/browse/HBASE-22006
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.0.5, 2.1.5, 2.2.1
>
>
> Seems like an old bit of code but its causing warning:
> Dodgy code Warnings
> Code Warning
> DLS Dead store to $L8 in
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeMemStoreChunkCreator()
> Bug type DLS_DEAD_LOCAL_STORE (click for details)
> In class org.apache.hadoop.hbase.regionserver.HRegionServer
> In method
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeMemStoreChunkCreator()
> Local variable stored in JVM register 8
> At HRegionServer.java:[line 1593]
> This seems to be only thing between 2.1 being a blue run in last few
> instances. Let me fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22006) Fix branch-2.1 findbugs warning; causes nightly show as failed.
[
https://issues.apache.org/jira/browse/HBASE-22006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787200#comment-16787200
]
Hudson commented on HBASE-22006:
Results for branch branch-2.1
[build #932 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/932/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/932//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/932//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/932//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.1/932//console].
> Fix branch-2.1 findbugs warning; causes nightly show as failed.
> ---
>
> Key: HBASE-22006
> URL: https://issues.apache.org/jira/browse/HBASE-22006
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.0.5, 2.1.5, 2.2.1
>
>
> Seems like an old bit of code but its causing warning:
> Dodgy code Warnings
> Code Warning
> DLS Dead store to $L8 in
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeMemStoreChunkCreator()
> Bug type DLS_DEAD_LOCAL_STORE (click for details)
> In class org.apache.hadoop.hbase.regionserver.HRegionServer
> In method
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeMemStoreChunkCreator()
> Local variable stored in JVM register 8
> At HRegionServer.java:[line 1593]
> This seems to be only thing between 2.1 being a blue run in last few
> instances. Let me fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22006) Fix branch-2.1 findbugs warning; causes nightly show as failed.
[
https://issues.apache.org/jira/browse/HBASE-22006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787189#comment-16787189
]
Hudson commented on HBASE-22006:
Results for branch branch-2.2
[build #91 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/91/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/91//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/91//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/91//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix branch-2.1 findbugs warning; causes nightly show as failed.
> ---
>
> Key: HBASE-22006
> URL: https://issues.apache.org/jira/browse/HBASE-22006
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.0.5, 2.1.5, 2.2.1
>
>
> Seems like an old bit of code but its causing warning:
> Dodgy code Warnings
> Code Warning
> DLS Dead store to $L8 in
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeMemStoreChunkCreator()
> Bug type DLS_DEAD_LOCAL_STORE (click for details)
> In class org.apache.hadoop.hbase.regionserver.HRegionServer
> In method
> org.apache.hadoop.hbase.regionserver.HRegionServer.initializeMemStoreChunkCreator()
> Local variable stored in JVM register 8
> At HRegionServer.java:[line 1593]
> This seems to be only thing between 2.1 being a blue run in last few
> instances. Let me fix.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21666) Break up the TestExportSnapshot UTs; they can timeout
[
https://issues.apache.org/jira/browse/HBASE-21666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787181#comment-16787181
]
Hadoop QA commented on HBASE-21666:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
31s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 59s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 15m
40s{color} | {color:green} hbase-mapreduce in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21666 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961616/HBASE-21666.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 0ea4232e2a17 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a7bbff170a |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16297/testReport/ |
| Max. process+thread count | 5003 (vs. ulimit of 1) |
| modules | C: hbase-mapreduce U: hbase-mapreduce |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16297/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generat
[jira] [Updated] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21999: Resolution: Fixed Status: Resolved (was: Patch Available) okay all set. latest website build also succeeded. > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch, > HBASE-21999-addendum-v1.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21963) Add a script for building and verifying release candidate
[
https://issues.apache.org/jira/browse/HBASE-21963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tak Lon (Stephen) Wu updated HBASE-21963:
-
Attachment: HBASE-21963.master.005.patch
> Add a script for building and verifying release candidate
> -
>
> Key: HBASE-21963
> URL: https://issues.apache.org/jira/browse/HBASE-21963
> Project: HBase
> Issue Type: Test
> Components: release, scripts
>Affects Versions: 3.0.0, 2.1.3
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
> Attachments: HBASE-21963.master.001.patch,
> HBASE-21963.master.002.patch, HBASE-21963.master.003.patch,
> HBASE-21963.master.004.patch, HBASE-21963.master.005.patch
>
>
> During the release vote for HBase 2.1.3RC1, a driver/helper script was
> mentioned and can potentially help contributors prepare to vote for a release
> candidate. As recommended, we decided to move toward this tool to under
> {{dev-support/}}
> Here the driver script provides the following automation:
> 1. Import and check publisher key(s)
> 2. Checksum of sources and binaries
> 3. Signature of sources and binaries
> 4. Rat check
> 5. Built from source
> 6. Verify unit tests
> {code}
> # example usage
> $ bash dev-support/hbase-vote.sh -s
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.5.0RC2/
> $ bash dev-support/hbase-vote.sh -h
> hbase-vote. A script for standard vote which verifies the following items
> 1. Checksum of sources and binaries
> 2. Signature of sources and binaries
> 3. Rat check
> 4. Built from source
> 5. Unit tests
> Usage: hbase-vote.sh -s | --source [-k | --key ] [-f |
> --keys-file-url ]
>hbase-vote.sh -h | --help
> -h | --help Show this screen.
> -s | --source '' A URL pointing to the release candidate
> sources and binaries
> e.g.
> https://dist.apache.org/repos/dist/dev/hbase/hbase-RC0/
> -k | --key '' A signature of the public key, e.g. 9AD2AE49
> -f | --keys-file-url '' the URL of the key file, default is
> http://www.apache.org/dist/hbase/KEYS
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HBASE-22010) docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
[
https://issues.apache.org/jira/browse/HBASE-22010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-22010 started by Sean Busbey.
---
> docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
> ---
>
> Key: HBASE-22010
> URL: https://issues.apache.org/jira/browse/HBASE-22010
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0, 2.2.0
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
>
> heading doesn't work, which also means no linking to it.
> the rendered page has an unrendered bit, e.g.
> {code}
> [[upgrade 2.2]] === Upgrade from 2.0 or 2.1 to 2.2+
> HBase 2.2+ uses a new Procedure form assiging/unassigning/moving Regions. It
> does not process HBase 2.1 and 2.0’s Unassign/Assign Procedure types. Upgrade
> requires that we first drain the Master Procedure Store of old style
> Procedures before starting the new 2.2 Master. So you need to make sure that
> before you kill the old version (2.0 or 2.1) Master, there is no region in
> transition. And once the new version (2.2+) Master is up, you can rolling
> upgrade RegionServers one by one.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (HBASE-22010) docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
Sean Busbey created HBASE-22010:
---
Summary: docs on upgrade from 2.0,2.1 -> 2.2 renders incorrectly
Key: HBASE-22010
URL: https://issues.apache.org/jira/browse/HBASE-22010
Project: HBase
Issue Type: Bug
Components: documentation
Affects Versions: 3.0.0, 2.2.0
Reporter: Sean Busbey
Assignee: Sean Busbey
heading doesn't work, which also means no linking to it.
the rendered page has an unrendered bit, e.g.
{code}
[[upgrade 2.2]] === Upgrade from 2.0 or 2.1 to 2.2+
HBase 2.2+ uses a new Procedure form assiging/unassigning/moving Regions. It
does not process HBase 2.1 and 2.0’s Unassign/Assign Procedure types. Upgrade
requires that we first drain the Master Procedure Store of old style Procedures
before starting the new 2.2 Master. So you need to make sure that before you
kill the old version (2.0 or 2.1) Master, there is no region in transition. And
once the new version (2.2+) Master is up, you can rolling upgrade RegionServers
one by one.
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21763) [HBCK2] hbck2 options does not work and throws exceptions
[ https://issues.apache.org/jira/browse/HBASE-21763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787149#comment-16787149 ] Wellington Chevreuil commented on HBASE-21763: -- LGTM, +1 (non binding). [~busbey], would you have any suggestions, or would this be good to go? > [HBCK2] hbck2 options does not work and throws exceptions > - > > Key: HBASE-21763 > URL: https://issues.apache.org/jira/browse/HBASE-21763 > Project: HBase > Issue Type: Bug > Components: hbck2 >Affects Versions: hbck2-1.0.0 >Reporter: Syeda Arshiya Tabreen >Assignee: Syeda Arshiya Tabreen >Priority: Minor > Fix For: hbck2-1.0.0 > > Attachments: HBASE-21763.001.patch, HBASE-21763.002.patch, > HBASE-21763.003.patch, HBASE-21763.patch > > > HBCK2 options throws below exceptions when executed > 1. *--version* option throws NullPointerException > 2. *--hbase.zookeeper.property.clientPort* option throws NumberFormatException > 3. *--zookeeper.znode.parent* option throws IllegalArgumentException -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21666) Break up the TestExportSnapshot UTs; they can timeout
[ https://issues.apache.org/jira/browse/HBASE-21666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-21666: - Attachment: HBASE-21666.master.002.patch > Break up the TestExportSnapshot UTs; they can timeout > - > > Key: HBASE-21666 > URL: https://issues.apache.org/jira/browse/HBASE-21666 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0 >Reporter: stack >Assignee: Tak Lon (Stephen) Wu >Priority: Major > Labels: beginner > Attachments: HBASE-21666.master.001.patch, > HBASE-21666.master.002.patch, HBASE-21666.master.003.patch > > > These timed out for [~Apache9] when he ran with the -PrunAllTests. Suggests > breaking them up into smaller tests so less likely they'll timeout. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21666) Break up the TestExportSnapshot UTs; they can timeout
[ https://issues.apache.org/jira/browse/HBASE-21666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tak Lon (Stephen) Wu updated HBASE-21666: - Attachment: HBASE-21666.master.003.patch > Break up the TestExportSnapshot UTs; they can timeout > - > > Key: HBASE-21666 > URL: https://issues.apache.org/jira/browse/HBASE-21666 > Project: HBase > Issue Type: Bug > Components: test >Affects Versions: 3.0.0 >Reporter: stack >Assignee: Tak Lon (Stephen) Wu >Priority: Major > Labels: beginner > Attachments: HBASE-21666.master.001.patch, > HBASE-21666.master.002.patch, HBASE-21666.master.003.patch > > > These timed out for [~Apache9] when he ran with the -PrunAllTests. Suggests > breaking them up into smaller tests so less likely they'll timeout. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21874) Bucket cache on Persistent memory
[
https://issues.apache.org/jira/browse/HBASE-21874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787143#comment-16787143
]
Hudson commented on HBASE-21874:
Results for branch branch-2
[build #1735 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1735/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1735//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1735//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1735//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/1735//console].
> Bucket cache on Persistent memory
> -
>
> Key: HBASE-21874
> URL: https://issues.apache.org/jira/browse/HBASE-21874
> Project: HBase
> Issue Type: New Feature
> Components: BucketCache
>Affects Versions: 3.0.0
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21874.patch, HBASE-21874.patch,
> HBASE-21874_V2.patch, HBASE-21874_V4.patch, HBASE-21874_V5.patch,
> HBASE-21874_V6.patch, Pmem_BC.png
>
>
> Non volatile persistent memory devices are byte addressable like DRAM (for
> eg. Intel DCPMM). Bucket cache implementation can take advantage of this new
> memory type and can make use of the existing offheap data structures to serve
> data directly from this memory area without having to bring the data to
> onheap.
> The patch is a new IOEngine implementation that works with the persistent
> memory.
> Note : Here we don't make use of the persistence nature of the device and
> just make use of the big memory it provides.
> Performance numbers to follow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21963) Add a script for building and verifying release candidate
[
https://issues.apache.org/jira/browse/HBASE-21963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787138#comment-16787138
]
Hadoop QA commented on HBASE-21963:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
0s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 0m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21963 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961612/HBASE-21963.master.005.patch
|
| Optional Tests | dupname asflicense shellcheck shelldocs |
| uname | Linux 25fa448df8a6 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / a7bbff170a |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| shellcheck | v0.4.4 |
| Max. process+thread count | 43 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16296/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Add a script for building and verifying release candidate
> -
>
> Key: HBASE-21963
> URL: https://issues.apache.org/jira/browse/HBASE-21963
> Project: HBase
> Issue Type: Test
> Components: release, scripts
>Affects Versions: 3.0.0, 2.1.3
>Reporter: Tak Lon (Stephen) Wu
>Assignee: Tak Lon (Stephen) Wu
>Priority: Minor
> Attachments: HBASE-21963.master.001.patch,
> HBASE-21963.master.002.patch, HBASE-21963.master.003.patch,
> HBASE-21963.master.004.patch, HBASE-21963.master.005.patch
>
>
> During the release vote for HBase 2.1.3RC1, a driver/helper script was
> mentioned and can potentially help contributors prepare to vote for a release
> candidate. As recommended, we decided to move toward this tool to under
> {{dev-support/}}
> Here the driver script provides the following automation:
> 1. Import and check publisher key(s)
> 2. Checksum of sources and binaries
> 3. Signature of sources and binaries
> 4. Rat check
> 5. Built from source
> 6. Verify unit tests
> {code}
> # example usage
> $ bash dev-support/hbase-vote.sh -s
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.5.0RC2/
> $ bash dev-support/hbase-vote.sh -h
> hbase-vote. A script for standard vote which verifies the following items
> 1. Checksum of sources and binaries
> 2. Signature of sources and binaries
> 3. Rat check
> 4. Built from source
> 5. Unit tests
> Usage: hbase-vote.sh -s | --source [-k | --key ] [-f |
> --keys-file-url ]
>hbase-vote.sh -h | --help
> -h | --help Show this screen.
> -s | --source '' A URL pointing to the release candidate
> sources and binaries
> e.g.
> https://dist.apache.org/repos/dist/dev/hbase/hbase-RC0/
> -k | --key '' A signature of the public key, e.g. 9AD2AE49
> -f | --keys-file-url '' the URL of the key file, default is
> http://www.apache.org/dist/hbase/KEYS
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787119#comment-16787119 ] stack commented on HBASE-21999: --- Go for it @busbey. Push as is. There's a bunch more info now that wasn't there previous so should be able to untangle why no revision should it arise in future. bq. it does not use docker. I'm happy if someone wants to modernize it, but I personally don't want to spend the time on the ATM. nod > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch, > HBASE-21999-addendum-v1.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[
https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787065#comment-16787065
]
Hadoop QA commented on HBASE-21999:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
0s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
1s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
7m 45s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
33s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21999 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961601/HBASE-21999-addendum-v1.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile shellcheck shelldocs |
| uname | Linux 79b084180405 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 65149bd8fc |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| shellcheck | v0.4.4 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16295/testReport/ |
| Max. process+thread count | 310 (vs. ulimit of 1) |
| modules | C: hbase-common U: hbase-common |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16295/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> [DEBUG] Exit if git returns empty revision!
> ---
>
> Key:
[jira] [Commented] (HBASE-22005) Use ByteBuff's refcnt to track the life cycle of data block
[ https://issues.apache.org/jira/browse/HBASE-22005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787053#comment-16787053 ] Steve Loughran commented on HBASE-22005: I think you'll need a plan to continue to work with stores which don't support BB; that includes object stores which ship with HBase support today (hello Azure!) and whose users will be unhappy when things not working. bq. I still think those basic fs, such as LocalFileSystem/DistributedFileSystem need the ByteBuffer read/pread method, it's so common to use I see the world moving away from Posix in two directions * near-RAM-speed solid state storage. Here memory access operations make a lot more sense than the stream API, because in hardware these can be part of the memory space of the application. Why copy it into process memory at all, when it can just be memory mapped? * object storage. Here we go the other way - high latency IO where the cost of a seek() is such that you can see the logs pause and you'll know "hey! it's a GET". There we're looking at async IO APIs, vectored IO ops etc. I don't expect stores to implement ByteBufferReadable; async vector reads where you provide a list of Use ByteBuff's refcnt to track the life cycle of data block > --- > > Key: HBASE-22005 > URL: https://issues.apache.org/jira/browse/HBASE-22005 > Project: HBase > Issue Type: Sub-task >Reporter: Zheng Hu >Assignee: Zheng Hu >Priority: Major > Attachments: HBASE-22005.HBASE-21879.v1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787079#comment-16787079 ] Sean Busbey commented on HBASE-21999: - good for me to push [~stack]? or you want the "fail loudly during a release" thing? > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch, > HBASE-21999-addendum-v1.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21879) Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
[
https://issues.apache.org/jira/browse/HBASE-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787063#comment-16787063
]
Hudson commented on HBASE-21879:
Results for branch HBASE-21879
[build #17 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/17/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/17//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/17//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/17//JDK8_Nightly_Build_Report_(Hadoop3)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/17//console].
> Read HFile's block to ByteBuffer directly instead of to byte for reducing
> young gc purpose
> --
>
> Key: HBASE-21879
> URL: https://issues.apache.org/jira/browse/HBASE-21879
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21879.v1.patch, HBASE-21879.v1.patch,
> QPS-latencies-before-HBASE-21879.png, gc-data-before-HBASE-21879.png
>
>
> In HFileBlock#readBlockDataInternal, we have the following:
> {code}
> @VisibleForTesting
> protected HFileBlock readBlockDataInternal(FSDataInputStream is, long offset,
> long onDiskSizeWithHeaderL, boolean pread, boolean verifyChecksum,
> boolean updateMetrics)
> throws IOException {
> // .
> // TODO: Make this ByteBuffer-based. Will make it easier to go to HDFS with
> BBPool (offheap).
> byte [] onDiskBlock = new byte[onDiskSizeWithHeader + hdrSize];
> int nextBlockOnDiskSize = readAtOffset(is, onDiskBlock, preReadHeaderSize,
> onDiskSizeWithHeader - preReadHeaderSize, true, offset +
> preReadHeaderSize, pread);
> if (headerBuf != null) {
> // ...
> }
> // ...
> }
> {code}
> In the read path, we still read the block from hfile to on-heap byte[], then
> copy the on-heap byte[] to offheap bucket cache asynchronously, and in my
> 100% get performance test, I also observed some frequent young gc, The
> largest memory footprint in the young gen should be the on-heap block byte[].
> In fact, we can read HFile's block to ByteBuffer directly instead of to
> byte[] for reducing young gc purpose. we did not implement this before,
> because no ByteBuffer reading interface in the older HDFS client, but 2.7+
> has supported this now, so we can fix this now. I think.
> Will provide an patch and some perf-comparison for this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-21977) Skip replay WAL and update seqid when open regions restored from snapshot
[
https://issues.apache.org/jira/browse/HBASE-21977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787045#comment-16787045
]
Hadoop QA commented on HBASE-21977:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
42s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
50s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
9m 1s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}251m 41s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}292m 8s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.client.TestAdmin1 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21977 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961544/HBASE-21977.master.003.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux aaf76bc030a2 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 63d0e6ed4a |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/16291/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16291/testReport/ |
| Max. process+thread count | 5071 (vs. ulimit of 1) |
| modules | C: hbase-server U: hbase-server |
| Console ou
[jira] [Commented] (HBASE-21774) do not use currentTimeMillis to measure intervals
[ https://issues.apache.org/jira/browse/HBASE-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787044#comment-16787044 ] Andrew Purtell commented on HBASE-21774: As to your other question, we can commit this change and then do the work of converting interval measurements to use the new method in subtasks and/or followup issues. > do not use currentTimeMillis to measure intervals > - > > Key: HBASE-21774 > URL: https://issues.apache.org/jira/browse/HBASE-21774 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Dmitriy Kuharev >Priority: Minor > Attachments: HBASE-21774.master.001.patch, > HBASE-21774.master.002.patch, HBASE-21774.master.003.patch > > > I've noticed it in a few places in the code... > currentMillis can go backwards and have other artifacts. > nanoTime should be used for intervals (see > [https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()|https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()] > ) unless it's both the case that the calls are frequent and nanoTime will > result in perf overhead, and also that artifacts from negative intervals and > such are relatively harmless or possible to work around in the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HBASE-21774) do not use currentTimeMillis to measure intervals
[
https://issues.apache.org/jira/browse/HBASE-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787039#comment-16787039
]
Andrew Purtell edited comment on HBASE-21774 at 3/7/19 6:01 PM:
The patch looks fine.
I think the javadoc of monotonicTime should reiterate that it is NOT a better
"currentTime", for sake of clarity. Also, I think we would be missing the other
caveat from Hadoop's javadoc:
{quote}
This function can return a negative value and it must be handled correctly by
callers. See the documentation of System#nanoTime for caveats.
{quote}
Make these changes and I'd be +1
was (Author: apurtell):
The patch looks fine.
I think the javadoc of monotonicTime should reiterate that it is NOT a better
"currentTime", for sake of clarity. Also, I think we would be missing the other
caveat from Hadoop's javadoc:
{quote}
* This function can return a negative value and it must be handled correctly
* by callers. See the documentation of System#nanoTime for caveats.
{quote}
Make these changes and I'd be +1
> do not use currentTimeMillis to measure intervals
> -
>
> Key: HBASE-21774
> URL: https://issues.apache.org/jira/browse/HBASE-21774
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Dmitriy Kuharev
>Priority: Minor
> Attachments: HBASE-21774.master.001.patch,
> HBASE-21774.master.002.patch, HBASE-21774.master.003.patch
>
>
> I've noticed it in a few places in the code...
> currentMillis can go backwards and have other artifacts.
> nanoTime should be used for intervals (see
> [https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()|https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()]
> ) unless it's both the case that the calls are frequent and nanoTime will
> result in perf overhead, and also that artifacts from negative intervals and
> such are relatively harmless or possible to work around in the code.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21774) do not use currentTimeMillis to measure intervals
[ https://issues.apache.org/jira/browse/HBASE-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned HBASE-21774: -- Assignee: Andrew Purtell (was: Dmitriy Kuharev) > do not use currentTimeMillis to measure intervals > - > > Key: HBASE-21774 > URL: https://issues.apache.org/jira/browse/HBASE-21774 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Andrew Purtell >Priority: Minor > Attachments: HBASE-21774.master.001.patch, > HBASE-21774.master.002.patch, HBASE-21774.master.003.patch > > > I've noticed it in a few places in the code... > currentMillis can go backwards and have other artifacts. > nanoTime should be used for intervals (see > [https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()|https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()] > ) unless it's both the case that the calls are frequent and nanoTime will > result in perf overhead, and also that artifacts from negative intervals and > such are relatively harmless or possible to work around in the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21774) do not use currentTimeMillis to measure intervals
[
https://issues.apache.org/jira/browse/HBASE-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787039#comment-16787039
]
Andrew Purtell commented on HBASE-21774:
The patch looks fine.
I think the javadoc of monotonicTime should reiterate that it is NOT a better
"currentTime", for sake of clarity. Also, I think we would be missing the other
caveat from Hadoop's javadoc:
{quote}
* This function can return a negative value and it must be handled correctly
* by callers. See the documentation of System#nanoTime for caveats.
{quote}
Make these changes and I'd be +1
> do not use currentTimeMillis to measure intervals
> -
>
> Key: HBASE-21774
> URL: https://issues.apache.org/jira/browse/HBASE-21774
> Project: HBase
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Dmitriy Kuharev
>Priority: Minor
> Attachments: HBASE-21774.master.001.patch,
> HBASE-21774.master.002.patch, HBASE-21774.master.003.patch
>
>
> I've noticed it in a few places in the code...
> currentMillis can go backwards and have other artifacts.
> nanoTime should be used for intervals (see
> [https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()|https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()]
> ) unless it's both the case that the calls are frequent and nanoTime will
> result in perf overhead, and also that artifacts from negative intervals and
> such are relatively harmless or possible to work around in the code.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HBASE-21774) do not use currentTimeMillis to measure intervals
[ https://issues.apache.org/jira/browse/HBASE-21774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned HBASE-21774: -- Assignee: Dmitriy Kuharev (was: Andrew Purtell) > do not use currentTimeMillis to measure intervals > - > > Key: HBASE-21774 > URL: https://issues.apache.org/jira/browse/HBASE-21774 > Project: HBase > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Dmitriy Kuharev >Priority: Minor > Attachments: HBASE-21774.master.001.patch, > HBASE-21774.master.002.patch, HBASE-21774.master.003.patch > > > I've noticed it in a few places in the code... > currentMillis can go backwards and have other artifacts. > nanoTime should be used for intervals (see > [https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()|https://docs.oracle.com/javase/7/docs/api/java/lang/System.html#nanoTime()] > ) unless it's both the case that the calls are frequent and nanoTime will > result in perf overhead, and also that artifacts from negative intervals and > such are relatively harmless or possible to work around in the code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21999: Attachment: HBASE-21999-addendum-v1.patch > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch, > HBASE-21999-addendum-v1.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787030#comment-16787030 ] Sean Busbey commented on HBASE-21999: - v1 - fix the shellcheck complaint. > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch, > HBASE-21999-addendum-v1.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787031#comment-16787031 ] Sean Busbey commented on HBASE-21999: - it does not use docker. I'm happy if someone wants to modernize it, but I personally don't want to spend the time on the ATM. > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch, > HBASE-21999-addendum-v1.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787024#comment-16787024 ] stack commented on HBASE-21999: --- Script addition looks lovely [~busbey]. Bummer on site fail. Sorry about that. PITA. Update its docker image? > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[
https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787014#comment-16787014
]
Hadoop QA commented on HBASE-21999:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
0s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} shellcheck {color} | {color:red} 0m
1s{color} | {color:red} The patch generated 1 new + 10 unchanged - 0 fixed = 11
total (was 10) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
29s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 45s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
31s{color} | {color:green} hbase-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-21999 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961592/HBASE-21999-addendum-v0.patch
|
| Optional Tests | dupname asflicense javac javadoc unit shadedjars
hadoopcheck xml compile shellcheck shelldocs |
| uname | Linux ca69c058d6b7 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 65149bd8fc |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| shellcheck | v0.4.4 |
| shellcheck |
https://builds.apache.org/job/PreCommit-HBASE-Build/16294/artifact/patchprocess/diff-patch-shellcheck.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16294/testReport/ |
| Max. process+thread count | 275 (vs. ulimit of 1) |
| modules | C: hbase-common U: hbase-common |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16294/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
Th
[jira] [Commented] (HBASE-21968) Release 2.1.4
[ https://issues.apache.org/jira/browse/HBASE-21968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787005#comment-16787005 ] stack commented on HBASE-21968: --- We had our first blue build on nightlies for first time in a long time: https://builds.apache.org/view/H-L/view/HBase/job/HBase%20Nightly/job/branch-2.1/ See #930 > Release 2.1.4 > - > > Key: HBASE-21968 > URL: https://issues.apache.org/jira/browse/HBASE-21968 > Project: HBase > Issue Type: Umbrella > Components: release >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786986#comment-16786986
]
Hadoop QA commented on HBASE-22009:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
47s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 56s{color} | {color:green} Patch does not cause any errors with Hadoop 2.7.4
or 3.0.0. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
26s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:b002b0b |
| JIRA Issue | HBASE-22009 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12961587/HBASE-22009.master.000.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e8ef88775cbd 4.4.0-139-generic #165~14.04.1-Ubuntu SMP Wed Oct
31 10:55:11 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 65149bd8fc |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/16293/testReport/ |
| Max. process+thread count | 3414 (vs. ulimit of 1) |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCom
[jira] [Updated] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21999: Component/s: build > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task > Components: build >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21999: Status: Patch Available (was: Reopened) > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786985#comment-16786985 ] Sean Busbey commented on HBASE-21999: - here's an example addendum * fail the build when the saveVersion script exits, rather than skipping the creation of Version.java and waiting for a later failure * make the name of the build step easier to pick out in the maven output * don't fail on VCS command failure * WARN on VCS command failure and use the existing "unknown" that we'd use if no VCS alternatively, I could make things fail when we're in a release build since that's when we care about having this kind of revision info. wdyt? > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-21999: Attachment: HBASE-21999-addendum-v0.patch > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > Attachments: HBASE-21999-addendum-v0.patch > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (HBASE-21999) [DEBUG] Exit if git returns empty revision!
[ https://issues.apache.org/jira/browse/HBASE-21999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey reopened HBASE-21999: - This is breaking the website build. I'd guess the git version there is too old, which causes the same kind of failure we saw in the old docker image. What do you think about an addendum that refactors things so we get the "Unknown" revision path in this kind of case? AFAIK the website doesn't care if the generated Version class knows what git revision it came from. > [DEBUG] Exit if git returns empty revision! > --- > > Key: HBASE-21999 > URL: https://issues.apache.org/jira/browse/HBASE-21999 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 3.0.0, 2.0.5, 2.1.4, 2.2.1 > > > Let me commit a bit of debug on branch-2.0. Can't figure out how we are > getting empty git revision string. Have the build fail if empty -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20918) Re-enable TestRpcHandlerException
[
https://issues.apache.org/jira/browse/HBASE-20918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786937#comment-16786937
]
Hadoop QA commented on HBASE-20918:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HBASE-20918 does not apply to master. Rebase required? Wrong
Branch? See https://yetus.apache.org/documentation/0.8.0/precommit-patchnames
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HBASE-20918 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12932579/HBASE-20918.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/16292/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Re-enable TestRpcHandlerException
> --
>
> Key: HBASE-20918
> URL: https://issues.apache.org/jira/browse/HBASE-20918
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Jack Bearden
>Assignee: Jack Bearden
>Priority: Minor
> Attachments: HBASE-20918.001.patch
>
>
> Work done to re-enable this test as part of the HBASE-20266 review.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22009:
-
Description:
{code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
private SortedSet getDefaultServers() throws IOException {
SortedSet defaultServers = Sets.newTreeSet();
for (ServerName serverName : getOnlineRS()) {
Address server = Address.fromParts(serverName.getHostname(),
serverName.getPort());
boolean found = false;
for (RSGroupInfo rsgi : listRSGroups()) {
if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
rsgi.containsServer(server)) {
found = true;
break;
}
}
if (!found) {
defaultServers.add(server);
}
}
return defaultServers;
}
{code}
That is a logic of 2 nest loops and for each server, listRSGroups() allocates a
new LinkedList and all Map#values(), both of which are very heavy operations.
Maybe the inner loop could be moved out, that is
# Build a list of servers of other groups than default group
# Iterate each online servers and check if it is in the list above. If it is
not, then it belongs to default group.
was:
{code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
private SortedSet getDefaultServers() throws IOException {
SortedSet defaultServers = Sets.newTreeSet();
for (ServerName serverName : getOnlineRS()) {
Address server = Address.fromParts(serverName.getHostname(),
serverName.getPort());
boolean found = false;
for (RSGroupInfo rsgi : listRSGroups()) {
if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
rsgi.containsServer(server)) {
found = true;
break;
}
}
if (!found) {
defaultServers.add(server);
}
}
return defaultServers;
}
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops and for each server, listRSGroups() allocates
> a new LinkedList and all Map#values(), both of which are very heavy
> operations. Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786942#comment-16786942
]
Xiang Li commented on HBASE-22009:
--
Update the very first patch v000. Please have a review [~xucang], at your
convenience. Thanks!
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops and for each server, listRSGroups() allocates
> a new LinkedList and all Map#values(), both of which are very heavy
> operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-20918) Re-enable TestRpcHandlerException
[ https://issues.apache.org/jira/browse/HBASE-20918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-20918: Component/s: test > Re-enable TestRpcHandlerException > -- > > Key: HBASE-20918 > URL: https://issues.apache.org/jira/browse/HBASE-20918 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Jack Bearden >Assignee: Jack Bearden >Priority: Major > Attachments: HBASE-20918.001.patch > > > Work done to re-enable this test as part of the HBASE-20266 review. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HBASE-20918) Re-enable TestRpcHandlerException
[ https://issues.apache.org/jira/browse/HBASE-20918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786938#comment-16786938 ] Sean Busbey commented on HBASE-20918: - test ran fine locally. pushed to master. if it looks good there then I'll push to older branches and close. > Re-enable TestRpcHandlerException > -- > > Key: HBASE-20918 > URL: https://issues.apache.org/jira/browse/HBASE-20918 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Jack Bearden >Assignee: Jack Bearden >Priority: Minor > Attachments: HBASE-20918.001.patch > > > Work done to re-enable this test as part of the HBASE-20266 review. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22009:
-
Status: Patch Available (was: Open)
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops and for each server, listRSGroups() allocates
> a new LinkedList and all Map#values(), both of which are very heavy
> operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22009:
-
Attachment: HBASE-22009.master.000.patch
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
> Attachments: HBASE-22009.master.000.patch
>
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops and for each server, listRSGroups() allocates
> a new LinkedList and all Map#values(), both of which are very heavy
> operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22009:
-
Description:
{code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
private SortedSet getDefaultServers() throws IOException {
SortedSet defaultServers = Sets.newTreeSet();
for (ServerName serverName : getOnlineRS()) {
Address server = Address.fromParts(serverName.getHostname(),
serverName.getPort());
boolean found = false;
for (RSGroupInfo rsgi : listRSGroups()) {
if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
rsgi.containsServer(server)) {
found = true;
break;
}
}
if (!found) {
defaultServers.add(server);
}
}
return defaultServers;
}
{code}
was:
{code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}}
private SortedSet getDefaultServers() throws IOException {
SortedSet defaultServers = Sets.newTreeSet();
for (ServerName serverName : getOnlineRS()) {
Address server = Address.fromParts(serverName.getHostname(),
serverName.getPort());
boolean found = false;
for (RSGroupInfo rsgi : listRSGroups()) {
if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
rsgi.containsServer(server)) {
found = true;
break;
}
}
if (!found) {
defaultServers.add(server);
}
}
return defaultServers;
}
{code}
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22009:
-
Description:
{code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
private SortedSet getDefaultServers() throws IOException {
SortedSet defaultServers = Sets.newTreeSet();
for (ServerName serverName : getOnlineRS()) {
Address server = Address.fromParts(serverName.getHostname(),
serverName.getPort());
boolean found = false;
for (RSGroupInfo rsgi : listRSGroups()) {
if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
rsgi.containsServer(server)) {
found = true;
break;
}
}
if (!found) {
defaultServers.add(server);
}
}
return defaultServers;
}
{code}
That is a logic of 2 nest loops and for each server, listRSGroups() allocates a
new LinkedList and all Map#values(), both of which are very heavy operations.
Maybe the inner loop could be moved out, that is
# Build a list of servers of other groups than default group
# Iterate each online servers and check if it is in the list above. If it is
not, then it belongs to default group.
was:
{code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
private SortedSet getDefaultServers() throws IOException {
SortedSet defaultServers = Sets.newTreeSet();
for (ServerName serverName : getOnlineRS()) {
Address server = Address.fromParts(serverName.getHostname(),
serverName.getPort());
boolean found = false;
for (RSGroupInfo rsgi : listRSGroups()) {
if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
rsgi.containsServer(server)) {
found = true;
break;
}
}
if (!found) {
defaultServers.add(server);
}
}
return defaultServers;
}
{code}
That is a logic of 2 nest loops and for each server, listRSGroups() allocates a
new LinkedList and all Map#values(), both of which are very heavy operations.
Maybe the inner loop could be moved out, that is
# Build a list of servers of other groups than default group
# Iterate each online servers and check if it is in the list above. If it is
not, then it belongs to default group.
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops and for each server, listRSGroups() allocates
> a new LinkedList and all Map#values(), both of which are very heavy
> operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HBASE-22009) Improve RSGroupInfoManagerImpl#getDefaultServers()
[
https://issues.apache.org/jira/browse/HBASE-22009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiang Li updated HBASE-22009:
-
Description:
{code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
private SortedSet getDefaultServers() throws IOException {
SortedSet defaultServers = Sets.newTreeSet();
for (ServerName serverName : getOnlineRS()) {
Address server = Address.fromParts(serverName.getHostname(),
serverName.getPort());
boolean found = false;
for (RSGroupInfo rsgi : listRSGroups()) {
if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
rsgi.containsServer(server)) {
found = true;
break;
}
}
if (!found) {
defaultServers.add(server);
}
}
return defaultServers;
}
{code}
That is a logic of 2 nest loops and for each server, listRSGroups() allocates a
new LinkedList and all Map#values(), both of which are very heavy operations.
Maybe the inner loop could be moved out, that is
# Build a list of servers of other groups than default group
# Iterate each online servers and check if it is in the list above. If it is
not, then it belongs to default group.
was:
{code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
private SortedSet getDefaultServers() throws IOException {
SortedSet defaultServers = Sets.newTreeSet();
for (ServerName serverName : getOnlineRS()) {
Address server = Address.fromParts(serverName.getHostname(),
serverName.getPort());
boolean found = false;
for (RSGroupInfo rsgi : listRSGroups()) {
if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
rsgi.containsServer(server)) {
found = true;
break;
}
}
if (!found) {
defaultServers.add(server);
}
}
return defaultServers;
}
{code}
That is a logic of 2 nest loops and for each server, listRSGroups() allocates a
new LinkedList and all Map#values(), both of which are very heavy operations.
Maybe the inner loop could be moved out, that is
# Build a list of servers of other groups than default group
# Iterate each online servers and check if it is in the list above. If it is
not, then it belongs to default group.
> Improve RSGroupInfoManagerImpl#getDefaultServers()
> --
>
> Key: HBASE-22009
> URL: https://issues.apache.org/jira/browse/HBASE-22009
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Xiang Li
>Assignee: Xiang Li
>Priority: Minor
>
> {code:title=RSGroupInfoManagerImpl.java|borderStyle=solid}
> private SortedSet getDefaultServers() throws IOException {
> SortedSet defaultServers = Sets.newTreeSet();
> for (ServerName serverName : getOnlineRS()) {
> Address server = Address.fromParts(serverName.getHostname(),
> serverName.getPort());
> boolean found = false;
> for (RSGroupInfo rsgi : listRSGroups()) {
> if (!RSGroupInfo.DEFAULT_GROUP.equals(rsgi.getName()) &&
> rsgi.containsServer(server)) {
> found = true;
> break;
> }
> }
> if (!found) {
> defaultServers.add(server);
> }
> }
> return defaultServers;
> }
> {code}
> That is a logic of 2 nest loops and for each server, listRSGroups() allocates
> a new LinkedList and all Map#values(), both of which are very heavy
> operations.
> Maybe the inner loop could be moved out, that is
> # Build a list of servers of other groups than default group
> # Iterate each online servers and check if it is in the list above. If it is
> not, then it belongs to default group.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
