[GitHub] [hbase] Apache9 commented on a change in pull request #498: HBASE-22819 Automatically migrate the rs group config for table after…
Apache9 commented on a change in pull request #498: HBASE-22819 Automatically
migrate the rs group config for table after…
URL: https://github.com/apache/hbase/pull/498#discussion_r315532864
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java
##
@@ -356,9 +363,129 @@ public synchronized void removeServers(Set
servers) throws IOException
return RSGroupInfoList;
}
- @Override
- public void refresh() throws IOException {
-refresh(false);
+ private void waitUntilSomeProcsDone(Set pendingProcIds) {
+int size = pendingProcIds.size();
+while (!masterServices.isStopped()) {
+ for (Iterator iter = pendingProcIds.iterator(); iter.hasNext();) {
+long procId = iter.next();
+if (masterServices.getMasterProcedureExecutor().isFinished(procId)) {
+ iter.remove();
+}
+ }
+ if (pendingProcIds.size() < size) {
+return;
+ }
+ try {
+Thread.sleep(1000);
+ } catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+ }
+}
+ }
+
+ private void waitUntilMasterStarted() {
+while (!masterServices.isInitialized() && !masterServices.isStopped()) {
+ try {
+Thread.sleep(1000);
+ } catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+ }
+}
+ }
+
+ private void migrate(Collection groupList, int maxConcurrency) {
+waitUntilMasterStarted();
+Set pendingProcIds = new HashSet<>();
+for (RSGroupInfo groupInfo : groupList) {
+ if (groupInfo.getName().equals(RSGroupInfo.DEFAULT_GROUP)) {
+continue;
+ }
+ SortedSet failedTables = new TreeSet<>();
+ for (TableName tableName : groupInfo.getTables()) {
+LOG.info("Migrating {} in group {}", tableName, groupInfo.getName());
+TableDescriptor oldTd;
+try {
+ oldTd = masterServices.getTableDescriptors().get(tableName);
+} catch (IOException e) {
+ LOG.warn("Failed to migrate {} in group {}", tableName,
groupInfo.getName(), e);
+ failedTables.add(tableName);
+ continue;
+}
+if (oldTd == null) {
+ continue;
+}
+if (oldTd.getRegionServerGroup().isPresent()) {
+ // either we have already migrated it or that user has set the rs
group using the new
+ // code which will set the group directly on table descriptor, skip.
+ LOG.debug("Skip migrating {} since it is already in group {}",
tableName,
+oldTd.getRegionServerGroup().get());
+ continue;
+}
+TableDescriptor newTd = TableDescriptorBuilder.newBuilder(oldTd)
+ .setRegionServerGroup(groupInfo.getName()).build();
+try {
+ pendingProcIds.add(
Review comment:
I think doing things automatically will be more friendly? As users may
forget to user the tool and all the table configs will be ignored...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22806) Recreating a deleted column family brings back the deleted cells
[ https://issues.apache.org/jira/browse/HBASE-22806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911052#comment-16911052 ] Pankaj Kumar commented on HBASE-22806: -- The problem happens when region memstore contain entries for the deleted CF and CF is deleted dynamically (without disabling the table). Since we delete the CF from FS first and then reopen the region and during reopen RS will flush the memstore content to FS. So store fille will contain the memstore content for the deleted CF. > Recreating a deleted column family brings back the deleted cells > > > Key: HBASE-22806 > URL: https://issues.apache.org/jira/browse/HBASE-22806 > Project: HBase > Issue Type: Bug > Components: API >Affects Versions: 2.1.3 > Environment: Scala > HBase Java Client > Mac/Linux >Reporter: Chao >Priority: Major > > Steps to reproduce the bug: > # Create a table with column family CF > # Add some cells C1, C2 in CF > # Remove CF using either: > ** TableDescriptorBuilder.removeColumnFamily() and Admin.modifyTable() > ** Admin.deleteColumnFamily() > # Create CF again > Expected: no cells (all cells marked as deleted) in CF > Actual: C1, C2 shows up automatically in CF > Extra information: in step 3, if doing Admin.modifyColumnFamily() without > actually changing anything, then after step 4 the cells won't come back. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (HBASE-22806) Recreating a deleted column family brings back the deleted cells
[ https://issues.apache.org/jira/browse/HBASE-22806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911052#comment-16911052 ] Pankaj Kumar edited comment on HBASE-22806 at 8/20/19 7:02 AM: --- The problem happens when region memstore contain entries for the deleted CF and CF is deleted dynamically (without disabling the table). Since we delete the CF from FS first and then reopen the region, during reopen RS will flush the memstore content to FS. So deleted CF store will contain the memstore content for the deleted CF. was (Author: pankaj2461): The problem happens when region memstore contain entries for the deleted CF and CF is deleted dynamically (without disabling the table). Since we delete the CF from FS first and then reopen the region and during reopen RS will flush the memstore content to FS. So store fille will contain the memstore content for the deleted CF. > Recreating a deleted column family brings back the deleted cells > > > Key: HBASE-22806 > URL: https://issues.apache.org/jira/browse/HBASE-22806 > Project: HBase > Issue Type: Bug > Components: API >Affects Versions: 2.1.3 > Environment: Scala > HBase Java Client > Mac/Linux >Reporter: Chao >Priority: Major > > Steps to reproduce the bug: > # Create a table with column family CF > # Add some cells C1, C2 in CF > # Remove CF using either: > ** TableDescriptorBuilder.removeColumnFamily() and Admin.modifyTable() > ** Admin.deleteColumnFamily() > # Create CF again > Expected: no cells (all cells marked as deleted) in CF > Actual: C1, C2 shows up automatically in CF > Extra information: in step 3, if doing Admin.modifyColumnFamily() without > actually changing anything, then after step 4 the cells won't come back. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22806) Recreating a deleted column family brings back the deleted cells
[ https://issues.apache.org/jira/browse/HBASE-22806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911056#comment-16911056 ] Reid Chan commented on HBASE-22806: --- (y)[~pankaj2461], make sense to me, is there an upcoming patch? (expecting) > Recreating a deleted column family brings back the deleted cells > > > Key: HBASE-22806 > URL: https://issues.apache.org/jira/browse/HBASE-22806 > Project: HBase > Issue Type: Bug > Components: API >Affects Versions: 2.1.3 > Environment: Scala > HBase Java Client > Mac/Linux >Reporter: Chao >Priority: Major > > Steps to reproduce the bug: > # Create a table with column family CF > # Add some cells C1, C2 in CF > # Remove CF using either: > ** TableDescriptorBuilder.removeColumnFamily() and Admin.modifyTable() > ** Admin.deleteColumnFamily() > # Create CF again > Expected: no cells (all cells marked as deleted) in CF > Actual: C1, C2 shows up automatically in CF > Extra information: in step 3, if doing Admin.modifyColumnFamily() without > actually changing anything, then after step 4 the cells won't come back. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] infraio commented on a change in pull request #498: HBASE-22819 Automatically migrate the rs group config for table after…
infraio commented on a change in pull request #498: HBASE-22819 Automatically
migrate the rs group config for table after…
URL: https://github.com/apache/hbase/pull/498#discussion_r315535097
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupInfoManagerImpl.java
##
@@ -356,9 +363,129 @@ public synchronized void removeServers(Set
servers) throws IOException
return RSGroupInfoList;
}
- @Override
- public void refresh() throws IOException {
-refresh(false);
+ private void waitUntilSomeProcsDone(Set pendingProcIds) {
+int size = pendingProcIds.size();
+while (!masterServices.isStopped()) {
+ for (Iterator iter = pendingProcIds.iterator(); iter.hasNext();) {
+long procId = iter.next();
+if (masterServices.getMasterProcedureExecutor().isFinished(procId)) {
+ iter.remove();
+}
+ }
+ if (pendingProcIds.size() < size) {
+return;
+ }
+ try {
+Thread.sleep(1000);
+ } catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+ }
+}
+ }
+
+ private void waitUntilMasterStarted() {
+while (!masterServices.isInitialized() && !masterServices.isStopped()) {
+ try {
+Thread.sleep(1000);
+ } catch (InterruptedException e) {
+Thread.currentThread().interrupt();
+ }
+}
+ }
+
+ private void migrate(Collection groupList, int maxConcurrency) {
+waitUntilMasterStarted();
+Set pendingProcIds = new HashSet<>();
+for (RSGroupInfo groupInfo : groupList) {
+ if (groupInfo.getName().equals(RSGroupInfo.DEFAULT_GROUP)) {
+continue;
+ }
+ SortedSet failedTables = new TreeSet<>();
+ for (TableName tableName : groupInfo.getTables()) {
+LOG.info("Migrating {} in group {}", tableName, groupInfo.getName());
+TableDescriptor oldTd;
+try {
+ oldTd = masterServices.getTableDescriptors().get(tableName);
+} catch (IOException e) {
+ LOG.warn("Failed to migrate {} in group {}", tableName,
groupInfo.getName(), e);
+ failedTables.add(tableName);
+ continue;
+}
+if (oldTd == null) {
+ continue;
+}
+if (oldTd.getRegionServerGroup().isPresent()) {
+ // either we have already migrated it or that user has set the rs
group using the new
+ // code which will set the group directly on table descriptor, skip.
+ LOG.debug("Skip migrating {} since it is already in group {}",
tableName,
+oldTd.getRegionServerGroup().get());
+ continue;
+}
+TableDescriptor newTd = TableDescriptorBuilder.newBuilder(oldTd)
+ .setRegionServerGroup(groupInfo.getName()).build();
+try {
+ pendingProcIds.add(
Review comment:
Need a new issue to add document for this? And add a config for
maxConcurrency? If user decide to upgrade, user can use this config to reduce
the impact for availability.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Reidddddd commented on a change in pull request #511: HBASE-22879 user_permission command failed to show global permission
Reidd commented on a change in pull request #511: HBASE-22879
user_permission command failed to show global permission
URL: https://github.com/apache/hbase/pull/511#discussion_r315536529
##
File path:
hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestShell.java
##
@@ -35,7 +35,7 @@
@Test
public void testRunShellTests() throws IOException {
System.setProperty("shell.test.exclude",
"replication_admin_test.rb,rsgroup_shell_test.rb," +
- "admin_test.rb,table_test.rb,quotas_test.rb");
+ "admin_test.rb,table_test.rb,quotas_test.rb,admin2_test.rb");
Review comment:
What's the story of `admin2_test.rb`, why is it related to this fix?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22806) Recreating a deleted column family brings back the deleted cells
[ https://issues.apache.org/jira/browse/HBASE-22806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911060#comment-16911060 ] Pankaj Kumar commented on HBASE-22806: -- So actual problem is the stale CF. > Recreating a deleted column family brings back the deleted cells > > > Key: HBASE-22806 > URL: https://issues.apache.org/jira/browse/HBASE-22806 > Project: HBase > Issue Type: Bug > Components: API >Affects Versions: 2.1.3 > Environment: Scala > HBase Java Client > Mac/Linux >Reporter: Chao >Priority: Major > > Steps to reproduce the bug: > # Create a table with column family CF > # Add some cells C1, C2 in CF > # Remove CF using either: > ** TableDescriptorBuilder.removeColumnFamily() and Admin.modifyTable() > ** Admin.deleteColumnFamily() > # Create CF again > Expected: no cells (all cells marked as deleted) in CF > Actual: C1, C2 shows up automatically in CF > Extra information: in step 3, if doing Admin.modifyColumnFamily() without > actually changing anything, then after step 4 the cells won't come back. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22806) Recreating a deleted column family brings back the deleted cells
[ https://issues.apache.org/jira/browse/HBASE-22806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-22806: - Attachment: HBASE-22806_UT.patch > Recreating a deleted column family brings back the deleted cells > > > Key: HBASE-22806 > URL: https://issues.apache.org/jira/browse/HBASE-22806 > Project: HBase > Issue Type: Bug > Components: API >Affects Versions: 2.1.3 > Environment: Scala > HBase Java Client > Mac/Linux >Reporter: Chao >Priority: Major > Attachments: HBASE-22806_UT.patch > > > Steps to reproduce the bug: > # Create a table with column family CF > # Add some cells C1, C2 in CF > # Remove CF using either: > ** TableDescriptorBuilder.removeColumnFamily() and Admin.modifyTable() > ** Admin.deleteColumnFamily() > # Create CF again > Expected: no cells (all cells marked as deleted) in CF > Actual: C1, C2 shows up automatically in CF > Extra information: in step 3, if doing Admin.modifyColumnFamily() without > actually changing anything, then after step 4 the cells won't come back. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22806) Recreating a deleted column family brings back the deleted cells
[ https://issues.apache.org/jira/browse/HBASE-22806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911061#comment-16911061 ] Pankaj Kumar commented on HBASE-22806: -- Attached the UT patch. > Recreating a deleted column family brings back the deleted cells > > > Key: HBASE-22806 > URL: https://issues.apache.org/jira/browse/HBASE-22806 > Project: HBase > Issue Type: Bug > Components: API >Affects Versions: 2.1.3 > Environment: Scala > HBase Java Client > Mac/Linux >Reporter: Chao >Priority: Major > Attachments: HBASE-22806_UT.patch > > > Steps to reproduce the bug: > # Create a table with column family CF > # Add some cells C1, C2 in CF > # Remove CF using either: > ** TableDescriptorBuilder.removeColumnFamily() and Admin.modifyTable() > ** Admin.deleteColumnFamily() > # Create CF again > Expected: no cells (all cells marked as deleted) in CF > Actual: C1, C2 shows up automatically in CF > Extra information: in step 3, if doing Admin.modifyColumnFamily() without > actually changing anything, then after step 4 the cells won't come back. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] infraio commented on a change in pull request #510: HBASE-22878 Show table throttle quotas in table jsp
infraio commented on a change in pull request #510: HBASE-22878 Show table
throttle quotas in table jsp
URL: https://github.com/apache/hbase/pull/510#discussion_r315550602
##
File path: hbase-server/src/main/resources/hbase-webapps/master/table.jsp
##
@@ -77,6 +80,20 @@
setType(HBaseProtos.RegionSpecifier.RegionSpecifierType.REGION_NAME).
setValue(ByteString.copyFrom(regionInfo.getRegionName())).build()).build());
}
+
+ private List getThrottles(Configuration conf, QuotaFilter
filter)
+throws IOException {
Review comment:
Can we get the quota info from master?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-22882) TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts is flakey (was written flakey)
[
https://issues.apache.org/jira/browse/HBASE-22882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911095#comment-16911095
]
Hudson commented on HBASE-22882:
Results for branch master
[build #1346 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1346/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1346//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1346//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1346//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts is flakey (was
> written flakey)
> -
>
> Key: HBASE-22882
> URL: https://issues.apache.org/jira/browse/HBASE-22882
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: 22882.txt, Screen Shot 2019-08-19 at 12.46.33 PM.png,
> Screen Shot 2019-08-19 at 12.46.33 PM.png
>
>
> See HBASE-7536 which adds this test. The author and reviewers note the
> indeterminacy of the test. In branch-2.0 it fails 90% of the time on the GCE
> runs. Disabling it. In the original issue, its noted that it would take a
> bunch of work to make it deterministic. Meantime disabling it.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22857) Fix the failed ut TestHRegion and TestHRegionWithInMemoryFlush
[
https://issues.apache.org/jira/browse/HBASE-22857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911094#comment-16911094
]
Hudson commented on HBASE-22857:
Results for branch master
[build #1346 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1346/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1346//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1346//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1346//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix the failed ut TestHRegion and TestHRegionWithInMemoryFlush
> --
>
> Key: HBASE-22857
> URL: https://issues.apache.org/jira/browse/HBASE-22857
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.2.1
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Major
>
> Duplicate code in unit test...
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22875) TestShell and TestAdminShell2 are broken
[
https://issues.apache.org/jira/browse/HBASE-22875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911093#comment-16911093
]
Hudson commented on HBASE-22875:
Results for branch master
[build #1346 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1346/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1346//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1346//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1346//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestShell and TestAdminShell2 are broken
>
>
> Key: HBASE-22875
> URL: https://issues.apache.org/jira/browse/HBASE-22875
> Project: HBase
> Issue Type: Bug
> Components: shell, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22601) Misconfigured addition of peers leads to cluster shutdown.
[
https://issues.apache.org/jira/browse/HBASE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911100#comment-16911100
]
Hadoop QA commented on HBASE-22601:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
24s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
16s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
34s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 20s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}147m 34s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}203m 52s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush |
| | hadoop.hbase.regionserver.TestHRegion |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-499/17/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/499 |
| JIRA Issue | HBASE-22601 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 81e17bcbdf7a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-499/out/
[GitHub] [hbase] Apache-HBase commented on issue #499: [HBASE-22601] Misconfigured addition of peers leads to cluster shutdown.
Apache-HBase commented on issue #499: [HBASE-22601] Misconfigured addition of peers leads to cluster shutdown. URL: https://github.com/apache/hbase/pull/499#issuecomment-522908696 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 323 | master passed | | +1 | compile | 50 | master passed | | +1 | checkstyle | 73 | master passed | | +1 | shadedjars | 264 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 33 | master passed | | 0 | spotbugs | 256 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 254 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 306 | the patch passed | | +1 | compile | 53 | the patch passed | | +1 | javac | 53 | the patch passed | | +1 | checkstyle | 73 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 274 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 920 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 39 | the patch passed | | +1 | findbugs | 264 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 8854 | hbase-server in the patch failed. | | +1 | asflicense | 22 | The patch does not generate ASF License warnings. | | | | 12232 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | hadoop.hbase.regionserver.TestHRegion | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-499/17/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/499 | | JIRA Issue | HBASE-22601 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 81e17bcbdf7a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-499/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-499/17/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-499/17/testReport/ | | Max. process+thread count | 4969 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-499/17/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] openinx opened a new pull request #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
openinx opened a new pull request #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table URL: https://github.com/apache/hbase/pull/513 …threads in our cluster with thousands table This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] openinx commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
openinx commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table URL: https://github.com/apache/hbase/pull/513#issuecomment-522911427 PR an initial patch, need more consideration.. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien…
Apache-HBase commented on issue #361: HBase-22027: Split non-MR related parts of TokenUtil off into a Clien… URL: https://github.com/apache/hbase/pull/361#issuecomment-522913544 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 40 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 37 | Maven dependency ordering for branch | | +1 | mvninstall | 340 | master passed | | +1 | compile | 78 | master passed | | +1 | checkstyle | 103 | master passed | | +1 | shadedjars | 285 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 54 | master passed | | 0 | spotbugs | 248 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 312 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 15 | Maven dependency ordering for patch | | +1 | mvninstall | 304 | the patch passed | | +1 | compile | 75 | the patch passed | | +1 | javac | 75 | the patch passed | | +1 | checkstyle | 96 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 276 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 935 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 54 | the patch passed | | +1 | findbugs | 324 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 108 | hbase-client in the patch passed. | | -1 | unit | 8426 | hbase-server in the patch failed. | | +1 | asflicense | 49 | The patch does not generate ASF License warnings. | | | | 12296 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | hadoop.hbase.regionserver.TestHRegion | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/114/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/361 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 175d16426406 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-361/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/114/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/114/testReport/ | | Max. process+thread count | 4906 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-361/114/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #479: HBASE-22802 Avoid temp ByteBuffer allocation in FileIOEngine#read
Apache-HBase commented on issue #479: HBASE-22802 Avoid temp ByteBuffer allocation in FileIOEngine#read URL: https://github.com/apache/hbase/pull/479#issuecomment-522914330 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 38 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 34 | Maven dependency ordering for branch | | +1 | mvninstall | 330 | master passed | | +1 | compile | 81 | master passed | | +1 | checkstyle | 106 | master passed | | +1 | shadedjars | 290 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 57 | master passed | | 0 | spotbugs | 264 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 316 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 332 | the patch passed | | +1 | compile | 79 | the patch passed | | +1 | javac | 79 | the patch passed | | +1 | checkstyle | 25 | hbase-common: The patch generated 0 new + 48 unchanged - 2 fixed = 48 total (was 50) | | -1 | checkstyle | 78 | hbase-server: The patch generated 1 new + 48 unchanged - 0 fixed = 49 total (was 48) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 285 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1000 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 53 | the patch passed | | +1 | findbugs | 330 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 182 | hbase-common in the patch passed. | | -1 | unit | 8821 | hbase-server in the patch failed. | | +1 | asflicense | 46 | The patch does not generate ASF License warnings. | | | | 12902 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | hadoop.hbase.regionserver.TestHRegion | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/28/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/479 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 9fd0daba472a 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-479/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/28/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/28/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/28/testReport/ | | Max. process+thread count | 4750 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-479/28/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22882) TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts is flakey (was written flakey)
[
https://issues.apache.org/jira/browse/HBASE-22882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911107#comment-16911107
]
Hudson commented on HBASE-22882:
Results for branch branch-2
[build #2179 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2179/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2179//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2179//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2179//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts is flakey (was
> written flakey)
> -
>
> Key: HBASE-22882
> URL: https://issues.apache.org/jira/browse/HBASE-22882
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
>Assignee: stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6
>
> Attachments: 22882.txt, Screen Shot 2019-08-19 at 12.46.33 PM.png,
> Screen Shot 2019-08-19 at 12.46.33 PM.png
>
>
> See HBASE-7536 which adds this test. The author and reviewers note the
> indeterminacy of the test. In branch-2.0 it fails 90% of the time on the GCE
> runs. Disabling it. In the original issue, its noted that it would take a
> bunch of work to make it deterministic. Meantime disabling it.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22013) SpaceQuota utilization issue with region replicas enabled
[ https://issues.apache.org/jira/browse/HBASE-22013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911124#comment-16911124 ] Shardul Singh commented on HBASE-22013: --- Hi [~jatsakthi], can we push this in if everything is fine?..thanks :) > SpaceQuota utilization issue with region replicas enabled > - > > Key: HBASE-22013 > URL: https://issues.apache.org/jira/browse/HBASE-22013 > Project: HBase > Issue Type: Bug >Reporter: Ajeet Rai >Assignee: Shardul Singh >Priority: Major > Labels: Quota, Space > Fix For: 2.1.0 > > Attachments: HBASE-22013.master.001.patch, > HBASE-22013.master.002.patch > > > Space Quota: Space Quota Issue: If a table is created with region replica > then quota calculation is not happening > Steps: > 1: Create a table with 100 regions with region replica 3 > 2: Observe that 'hbase:quota' table doesn't have entry of usage for this > table So In UI only policy Limit and Policy is shown but not Usage and State. > Reason: > It looks like File system utilization core is sending data of 100 reasons > but not the size of region replicas. > But in quota observer chore, it is considering total region(actual regions+ > replica reasons) > So the ratio of reported regions is less then configured > percentRegionsReportedThreshold. > SO quota calculation is not happening -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22867) The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
[
https://issues.apache.org/jira/browse/HBASE-22867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911127#comment-16911127
]
Reid Chan commented on HBASE-22867:
---
First skimmed, as i said, the root cause is not the choice of FJP or TP (but i
do agree the `cap` of FJP is a concern).
The current pr will just end up piling up the BlockinqQueue of TP with
SnapshotHFileCleaner#getDeletableFiles tasks.
> The ForkJoinPool in CleanerChore will spawn thousands of threads in our
> cluster with thousands table
>
>
> Key: HBASE-22867
> URL: https://issues.apache.org/jira/browse/HBASE-22867
> Project: HBase
> Issue Type: Bug
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Critical
> Attachments: 31162.stack.1
>
>
> The thousands of spawned threads make the safepoint cost 80+s in our Master
> JVM processs.
> {code}
> 2019-08-15,19:35:35,861 INFO [main-SendThread(zjy-hadoop-prc-zk02.bj:11000)]
> org.apache.zookeeper.ClientCnxn: Client session timed out, have not heard
> from server in 82260ms for sessionid 0x1691332e2d3aae5, closing socket
> connection and at
> tempting reconnect
> {code}
> The stdout from JVM (can see from here there're 9126 threads & sync cost 80+s)
> {code}
> vmop[threads: total initially_running wait_to_block]
> [time: spin block sync cleanup vmop] page_trap_count
> 32358.859: ForceAsyncSafepoint [9126 67
> 474] [ 128 8659687 101] 0
> {code}
> Also we got the jstack:
> {code}
> $ cat 31162.stack.1 | grep 'ForkJoinPool-1-worker' | wc -l
> 8648
> {code}
> It's a dangerous bug, make it as blocker.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22867) The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
[
https://issues.apache.org/jira/browse/HBASE-22867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911136#comment-16911136
]
Zheng Hu commented on HBASE-22867:
--
I agree there're two problems here:
1. no limit for FJP;
2. the getDeletableFiles .
I plan to fix them in two separate issues, because seems two different problems.
Thanks.
> The ForkJoinPool in CleanerChore will spawn thousands of threads in our
> cluster with thousands table
>
>
> Key: HBASE-22867
> URL: https://issues.apache.org/jira/browse/HBASE-22867
> Project: HBase
> Issue Type: Bug
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Critical
> Attachments: 31162.stack.1
>
>
> The thousands of spawned threads make the safepoint cost 80+s in our Master
> JVM processs.
> {code}
> 2019-08-15,19:35:35,861 INFO [main-SendThread(zjy-hadoop-prc-zk02.bj:11000)]
> org.apache.zookeeper.ClientCnxn: Client session timed out, have not heard
> from server in 82260ms for sessionid 0x1691332e2d3aae5, closing socket
> connection and at
> tempting reconnect
> {code}
> The stdout from JVM (can see from here there're 9126 threads & sync cost 80+s)
> {code}
> vmop[threads: total initially_running wait_to_block]
> [time: spin block sync cleanup vmop] page_trap_count
> 32358.859: ForceAsyncSafepoint [9126 67
> 474] [ 128 8659687 101] 0
> {code}
> Also we got the jstack:
> {code}
> $ cat 31162.stack.1 | grep 'ForkJoinPool-1-worker' | wc -l
> 8648
> {code}
> It's a dangerous bug, make it as blocker.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
Apache-HBase commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table URL: https://github.com/apache/hbase/pull/513#issuecomment-522928629 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 141 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 474 | master passed | | +1 | compile | 76 | master passed | | +1 | checkstyle | 104 | master passed | | +1 | shadedjars | 382 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 46 | master passed | | 0 | spotbugs | 308 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 306 | master passed | ||| _ Patch Compile Tests _ | | -1 | mvninstall | 194 | root in the patch failed. | | -1 | compile | 20 | hbase-server in the patch failed. | | -1 | javac | 20 | hbase-server in the patch failed. | | -1 | checkstyle | 89 | hbase-server: The patch generated 2 new + 3 unchanged - 0 fixed = 5 total (was 3) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | -1 | shadedjars | 248 | patch has 10 errors when building our shaded downstream artifacts. | | -1 | hadoopcheck | 132 | The patch causes 10 errors with Hadoop v2.8.5. | | -1 | hadoopcheck | 271 | The patch causes 10 errors with Hadoop v2.9.2. | | -1 | hadoopcheck | 414 | The patch causes 10 errors with Hadoop v3.1.2. | | +1 | javadoc | 45 | the patch passed | | -1 | findbugs | 21 | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 24 | hbase-server in the patch failed. | | +1 | asflicense | 15 | The patch does not generate ASF License warnings. | | | | 2699 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/513 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 71ec5656a698 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-513/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | mvninstall | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/patch-mvninstall-root.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/patch-compile-hbase-server.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/patch-compile-hbase-server.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/diff-checkstyle-hbase-server.txt | | shadedjars | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/patch-shadedjars.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/patch-javac-2.8.5.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/patch-javac-2.9.2.txt | | hadoopcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/patch-javac-3.1.2.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/patch-findbugs-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/testReport/ | | Max. process+thread count | 86 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apac
[jira] [Commented] (HBASE-22867) The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
[
https://issues.apache.org/jira/browse/HBASE-22867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911159#comment-16911159
]
Reid Chan commented on HBASE-22867:
---
Good to know your plan.
> The ForkJoinPool in CleanerChore will spawn thousands of threads in our
> cluster with thousands table
>
>
> Key: HBASE-22867
> URL: https://issues.apache.org/jira/browse/HBASE-22867
> Project: HBase
> Issue Type: Bug
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Critical
> Attachments: 31162.stack.1
>
>
> The thousands of spawned threads make the safepoint cost 80+s in our Master
> JVM processs.
> {code}
> 2019-08-15,19:35:35,861 INFO [main-SendThread(zjy-hadoop-prc-zk02.bj:11000)]
> org.apache.zookeeper.ClientCnxn: Client session timed out, have not heard
> from server in 82260ms for sessionid 0x1691332e2d3aae5, closing socket
> connection and at
> tempting reconnect
> {code}
> The stdout from JVM (can see from here there're 9126 threads & sync cost 80+s)
> {code}
> vmop[threads: total initially_running wait_to_block]
> [time: spin block sync cleanup vmop] page_trap_count
> 32358.859: ForceAsyncSafepoint [9126 67
> 474] [ 128 8659687 101] 0
> {code}
> Also we got the jstack:
> {code}
> $ cat 31162.stack.1 | grep 'ForkJoinPool-1-worker' | wc -l
> 8648
> {code}
> It's a dangerous bug, make it as blocker.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache9 commented on a change in pull request #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
Apache9 commented on a change in pull request #513: HBASE-22867 The
ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster
with thousands table
URL: https://github.com/apache/hbase/pull/513#discussion_r315609444
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/cleaner/CleanerChore.java
##
@@ -423,16 +426,18 @@ protected Boolean compute() {
allFilesDeleted = deleteAction(() -> checkAndDeleteFiles(files),
"files");
}
+ List> futures = new ArrayList<>();
boolean allSubdirsDeleted = true;
if (!subDirs.isEmpty()) {
-List tasks =
Lists.newArrayListWithCapacity(subDirs.size());
sortByConsumedSpace(subDirs);
-for (FileStatus subdir : subDirs) {
- CleanerTask task = new CleanerTask(subdir, false);
- tasks.add(task);
- task.fork();
-}
-allSubdirsDeleted = deleteAction(() -> getCleanResult(tasks),
"subdirs");
+subDirs.forEach(dir -> futures.add(pool.execute(new CleanerTask(dir,
false;
Review comment:
I think submitting the task to the same pool is not safe here. In
ForkJoinPool, it is safe to call join as in the join method, the current thread
will try to execute tasks if there are any, so in general, we can always make
progress and finally finish all the tasks. But here, the current thread will be
blocked on the future.get, so it is possible that all the threads in the pool
are blocked on a future.get and cause dead lock here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #446: HBASE-22781 optimize the test case TestStoreScanner.testScanSameTimes…
Apache-HBase commented on issue #446: HBASE-22781 optimize the test case TestStoreScanner.testScanSameTimes… URL: https://github.com/apache/hbase/pull/446#issuecomment-522948721 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 138 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 497 | master passed | | +1 | compile | 84 | master passed | | +1 | checkstyle | 100 | master passed | | +1 | shadedjars | 369 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 47 | master passed | | 0 | spotbugs | 319 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 317 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 424 | the patch passed | | +1 | compile | 82 | the patch passed | | +1 | javac | 82 | the patch passed | | +1 | checkstyle | 105 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 373 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1348 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 43 | the patch passed | | +1 | findbugs | 283 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 13557 | hbase-server in the patch failed. | | +1 | asflicense | 26 | The patch does not generate ASF License warnings. | | | | 18247 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | hadoop.hbase.regionserver.TestHRegion | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-446/58/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/446 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c97a62226e38 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-446/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-446/58/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-446/58/testReport/ | | Max. process+thread count | 4590 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-446/58/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #512: HBASE-22881 Fix Non-daemon threads in HMaster
Apache-HBase commented on issue #512: HBASE-22881 Fix Non-daemon threads in HMaster URL: https://github.com/apache/hbase/pull/512#issuecomment-522950927 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 306 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | 0 | mvndep | 64 | Maven dependency ordering for branch | | +1 | mvninstall | 423 | master passed | | +1 | compile | 75 | master passed | | +1 | checkstyle | 103 | master passed | | +1 | shadedjars | 319 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 59 | master passed | | 0 | spotbugs | 247 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 294 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 17 | Maven dependency ordering for patch | | +1 | mvninstall | 389 | the patch passed | | +1 | compile | 79 | the patch passed | | +1 | javac | 79 | the patch passed | | -1 | checkstyle | 77 | hbase-server: The patch generated 1 new + 24 unchanged - 0 fixed = 25 total (was 24) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 316 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1166 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 54 | the patch passed | | +1 | findbugs | 326 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 168 | hbase-common in the patch passed. | | -1 | unit | 20278 | hbase-server in the patch failed. | | +1 | asflicense | 132 | The patch does not generate ASF License warnings. | | | | 25103 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.procedure.TestTruncateTableProcedure | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.replication.TestReplicationKillSlaveRSWithSeparateOldWALs | | | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.client.TestAsyncTableAdminApi | | | hadoop.hbase.tool.TestSecureBulkLoadHFiles | | | hadoop.hbase.client.TestCloneSnapshotFromClientNormal | | | hadoop.hbase.client.TestSnapshotFromClient | | | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.client.TestFromClientSide3 | | | hadoop.hbase.security.access.TestSnapshotScannerHDFSAclController | | | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.regionserver.TestHRegion | | | hadoop.hbase.master.assignment.TestMergeTableRegionsProcedure | | | hadoop.hbase.tool.TestBulkLoadHFiles | | | hadoop.hbase.namespace.TestNamespaceAuditor | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-512/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/512 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 1fd4421eeba2 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-512/out/precommit/personality/provided.sh | | git revision | master / 67382809d3 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-512/1/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-512/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-512/1/testReport/ | | Max. process+thread count | 5106 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-512/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbug
[GitHub] [hbase] mymeiyi commented on a change in pull request #511: HBASE-22879 user_permission command failed to show global permission
mymeiyi commented on a change in pull request #511: HBASE-22879 user_permission
command failed to show global permission
URL: https://github.com/apache/hbase/pull/511#discussion_r315618423
##
File path:
hbase-shell/src/test/java/org/apache/hadoop/hbase/client/TestShell.java
##
@@ -35,7 +35,7 @@
@Test
public void testRunShellTests() throws IOException {
System.setProperty("shell.test.exclude",
"replication_admin_test.rb,rsgroup_shell_test.rb," +
- "admin_test.rb,table_test.rb,quotas_test.rb");
+ "admin_test.rb,table_test.rb,quotas_test.rb,admin2_test.rb");
Review comment:
It's not related to this fix. But admin2_test.rb is already tested in
TestAdminShell2, TestShell can skip it rather than test it twice?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-20607) hbase namespace table is not created/assigned to correct rsgroup
[
https://issues.apache.org/jira/browse/HBASE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911203#comment-16911203
]
Ashok shetty commented on HBASE-20607:
--
i tried the same thing but it a little different way it worked
steps
add_rsgroup 'my_group'
move_servers_rsgroup 'my_group',['server:port']
create_namespace 'test_namespace', {'hbase.rsgroup.name'=＞'my_group'}
create 'test_namespace:t11', 'f1'
create 'test_namespace:t12', 'f1'
create 'test_namespace:t13', 'f1'
now whatever tables you create under test_namespace it will under my_group
> hbase namespace table is not created/assigned to correct rsgroup
>
>
> Key: HBASE-20607
> URL: https://issues.apache.org/jira/browse/HBASE-20607
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
> Labels: quota, rsgroup
>
> This is a follow-up ticket of HBASE-20566
> * Create a {{rsgroup}} and assign {{hbase}} namespace to the new {{rsgroup}}
> * Enable {{quota}} feature by adding the property hbase.quota.enable
> and restarting {{hbase}}
> * By listing the {{rsgroups}}, the {{hbase:quota}} will listed under the
> {{default}} rsgroup.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (HBASE-20607) hbase namespace table is not created/assigned to correct rsgroup
[
https://issues.apache.org/jira/browse/HBASE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911203#comment-16911203
]
Ashok shetty edited comment on HBASE-20607 at 8/20/19 10:38 AM:
i tried the same thing but it a little different way it worked
steps
add_rsgroup 'my_group'
move_servers_rsgroup 'my_group',['server:port']
create_namespace 'test_namespace', {'hbase.rsgroup.name'=＞'my_group'}
create 'test_namespace:t11', 'f1'
create 'test_namespace:t12', 'f1'
create 'test_namespace:t13', 'f1'
now whatever tables you create under test_namespace it will added under my_group
was (Author: ashok_shetty):
i tried the same thing but it a little different way it worked
steps
add_rsgroup 'my_group'
move_servers_rsgroup 'my_group',['server:port']
create_namespace 'test_namespace', {'hbase.rsgroup.name'=＞'my_group'}
create 'test_namespace:t11', 'f1'
create 'test_namespace:t12', 'f1'
create 'test_namespace:t13', 'f1'
now whatever tables you create under test_namespace it will under my_group
> hbase namespace table is not created/assigned to correct rsgroup
>
>
> Key: HBASE-20607
> URL: https://issues.apache.org/jira/browse/HBASE-20607
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
> Labels: quota, rsgroup
>
> This is a follow-up ticket of HBASE-20566
> * Create a {{rsgroup}} and assign {{hbase}} namespace to the new {{rsgroup}}
> * Enable {{quota}} feature by adding the property hbase.quota.enable
> and restarting {{hbase}}
> * By listing the {{rsgroups}}, the {{hbase:quota}} will listed under the
> {{default}} rsgroup.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #510: HBASE-22878 Show table throttle quotas in table jsp
Apache-HBase commented on issue #510: HBASE-22878 Show table throttle quotas in table jsp URL: https://github.com/apache/hbase/pull/510#issuecomment-522959841 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 63 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | 0 | mvndep | 33 | Maven dependency ordering for branch | | +1 | mvninstall | 317 | master passed | | +1 | compile | 78 | master passed | | +1 | checkstyle | 105 | master passed | | +1 | shadedjars | 271 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 57 | master passed | | 0 | spotbugs | 235 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 304 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 15 | Maven dependency ordering for patch | | +1 | mvninstall | 305 | the patch passed | | +1 | compile | 77 | the patch passed | | +1 | javac | 77 | the patch passed | | +1 | checkstyle | 104 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 274 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 960 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 51 | the patch passed | | +1 | findbugs | 319 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 97 | hbase-client in the patch passed. | | -1 | unit | 17534 | hbase-server in the patch failed. | | +1 | asflicense | 53 | The patch does not generate ASF License warnings. | | | | 21419 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.replication.TestReplicationSyncUpTool | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.regionserver.TestHRegion | | | hadoop.hbase.util.TestFromClientSide3WoUnsafe | | | hadoop.hbase.client.TestFromClientSide3 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/510 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 7eb71da45323 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-510/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/5/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/5/testReport/ | | Max. process+thread count | 5278 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-510/5/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Issue Comment Deleted] (HBASE-20607) hbase namespace table is not created/assigned to correct rsgroup
[
https://issues.apache.org/jira/browse/HBASE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashok shetty updated HBASE-20607:
-
Comment: was deleted
(was: i tried the same thing but it a little different way it worked
steps
add_rsgroup 'my_group'
move_servers_rsgroup 'my_group',['server:port']
create_namespace 'test_namespace', {'hbase.rsgroup.name'=＞'my_group'}
create 'test_namespace:t11', 'f1'
create 'test_namespace:t12', 'f1'
create 'test_namespace:t13', 'f1'
now whatever tables you create under test_namespace it will added under
my_group)
> hbase namespace table is not created/assigned to correct rsgroup
>
>
> Key: HBASE-20607
> URL: https://issues.apache.org/jira/browse/HBASE-20607
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
> Labels: quota, rsgroup
>
> This is a follow-up ticket of HBASE-20566
> * Create a {{rsgroup}} and assign {{hbase}} namespace to the new {{rsgroup}}
> * Enable {{quota}} feature by adding the property hbase.quota.enable
> and restarting {{hbase}}
> * By listing the {{rsgroups}}, the {{hbase:quota}} will listed under the
> {{default}} rsgroup.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-20607) hbase namespace table is not created/assigned to correct rsgroup
[
https://issues.apache.org/jira/browse/HBASE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911219#comment-16911219
]
Ashok shetty commented on HBASE-20607:
--
i tried the same thing but it a little different way it worked steps
add_rsgroup 'my_group'
move_servers_rsgroup 'my_group',['server:port'] create_namespace
'test_namespace', {'hbase.rsgroup.name'=＞'my_group'}
create 'test_namespace:t11', 'f1'
create 'test_namespace:t12', 'f1'
create 'test_namespace:t13', 'f1'
now whatever tables you create under test_namespace it will added under
my_group)
> hbase namespace table is not created/assigned to correct rsgroup
>
>
> Key: HBASE-20607
> URL: https://issues.apache.org/jira/browse/HBASE-20607
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
> Labels: quota, rsgroup
>
> This is a follow-up ticket of HBASE-20566
> * Create a {{rsgroup}} and assign {{hbase}} namespace to the new {{rsgroup}}
> * Enable {{quota}} feature by adding the property hbase.quota.enable
> and restarting {{hbase}}
> * By listing the {{rsgroups}}, the {{hbase:quota}} will listed under the
> {{default}} rsgroup.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #477: HBASE-22834 Removed deprecated methods from HBaseTestingUtility
Apache-HBase commented on issue #477: HBASE-22834 Removed deprecated methods from HBaseTestingUtility URL: https://github.com/apache/hbase/pull/477#issuecomment-522968039 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 98 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 46 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 348 | master passed | | +1 | compile | 56 | master passed | | +1 | checkstyle | 99 | master passed | | +1 | shadedjars | 300 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 38 | master passed | | 0 | spotbugs | 277 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 275 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 326 | the patch passed | | +1 | compile | 56 | the patch passed | | +1 | javac | 56 | the patch passed | | +1 | checkstyle | 96 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 294 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1025 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 36 | the patch passed | | +1 | findbugs | 297 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 18417 | hbase-server in the patch failed. | | -1 | asflicense | 32 | The patch generated 19 ASF License warnings. | | | | 22206 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.TestInfoServers | | | hadoop.hbase.master.TestMasterAbortAndRSGotKilled | | | hadoop.hbase.client.TestAsyncTableAdminApi2 | | | hadoop.hbase.replication.regionserver.TestWALEntryStream | | | hadoop.hbase.client.TestMobCloneSnapshotFromClientCloneLinksAfterDelete | | | hadoop.hbase.client.TestServerBusyException | | | hadoop.hbase.replication.TestMultiSlaveReplication | | | hadoop.hbase.snapshot.TestSnapshotWhenChoreCleaning | | | hadoop.hbase.client.TestMobCloneSnapshotFromClientNormal | | | hadoop.hbase.master.snapshot.TestSnapshotWhileRSCrashes | | | hadoop.hbase.regionserver.TestMobStoreScanner | | | hadoop.hbase.regionserver.TestRegionReplicasWithRestartScenarios | | | hadoop.hbase.regionserver.wal.TestWALReplay | | | hadoop.hbase.security.access.TestAccessController3 | | | hadoop.hbase.master.snapshot.TestSnapshotFileCache | | | hadoop.hbase.replication.TestReplicationStatusSourceStartedTargetStoppedNoOps | | | hadoop.hbase.client.TestAsyncTableRSCrashPublish | | | hadoop.hbase.master.TestMetaAssignmentWithStopMaster | | | hadoop.hbase.client.TestMvccConsistentScanner | | | hadoop.hbase.replication.TestRemoveFromSerialReplicationPeer | | | hadoop.hbase.replication.TestReplicationWithTags | | | hadoop.hbase.replication.regionserver.TestGlobalReplicationThrottler | | | hadoop.hbase.client.TestCIPutOperationTimeout | | | hadoop.hbase.replication.regionserver.TestReplicationSource | | | hadoop.hbase.client.TestAsyncAdminWithRegionReplicas | | | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.security.visibility.TestVisibilityLablesWithGroups | | | hadoop.hbase.master.TestRetainAssignmentOnRestartSplitWithoutZk | | | hadoop.hbase.client.TestMultipleTimestamps | | | hadoop.hbase.master.TestMasterMetrics | | | hadoop.hbase.master.TestMasterMetricsWrapper | | | hadoop.hbase.client.TestMobRestoreSnapshotFromClientClone | | | hadoop.hbase.client.TestCloneSnapshotFromClientNormal | | | hadoop.hbase.tool.TestBulkLoadHFiles | | | hadoop.hbase.client.TestScannersFromClientSide | | | hadoop.hbase.client.TestCIGetRpcTimeout | | | hadoop.hbase.regionserver.TestDeleteMobTable | | | hadoop.hbase.client.TestTableOperationException | | | hadoop.hbase.regionserver.TestRegionServerReportForDuty | | | hadoop.hbase.backup.TestHFileArchiving | | | hadoop.hbase.client.TestMobSnapshotCloneIndependence | | | hadoop.hbase.regionserver.TestRegionMove | | | hadoop.hbase.regionserver.TestHRegion | | | hadoop.hbase.security.access.TestScanEarlyTermination | | | hadoop.hbase.replication.TestPerTableCFReplication | | | hadoop.hbase.io.encoding.TestChangingEncoding | | | hadoop.hbase.master.procedure.TestModifyTableProcedure | | | hadoop.hbase.client.TestFlushFromClient | | | hadoop.hbase.master.procedure.TestIgnoreUnknownFamily | | | hadoop.hbase.
[jira] [Comment Edited] (HBASE-20607) hbase namespace table is not created/assigned to correct rsgroup
[
https://issues.apache.org/jira/browse/HBASE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911219#comment-16911219
]
Ashok shetty edited comment on HBASE-20607 at 8/20/19 11:20 AM:
I tried similar scenario as mentioned below,
*Steps:*
1. add_rsgroup 'my_group'
2. move_servers_rsgroup 'my_group',['server:port']
3. create_namespace 'ns1'
4. create 'ns1:t1', 'f1'
5. create 'ns1:t2', 'f1'
6. create 'ns1:t3', 'f1'
7. move_namespaces_rsgroup 'my_group',['ns1']
8. create 'ns1:t4', 'f1'
At step-6, all tables under ''ns1' were assigned to "default" RSGroup
At step-7, all tables under ''ns1' were moved to my_group
New tables which are created after step-7 are assigned to 'default' RSGroup.
was (Author: ashok_shetty):
i tried the same thing but it a little different way it worked steps
add_rsgroup 'my_group'
move_servers_rsgroup 'my_group',['server:port'] create_namespace
'test_namespace', {'hbase.rsgroup.name'=＞'my_group'}
create 'test_namespace:t11', 'f1'
create 'test_namespace:t12', 'f1'
create 'test_namespace:t13', 'f1'
now whatever tables you create under test_namespace it will added under
my_group)
> hbase namespace table is not created/assigned to correct rsgroup
>
>
> Key: HBASE-20607
> URL: https://issues.apache.org/jira/browse/HBASE-20607
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
> Labels: quota, rsgroup
>
> This is a follow-up ticket of HBASE-20566
> * Create a {{rsgroup}} and assign {{hbase}} namespace to the new {{rsgroup}}
> * Enable {{quota}} feature by adding the property hbase.quota.enable
> and restarting {{hbase}}
> * By listing the {{rsgroups}}, the {{hbase:quota}} will listed under the
> {{default}} rsgroup.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #505: HBASE-22863 : Cleanup transitive Jackson1 vulnerable dependencies(forward-port HBASE-22728)
Apache-HBase commented on issue #505: HBASE-22863 : Cleanup transitive Jackson1 vulnerable dependencies(forward-port HBASE-22728) URL: https://github.com/apache/hbase/pull/505#issuecomment-522972641 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 78 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | 0 | mvndep | 42 | Maven dependency ordering for branch | | +1 | mvninstall | 439 | master passed | | +1 | compile | 243 | master passed | | +1 | shadedjars | 371 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 321 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 15 | Maven dependency ordering for patch | | +1 | mvninstall | 409 | the patch passed | | +1 | compile | 233 | the patch passed | | +1 | javac | 233 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | xml | 14 | The patch has no ill-formed XML file. | | +1 | shadedjars | 377 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1250 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 307 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 19171 | root in the patch failed. | | +1 | asflicense | 182 | The patch does not generate ASF License warnings. | | | | 23983 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.TestHRegion | | | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | hadoop.hbase.tool.TestSecureBulkLoadHFiles | | | hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster | | | hadoop.hbase.tool.TestBulkLoadHFiles | | | hadoop.hbase.client.TestAsyncTableBatch | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.client.TestFromClientSide3 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-505/13/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/505 | | Optional Tests | dupname asflicense javac javadoc unit shadedjars hadoopcheck xml compile | | uname | Linux 23c254d1b86e 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-505/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-505/13/artifact/out/patch-unit-root.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-505/13/testReport/ | | Max. process+thread count | 4933 (vs. ulimit of 1) | | modules | C: hbase-server hbase-mapreduce hbase-shaded/hbase-shaded-client-byo-hadoop hbase-shaded/hbase-shaded-mapreduce hbase-shaded/hbase-shaded-testing-util hbase-shaded/hbase-shaded-testing-util-tester . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-505/13/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-20607) hbase namespace table is not created/assigned to correct rsgroup
[
https://issues.apache.org/jira/browse/HBASE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911239#comment-16911239
]
Pankaj Kumar commented on HBASE-20607:
--
HBASE-20607 is about the system namespace table region assignment to the
specified group, [~nihaljain.cs] /[~anoop.hbase] let's finalize the solution
(solve this either in postStartMaster() or RSGroupBasedLoadBalancer).
Thanks [~Ashok_shetty] for sharing the observation. Currently
move_namespaces_rsgroup shell command move the namespace tables to new
destination group using RSGroupAdmin#moveTables(Set tables, String
targetGroup), which will move all the tables under the defined namespace to the
new group. New tables will not assigned as move_namespaces_rsgroup doesn't
modify the RSGroup in namespace descriptor.
> hbase namespace table is not created/assigned to correct rsgroup
>
>
> Key: HBASE-20607
> URL: https://issues.apache.org/jira/browse/HBASE-20607
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
> Labels: quota, rsgroup
>
> This is a follow-up ticket of HBASE-20566
> * Create a {{rsgroup}} and assign {{hbase}} namespace to the new {{rsgroup}}
> * Enable {{quota}} feature by adding the property hbase.quota.enable
> and restarting {{hbase}}
> * By listing the {{rsgroups}}, the {{hbase:quota}} will listed under the
> {{default}} rsgroup.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #509: HBASE-22877 WebHDFS based export snapshot will fail if hfile is in archive directory
Apache-HBase commented on issue #509: HBASE-22877 WebHDFS based export snapshot will fail if hfile is in archive directory URL: https://github.com/apache/hbase/pull/509#issuecomment-522979325 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 71 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 413 | master passed | | +1 | compile | 68 | master passed | | +1 | checkstyle | 95 | master passed | | +1 | shadedjars | 346 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 40 | master passed | | 0 | spotbugs | 284 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 281 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 333 | the patch passed | | +1 | compile | 59 | the patch passed | | +1 | javac | 59 | the patch passed | | -1 | checkstyle | 81 | hbase-server: The patch generated 1 new + 12 unchanged - 0 fixed = 13 total (was 12) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 301 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1215 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 39 | the patch passed | | +1 | findbugs | 326 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 20941 | hbase-server in the patch failed. | | +1 | asflicense | 88 | The patch does not generate ASF License warnings. | | | | 25224 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.tool.TestBulkLoadHFiles | | | hadoop.hbase.master.procedure.TestTruncateTableProcedure | | | hadoop.hbase.replication.multiwal.TestReplicationSyncUpToolWithMultipleWAL | | | hadoop.hbase.replication.TestReplicationKillSlaveRS | | | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.master.procedure.TestProcedurePriority | | | hadoop.hbase.regionserver.TestHRegion | | | hadoop.hbase.client.TestFromClientSide3 | | | hadoop.hbase.replication.TestReplicationKillSlaveRSWithSeparateOldWALs | | | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.tool.TestSecureBulkLoadHFiles | | | hadoop.hbase.security.access.TestSnapshotScannerHDFSAclController | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-509/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/509 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d5ac272a1ad6 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-509/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-509/5/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-509/5/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-509/5/testReport/ | | Max. process+thread count | 4940 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-509/5/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific commen
[GitHub] [hbase] Apache-HBase commented on issue #487: HBASE-22844 Fixed Checkstyle violations in client snapshot exceptions
Apache-HBase commented on issue #487: HBASE-22844 Fixed Checkstyle violations in client snapshot exceptions URL: https://github.com/apache/hbase/pull/487#issuecomment-522980200 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 73 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 363 | master passed | | +1 | compile | 27 | master passed | | +1 | checkstyle | 38 | master passed | | +1 | shadedjars | 271 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 34 | master passed | | 0 | spotbugs | 103 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 101 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 390 | the patch passed | | +1 | compile | 30 | the patch passed | | +1 | javac | 30 | the patch passed | | +1 | checkstyle | 38 | hbase-client: The patch generated 0 new + 0 unchanged - 13 fixed = 0 total (was 13) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 352 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1366 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 29 | the patch passed | | +1 | findbugs | 95 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 115 | hbase-client in the patch passed. | | +1 | asflicense | 15 | The patch does not generate ASF License warnings. | | | | 3859 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-487/21/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/487 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6196fa97e571 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-487/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-487/21/testReport/ | | Max. process+thread count | 263 (vs. ulimit of 1) | | modules | C: hbase-client U: hbase-client | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-487/21/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #478: HBASE-22755 Removed deprecated methods from Mutation
Apache-HBase commented on issue #478: HBASE-22755 Removed deprecated methods from Mutation URL: https://github.com/apache/hbase/pull/478#issuecomment-522982163 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 60 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | 0 | mvndep | 33 | Maven dependency ordering for branch | | +1 | mvninstall | 389 | master passed | | +1 | compile | 89 | master passed | | +1 | checkstyle | 126 | master passed | | +1 | shadedjars | 295 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 62 | master passed | | 0 | spotbugs | 262 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 330 | master passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 19 | Maven dependency ordering for patch | | +1 | mvninstall | 367 | the patch passed | | +1 | compile | 82 | the patch passed | | +1 | javac | 82 | the patch passed | | +1 | checkstyle | 111 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 335 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1122 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 58 | the patch passed | | +1 | findbugs | 362 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 108 | hbase-client in the patch passed. | | -1 | unit | 20676 | hbase-server in the patch failed. | | +1 | asflicense | 66 | The patch does not generate ASF License warnings. | | | | 25113 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.util.TestFromClientSide3WoUnsafe | | | hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.replication.TestReplicationKillSlaveRSWithSeparateOldWALs | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.client.TestAsyncTableAdminApi | | | hadoop.hbase.client.TestAsyncTableAdminApi3 | | | hadoop.hbase.client.TestCloneSnapshotFromClientNormal | | | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.client.TestFromClientSide3 | | | hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush | | | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.TestRegionRebalancing | | | hadoop.hbase.regionserver.TestHRegion | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-478/26/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/478 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e69dd641c233 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-478/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-478/26/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-478/26/testReport/ | | Max. process+thread count | 4920 (vs. ulimit of 1) | | modules | C: hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-478/26/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-20607) hbase namespace table is not created/assigned to correct rsgroup
[
https://issues.apache.org/jira/browse/HBASE-20607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911272#comment-16911272
]
Pankaj Kumar commented on HBASE-20607:
--
[~anoop.hbase] I wanted to know about the scope of *move_namespaces_rsgroup*
and other commands also. Are these commands doing things temporarily, as I can
see shell commands dont modify the *hbase:rsgroup* table and so new
instructions are not honored ** by RSGroupBasedLoadBalancer.
For example, if *move_namespaces_rsgroup* alter the namespace by setting
RSGroup as *my_group* then new tables regions under *ns1* will be assigned to
*my_group*.
Just a thought, kindly provide your opinion.
> hbase namespace table is not created/assigned to correct rsgroup
>
>
> Key: HBASE-20607
> URL: https://issues.apache.org/jira/browse/HBASE-20607
> Project: HBase
> Issue Type: Bug
> Components: master
>Reporter: Biju Nair
>Priority: Major
> Labels: quota, rsgroup
>
> This is a follow-up ticket of HBASE-20566
> * Create a {{rsgroup}} and assign {{hbase}} namespace to the new {{rsgroup}}
> * Enable {{quota}} feature by adding the property hbase.quota.enable
> and restarting {{hbase}}
> * By listing the {{rsgroups}}, the {{hbase:quota}} will listed under the
> {{default}} rsgroup.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] openinx commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
openinx commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table URL: https://github.com/apache/hbase/pull/513#issuecomment-523010301 Ping @Apache9 , I've rewriten the cleaner , please take a look again. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] openinx commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
openinx commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table URL: https://github.com/apache/hbase/pull/513#issuecomment-523019091 Seems still some problem, the UT are easy to hang now, Let me check. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22738) Fallback to default group to choose RS when there are no RS in current group
[ https://issues.apache.org/jira/browse/HBASE-22738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pankaj Kumar updated HBASE-22738: - Component/s: rsgroup > Fallback to default group to choose RS when there are no RS in current group > > > Key: HBASE-22738 > URL: https://issues.apache.org/jira/browse/HBASE-22738 > Project: HBase > Issue Type: Bug > Components: rsgroup >Reporter: Guanghao Zhang >Priority: Major > > We configure one regionserver for hbase system table. But when rolling > upgrade, you need move the region to other regionservers. But because there > are no other regionservers in this group, you cannot move the region... > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22884) Disable builds on EOL'd branch-2.0; undo nightly build
[
https://issues.apache.org/jira/browse/HBASE-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911368#comment-16911368
]
Sean Busbey commented on HBASE-22884:
-
you have to remove all of these files to disable all builds for a branch:
* {{dev-support/Jenkinsfile}}
* {{dev-support/Jenkinsfile_GitHub}}
* {{dev-support/flaky-tests/flaky-reporting.Jenkinsfile}}
* {{dev-support/flaky-tests/run-flaky-tests.Jenkinsfile}}
> Disable builds on EOL'd branch-2.0; undo nightly build
> --
>
> Key: HBASE-22884
> URL: https://issues.apache.org/jira/browse/HBASE-22884
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>Priority: Major
>
> I think all I have to do is remove the dev-support/Jenkinsfile (IIRC).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] openinx commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
openinx commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table URL: https://github.com/apache/hbase/pull/513#issuecomment-523025130 OK, I got the problem, forget to complete the future somewhere.. it's easy to fix . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] busbey commented on issue #16: HBASE-22825 [HBCK2] Add a client-side to hbase-operator-tools that ca…
busbey commented on issue #16: HBASE-22825 [HBCK2] Add a client-side to hbase-operator-tools that ca… URL: https://github.com/apache/hbase-operator-tools/pull/16#issuecomment-523026720 Do we require the change to bump the HBase 2.1 version to 2.1.6 specifically? If so we'll have to wait for the release. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-21879) Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
[
https://issues.apache.org/jira/browse/HBASE-21879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911389#comment-16911389
]
Hudson commented on HBASE-21879:
Results for branch HBASE-21879
[build #214 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/214/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/214//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/214//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-21879/214//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Read HFile's block to ByteBuffer directly instead of to byte for reducing
> young gc purpose
> --
>
> Key: HBASE-21879
> URL: https://issues.apache.org/jira/browse/HBASE-21879
> Project: HBase
> Issue Type: Improvement
>Reporter: Zheng Hu
>Assignee: Zheng Hu
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: HBASE-21879.v1.patch, HBASE-21879.v1.patch,
> QPS-latencies-before-HBASE-21879.png, gc-data-before-HBASE-21879.png
>
>
> In HFileBlock#readBlockDataInternal, we have the following:
> {code}
> @VisibleForTesting
> protected HFileBlock readBlockDataInternal(FSDataInputStream is, long offset,
> long onDiskSizeWithHeaderL, boolean pread, boolean verifyChecksum,
> boolean updateMetrics)
> throws IOException {
> // .
> // TODO: Make this ByteBuffer-based. Will make it easier to go to HDFS with
> BBPool (offheap).
> byte [] onDiskBlock = new byte[onDiskSizeWithHeader + hdrSize];
> int nextBlockOnDiskSize = readAtOffset(is, onDiskBlock, preReadHeaderSize,
> onDiskSizeWithHeader - preReadHeaderSize, true, offset +
> preReadHeaderSize, pread);
> if (headerBuf != null) {
> // ...
> }
> // ...
> }
> {code}
> In the read path, we still read the block from hfile to on-heap byte[], then
> copy the on-heap byte[] to offheap bucket cache asynchronously, and in my
> 100% get performance test, I also observed some frequent young gc, The
> largest memory footprint in the young gen should be the on-heap block byte[].
> In fact, we can read HFile's block to ByteBuffer directly instead of to
> byte[] for reducing young gc purpose. we did not implement this before,
> because no ByteBuffer reading interface in the older HDFS client, but 2.7+
> has supported this now, so we can fix this now. I think.
> Will provide an patch and some perf-comparison for this.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] openinx commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
openinx commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table URL: https://github.com/apache/hbase/pull/513#issuecomment-523034709 Fixed all the UT in cleanerChore, should be good now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r315718509 ## File path: hbase-hbtop/README.md ## @@ -0,0 +1,247 @@ + + +# hbtop + +## Overview + +`hbtop` is a real-time monitoring tool for HBase like Unix's top command. +It can display summary information as well as metrics per Region/Namespace/Table/RegionServer. +In this tool, you can see the metrics sorted by a selected field and filter the metrics to see only metrics you really want to see. +Also, with the drill-down feature, you can find hot regions easily in a top-down manner. + +## Usage + +You can run hbtop by the following command: Review comment: Thanks. Modified. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r315718618 ## File path: hbase-hbtop/README.md ## @@ -0,0 +1,247 @@ + + +# hbtop + +## Overview + +`hbtop` is a real-time monitoring tool for HBase like Unix's top command. +It can display summary information as well as metrics per Region/Namespace/Table/RegionServer. +In this tool, you can see the metrics sorted by a selected field and filter the metrics to see only metrics you really want to see. +Also, with the drill-down feature, you can find hot regions easily in a top-down manner. + +## Usage + +You can run hbtop by the following command: + +``` +$ hbase top +``` + +Also, you can specify your own zookeeper quorum and znode parent like the following: Review comment: Thanks. Modified. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r315718978 ## File path: hbase-hbtop/README.md ## @@ -0,0 +1,247 @@ + + +# hbtop + +## Overview + +`hbtop` is a real-time monitoring tool for HBase like Unix's top command. +It can display summary information as well as metrics per Region/Namespace/Table/RegionServer. +In this tool, you can see the metrics sorted by a selected field and filter the metrics to see only metrics you really want to see. +Also, with the drill-down feature, you can find hot regions easily in a top-down manner. + +## Usage + +You can run hbtop by the following command: + +``` +$ hbase top +``` + +Also, you can specify your own zookeeper quorum and znode parent like the following: + +``` +$ hbase top -Dhbase.client.zookeeper.quorum= -Dzookeeper.znode.parent= +``` + + + +The top screen consists of the summary part and the metrics part. +In the summary part, you can see `HBase Version`, `Cluster ID`, `The number of region servers`, `Region count`, `Average Cluster Load` and `Aggregated Request/s`. +In the metrics part, you can see metrics per Region/Namespace/Table/RegionServer depending on the selected mode. +The top screen is refreshed in a certain period – 3 seconds by default. + +### Scrolling metric records + +You can scroll the metric records in the metrics part. + + + +### Command line arguments + +| Argument | Description | +|---|---| +| -delay| The refresh delay (in seconds); default is 3 seconds | +| -help | Print usage; for help while the tool is running press `h` key | +| -mode | The mode; `n` (Namespace)|`t` (Table)|r (Region)|`s` (RegionServer), default is `r` (Region) | + +### Modes + +There are the following 4 modes in hbtop: + +| Mode | Description | +|---|---| +| Region | Showing metric records per region | +| Namespace | Showing metric records per namespace | +| Table | Showing metric records per table | +| RegionServer | Showing metric records per region server | + + Region mode + +In Region mode, the default sort field is `#REQ/S`. + +The fields in this mode are as follows: + +| Field | Description | Displayed by default | +|---|---|---| +| RNAME | Region Name | false | +| NAMESPACE | Namespace Name | true | +| TABLE | Table Name | true | +| SCODE | Start Code | false | +| REPID | Replica ID | false | +| REGION | Encoded Region Name | true | +| RS | Short Region Server Name | true | +| LRS | Long Region Server Name | false | +| #REQ/S | Request Count per second | true | +| #READ/S | Read Request Count per second | true | +| #FREAD/S | Filtered Read Request Count per second | true | +| #WRITE/S | Write Request Count per second | true | +| SF | StoreFile Size | true | +| USF | Uncompressed StoreFile Size | false | +| #SF | Number of StoreFiles | true | +| MEMSTORE | MemStore Size | true | +| LOCALITY | Block Locality | true | +| SKEY | Start Key | false | +| #COMPingCELL | Compacting Cell Count | false | +| #COMPedCELL | Compacted Cell Count | false | +| %COMP | Compaction Progress | false | +| LASTMCOMP | Last Major Compaction Time | false | + + Namespace mode + +In Namespace mode, the default sort field is `#REQ/S`. + +The fields in this mode are as follows: + +| Field | Description | Displayed by default | +|---|---|---| +| NAMESPACE | Namespace Name | true | +| #REGION | Region Count | true | +| #REQ/S | Request Count per second | true | +| #READ/S | Read Request Count per second | true | +| #FREAD/S | Filtered Read Request Count per second | true | +| #WRITE/S | Write Request Count per second | true | +| SF | StoreFile Size | true | +| USF | Uncompressed StoreFile Size | false | +| #SF | Number of StoreFiles | true | +| MEMSTORE | MemStore Size | true | + + Table mode + +In Table mode, the default sort field is `#REQ/S`. + +The fields in this mode are as follows: + +| Field | Description | Displayed by default | +|---|---|---| +| NAMESPACE | Namespace Name | true | +| TABLE | Table Name | true | +| #REGION | Region Count | true | +| #REQ/S | Request Count per second | true | +| #READ/S | Read Request Count per second | true | +| #FREAD/S | Filtered Read Request Count per second | true | +| #WRITE/S | Write Request Count per second | true | +| SF | StoreFile Size | true | +| USF | Uncompressed StoreFile Size | false | +| #SF | Number of StoreFiles | true | +| MEMSTORE | MemStore Size | true | + + RegionServer mode + +In RegionServer mode, the default sort field is `#REQ/S`. + +The fields in this mode are as follows: + +| Field | Description | Displayed by default | +|---|---|---| +| RS | Short Region Server Name | true | +| LRS | Long Region Server Name | false | +| #REGION | Region Count | true | +| #REQ/S | Request Count per second | true | +| #READ/S | Read Request Count per second | true
[GitHub] [hbase] brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop
brfrn169 commented on a change in pull request #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#discussion_r315718772 ## File path: hbase-hbtop/README.md ## @@ -0,0 +1,247 @@ + + +# hbtop + +## Overview + +`hbtop` is a real-time monitoring tool for HBase like Unix's top command. +It can display summary information as well as metrics per Region/Namespace/Table/RegionServer. +In this tool, you can see the metrics sorted by a selected field and filter the metrics to see only metrics you really want to see. +Also, with the drill-down feature, you can find hot regions easily in a top-down manner. + +## Usage + +You can run hbtop by the following command: + +``` +$ hbase top +``` + +Also, you can specify your own zookeeper quorum and znode parent like the following: + +``` +$ hbase top -Dhbase.client.zookeeper.quorum= -Dzookeeper.znode.parent= +``` + + + +The top screen consists of the summary part and the metrics part. Review comment: Thanks. Modified. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
Apache-HBase commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table URL: https://github.com/apache/hbase/pull/513#issuecomment-523041369 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 128 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 462 | master passed | | +1 | compile | 77 | master passed | | +1 | checkstyle | 99 | master passed | | +1 | shadedjars | 384 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 53 | master passed | | 0 | spotbugs | 339 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 336 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 426 | the patch passed | | +1 | compile | 74 | the patch passed | | +1 | javac | 74 | the patch passed | | +1 | checkstyle | 99 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 378 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1143 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 35 | the patch passed | | +1 | findbugs | 267 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 14097 | hbase-server in the patch failed. | | +1 | asflicense | 25 | The patch does not generate ASF License warnings. | | | | 18509 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestMobRestoreSnapshotFromClientClone | | | hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient | | | hadoop.hbase.client.TestRestoreSnapshotFromClientClone | | | hadoop.hbase.client.TestCloneSnapshotFromClientCloneLinksAfterDelete | | | hadoop.hbase.mob.compactions.TestMobCompactor | | | hadoop.hbase.client.TestMobCloneSnapshotFromClientCloneLinksAfterDelete | | | hadoop.hbase.master.cleaner.TestHFileLinkCleaner | | | hadoop.hbase.security.access.TestSnapshotScannerHDFSAclController | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/513 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux ad090ea2fa42 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-513/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/2/testReport/ | | Max. process+thread count | 4788 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22884) Disable builds on EOL'd branch-2.0; undo nightly build
[ https://issues.apache.org/jira/browse/HBASE-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911444#comment-16911444 ] stack commented on HBASE-22884: --- Thanks [~busbey] > Disable builds on EOL'd branch-2.0; undo nightly build > -- > > Key: HBASE-22884 > URL: https://issues.apache.org/jira/browse/HBASE-22884 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > > I think all I have to do is remove the dev-support/Jenkinsfile (IIRC). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
Apache-HBase commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table URL: https://github.com/apache/hbase/pull/513#issuecomment-523065430 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 260 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 398 | master passed | | +1 | compile | 62 | master passed | | +1 | checkstyle | 95 | master passed | | +1 | shadedjars | 302 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 39 | master passed | | 0 | spotbugs | 264 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 261 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 335 | the patch passed | | +1 | compile | 66 | the patch passed | | +1 | javac | 66 | the patch passed | | -1 | checkstyle | 125 | hbase-server: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 337 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1224 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 36 | the patch passed | | +1 | findbugs | 553 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 2844 | hbase-server in the patch failed. | | +1 | asflicense | 14 | The patch does not generate ASF License warnings. | | | | 7411 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.cleaner.TestCleanerChore | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/513 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 6831506eb8b3 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-513/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/3/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/3/testReport/ | | Max. process+thread count | 670 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (HBASE-22866) Multiple slf4j-log4j provider versions included in binary package (branch-1)
[
https://issues.apache.org/jira/browse/HBASE-22866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani reassigned HBASE-22866:
Assignee: Viraj Jasani
> Multiple slf4j-log4j provider versions included in binary package (branch-1)
>
>
> Key: HBASE-22866
> URL: https://issues.apache.org/jira/browse/HBASE-22866
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Andrew Purtell
>Assignee: Viraj Jasani
>Priority: Minor
> Fix For: 1.5.0
>
> Attachments: HBASE-22866.branch-1.000.patch,
> HBASE-22866.branch-1.000.patch
>
>
> Examining binary assembly results there are multiple versions of slf4j-log4j
> in lib/
> {noformat}
> slf4j-api-1.7.7.jar
> slf4j-log4j12-1.6.1.jar
> slf4j-log4j12-1.7.10.jar
> slf4j-log4j12-1.7.7.jar
> {noformat}
> We aren't managing slf4j-log4j12 dependency versions correctly, somehow.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] brfrn169 commented on issue #476: HBASE-11062 hbtop
brfrn169 commented on issue #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#issuecomment-523071494 Thank you for your comment. @saintstack > 1. We are adding this to hbase itself rather than to hbase-operator-tools; would be good if it could live outside of hbase. That said, I could go with having it committed to hbase initially and then moved out later if made sense. I think we can move it to hbase-operator-tools (and it's not hard actually), but one more reason why I want to add it to hbase itself is that we will be able to run hbtop without installation (we can just run `hbase top` command). I think this feature is useful for hbase administrators, so being able to use it easily may be wanted. > 2. We're adding a terminal and a console windowing system. Poking around I see some overlap but I don't know enough to say if doing it all ourselves is way to go or even if its possible to add this as is and then later backfill with libraries. For example, what if we wanted to add color to hbtop (does it do color?), would we have to implement it ourselves? Or if we wanted to change style? I was looking for a terminal UI library after deciding not to use Lanterna bacause of the license problem, but I couldn't find a good one. And I made an abstraction layer for Terminal (https://github.com/apache/hbase/pull/476/files#diff-c7618b5d136137dad5dfefe94d122a41), so we can create another implementation for terminal when we find a good library. And actually, I made basic features for colors (BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE: https://github.com/apache/hbase/pull/476/files#diff-8450df172184da9574f33a1db9e472d3) and styles (bold, blink, reverse, underline: https://github.com/apache/hbase/pull/476/files#diff-ad03be5d68e8d3eceabbd62c0af1e137) for the future. But someone might know a good library. So I will wait for others' comment. > I'd be game for committing this but can we get more reviewers? I see you asking for them and it is a big feature so it is hard to get the attention of busy folks. Have you put a request up on the dev list? Add a link to your fancy readme... so folks can see how fancy your lovely contrib is? Sure. Will put a review request on the dev list. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22631) assign failed may make gced parent region appear again !!!
[ https://issues.apache.org/jira/browse/HBASE-22631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-22631: - Attachment: HBASE-22631-branch-2.1-02.patch > assign failed may make gced parent region appear again !!! > -- > > Key: HBASE-22631 > URL: https://issues.apache.org/jira/browse/HBASE-22631 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Affects Versions: 2.1.1 >Reporter: yuhuiyang >Priority: Major > Attachments: HBASE-22631-branch-2.1-01.patch, > HBASE-22631-branch-2.1-02.patch, assign.png, assignProcedure.txt, > serverCrash.png, splitAndGc.png > > > When i assign a region A the process is as follows： > step1 : A is assigned to rs1 , and rs1 fails to open it . > step2 : assignProcedure handleFailure . > step3 : A is assign to rs2 and rs success to open it . > Above is the normal flow . However when rs1 is restart after the reigon A was > split and GCRegionProcedure was successed , the region A appare again ! > The region is that reigon A is not removed from the serverMap correctly when > assignprocedure handleFailure . Because the code regionNode.offline() make > the regionNode's regionLocation to be null and make regionNode's state to > OFFLINE . So when the code > env.getAssignmentManager().undoRegionAsOpening(regionNode) do nothing . So > when the rs1 restart event triggers a serverCrashProcedure, it will get > reigons from serverMap and it will get the region A then A will be assigned > and hdfs dir will be created. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22631) assign failed may make gced parent region appear again !!!
[
https://issues.apache.org/jira/browse/HBASE-22631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911470#comment-16911470
]
Wellington Chevreuil commented on HBASE-22631:
--
{quote}RegionNode.offline() will set the region state to State.OFFLINE and the
next step AssignmentManager().undoRegionAsOpening will check the region state
if state is in OPENING state .
{quote}
This is true indeed, but actually it's not enough to trigger the condition of
having the region brought back. This is because *RegionStateNode* objects
instances on *ServerStateNode.regions* are the same ones tracked on
*RegionStates.regionsMap*. So once the the region state is updated by any
procedure, such as *AssignProcedure* or *SplitProcedure* in this case, the
state will be reflected so it will not be taken into account if an SCP is
triggered. Also, once the SCP has completed successfully, the given entry for
this RS instance would had already been removed from *RegionStates.regionsMap*.
I had added a test that simulates the scenario described here, am attaching it
in a new patch. It succeeds even without the proposed fix, due to the
observations above. Nevertheless, the fact we may keep references in
*RegionStates.regionsMap* for regions not in the RS anymore seems like an
inconsistency, so I guess we can keep the proposed fix.
> assign failed may make gced parent region appear again !!!
> --
>
> Key: HBASE-22631
> URL: https://issues.apache.org/jira/browse/HBASE-22631
> Project: HBase
> Issue Type: Bug
> Components: proc-v2
>Affects Versions: 2.1.1
>Reporter: yuhuiyang
>Priority: Major
> Attachments: HBASE-22631-branch-2.1-01.patch,
> HBASE-22631-branch-2.1-02.patch, assign.png, assignProcedure.txt,
> serverCrash.png, splitAndGc.png
>
>
> When i assign a region A the process is as follows：
> step1 : A is assigned to rs1 , and rs1 fails to open it .
> step2 : assignProcedure handleFailure .
> step3 : A is assign to rs2 and rs success to open it .
> Above is the normal flow . However when rs1 is restart after the reigon A was
> split and GCRegionProcedure was successed , the region A appare again !
> The region is that reigon A is not removed from the serverMap correctly when
> assignprocedure handleFailure . Because the code regionNode.offline() make
> the regionNode's regionLocation to be null and make regionNode's state to
> OFFLINE . So when the code
> env.getAssignmentManager().undoRegionAsOpening(regionNode) do nothing . So
> when the rs1 restart event triggers a serverCrashProcedure, it will get
> reigons from serverMap and it will get the region A then A will be assigned
> and hdfs dir will be created.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22884) Disable builds on EOL'd branch-2.0; undo nightly build
[ https://issues.apache.org/jira/browse/HBASE-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911474#comment-16911474 ] stack commented on HBASE-22884: --- Ok. In addendum removed other three files Pushed on Master. > Disable builds on EOL'd branch-2.0; undo nightly build > -- > > Key: HBASE-22884 > URL: https://issues.apache.org/jira/browse/HBASE-22884 > Project: HBase > Issue Type: Bug >Reporter: stack >Priority: Major > > I think all I have to do is remove the dev-support/Jenkinsfile (IIRC). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (HBASE-22884) Disable builds on EOL'd branch-2.0; undo nightly build
[ https://issues.apache.org/jira/browse/HBASE-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-22884. --- Fix Version/s: 2.0.7 Release Note: Undoes the nightly and flakie builds for branch-2.0. Assignee: stack Resolution: Fixed > Disable builds on EOL'd branch-2.0; undo nightly build > -- > > Key: HBASE-22884 > URL: https://issues.apache.org/jira/browse/HBASE-22884 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.7 > > > I think all I have to do is remove the dev-support/Jenkinsfile (IIRC). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (HBASE-22885) Disable nightly builds on snoozing branch HBASE-20952
stack created HBASE-22885: - Summary: Disable nightly builds on snoozing branch HBASE-20952 Key: HBASE-22885 URL: https://issues.apache.org/jira/browse/HBASE-22885 Project: HBase Issue Type: Sub-task Reporter: stack Disabling builds while branch HBASE-20952 dev is on hiatus. Can add back the build files when it comes alive again. See parent for discussion on temporary disabling. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (HBASE-22885) Disable nightly builds on snoozing branch HBASE-20952
[ https://issues.apache.org/jira/browse/HBASE-22885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-22885. --- Fix Version/s: HBASE-20952 Release Note: Disabled builds temporarily by removing jenkins build files. Revert the below to reenable builds: commit b7aaac7751c62272fd790fe72b89306f0457dc6b (HEAD -> HBASE-20952, origin/HBASE-20952) Author: stack Date: Tue Aug 20 08:48:49 2019 -0700 HBASE-22885 Disable nightly builds on snoozing branch HBASE-20952 Assignee: stack Resolution: Fixed Pushed on HBASE-20952 branch. > Disable nightly builds on snoozing branch HBASE-20952 > - > > Key: HBASE-22885 > URL: https://issues.apache.org/jira/browse/HBASE-22885 > Project: HBase > Issue Type: Sub-task >Reporter: stack >Assignee: stack >Priority: Major > Fix For: HBASE-20952 > > > Disabling builds while branch HBASE-20952 dev is on hiatus. Can add back the > build files when it comes alive again. See parent for discussion on temporary > disabling. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22884) Disable builds on EOL'd branch-2.0; undo nightly build
[ https://issues.apache.org/jira/browse/HBASE-22884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911486#comment-16911486 ] stack commented on HBASE-22884: --- Undid build on HBASE-20952 too... See HBASE-22885. Ran 'Scan Multibranch Pipeline Now' too as per [~busbey] suggestion up in HBASE-20952. > Disable builds on EOL'd branch-2.0; undo nightly build > -- > > Key: HBASE-22884 > URL: https://issues.apache.org/jira/browse/HBASE-22884 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack >Priority: Major > Fix For: 2.0.7 > > > I think all I have to do is remove the dev-support/Jenkinsfile (IIRC). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-20952) Re-visit the WAL API
[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911491#comment-16911491
]
stack commented on HBASE-20952:
---
Disabled build with a sub-issue, HBASE-22885. Just run the revert to reenable
building.
> Re-visit the WAL API
>
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Improvement
> Components: wal
>Reporter: Josh Elser
>Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup&restore. Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B&R doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HBASE-22822) Re/Un-schedule balancer chore for balance_switch
[ https://issues.apache.org/jira/browse/HBASE-22822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911495#comment-16911495 ] Viraj Jasani commented on HBASE-22822: -- Based on the discussion on PR, closing the JIRA > Re/Un-schedule balancer chore for balance_switch > > > Key: HBASE-22822 > URL: https://issues.apache.org/jira/browse/HBASE-22822 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Affects Versions: 3.0.0, 1.5.0, 2.2.1 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.2.2 > > > balance_switch turns on/off balancer. When it is turned off, we don't remove > balancer chore from scheduled chores hence it keeps running only to > eventually find out that it is not supposed to perform any action(if balancer > was turned off). We can unschedule the chore to prevent the chore() execution > and reschedule it when it is turned on by balance_switch. > This should also facilitate running balancer immediately after triggering > balance_switch true, and then chore would continue running as per duration > provided in hbase.balancer.period. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22822) Re/Un-schedule balancer chore for balance_switch
[ https://issues.apache.org/jira/browse/HBASE-22822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-22822: - Resolution: Won't Fix Status: Resolved (was: Patch Available) > Re/Un-schedule balancer chore for balance_switch > > > Key: HBASE-22822 > URL: https://issues.apache.org/jira/browse/HBASE-22822 > Project: HBase > Issue Type: Improvement > Components: Balancer, master >Affects Versions: 3.0.0, 1.5.0, 2.2.1 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.2.2 > > > balance_switch turns on/off balancer. When it is turned off, we don't remove > balancer chore from scheduled chores hence it keeps running only to > eventually find out that it is not supposed to perform any action(if balancer > was turned off). We can unschedule the chore to prevent the chore() execution > and reschedule it when it is turned on by balance_switch. > This should also facilitate running balancer immediately after triggering > balance_switch true, and then chore would continue running as per duration > provided in hbase.balancer.period. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22874) Canary should not be IA.Public
[ https://issues.apache.org/jira/browse/HBASE-22874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-22874: --- Fix Version/s: 1.4.11 1.3.6 1.5.0 > Canary should not be IA.Public > -- > > Key: HBASE-22874 > URL: https://issues.apache.org/jira/browse/HBASE-22874 > Project: HBase > Issue Type: Bug > Components: canary >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > It can be marked as IA.LimitedPrivate(TOOLS). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HBASE-22823) Mark Canary as Public/Evolving
[ https://issues.apache.org/jira/browse/HBASE-22823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-22823: --- Fix Version/s: (was: 1.4.11) (was: 1.3.6) (was: 2.1.6) (was: 2.2.1) (was: 1.5.0) (was: 3.0.0) > Mark Canary as Public/Evolving > -- > > Key: HBASE-22823 > URL: https://issues.apache.org/jira/browse/HBASE-22823 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Caroline >Priority: Minor > Attachments: HBASE-22823.branch-1.000.patch, > HBASE-22823.branch-2.000.patch, HBASE-22823.master.000.patch > > > Canary is marked as a Private class. Its interfaces could change at any time. > Should we change the annotation on Canary to Public/Evolving? Or add > annotations on some of these subtypes? I think it depends on how we think > Canary results should be consumed. > In our production we find that scraping logs and parsing them is brittle and > not scalable. Although the scalability issue is more to do with the totality > of logs from a Hadoopish stack, if you run HBase then you have this problem, > and you wouldn't be using the canary if you didn't run HBase. We have a tool > that embeds the Canary and calls various methods and takes actions without > needing a round trip to the logs and whatever aggregates them. > I propose we promote Canary to Public/Evolving. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Resolved] (HBASE-22823) Mark Canary as Public/Evolving
[ https://issues.apache.org/jira/browse/HBASE-22823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-22823. Resolution: Won't Fix > Mark Canary as Public/Evolving > -- > > Key: HBASE-22823 > URL: https://issues.apache.org/jira/browse/HBASE-22823 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Caroline >Priority: Minor > Attachments: HBASE-22823.branch-1.000.patch, > HBASE-22823.branch-2.000.patch, HBASE-22823.master.000.patch > > > Canary is marked as a Private class. Its interfaces could change at any time. > Should we change the annotation on Canary to Public/Evolving? Or add > annotations on some of these subtypes? I think it depends on how we think > Canary results should be consumed. > In our production we find that scraping logs and parsing them is brittle and > not scalable. Although the scalability issue is more to do with the totality > of logs from a Hadoopish stack, if you run HBase then you have this problem, > and you wouldn't be using the canary if you didn't run HBase. We have a tool > that embeds the Canary and calls various methods and takes actions without > needing a round trip to the logs and whatever aggregates them. > I propose we promote Canary to Public/Evolving. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Reopened] (HBASE-22823) Mark Canary as Public/Evolving
[ https://issues.apache.org/jira/browse/HBASE-22823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reopened HBASE-22823: HBASE-22874 is filed against this as a blocker. Reverting this commit in lieu of whatever happens on HBASE-22874 > Mark Canary as Public/Evolving > -- > > Key: HBASE-22823 > URL: https://issues.apache.org/jira/browse/HBASE-22823 > Project: HBase > Issue Type: Sub-task >Reporter: Andrew Purtell >Assignee: Caroline >Priority: Minor > Fix For: 3.0.0, 1.5.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > Attachments: HBASE-22823.branch-1.000.patch, > HBASE-22823.branch-2.000.patch, HBASE-22823.master.000.patch > > > Canary is marked as a Private class. Its interfaces could change at any time. > Should we change the annotation on Canary to Public/Evolving? Or add > annotations on some of these subtypes? I think it depends on how we think > Canary results should be consumed. > In our production we find that scraping logs and parsing them is brittle and > not scalable. Although the scalability issue is more to do with the totality > of logs from a Hadoopish stack, if you run HBase then you have this problem, > and you wouldn't be using the canary if you didn't run HBase. We have a tool > that embeds the Canary and calls various methods and takes actions without > needing a round trip to the logs and whatever aggregates them. > I propose we promote Canary to Public/Evolving. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22874) Canary should not be IA.Public
[ https://issues.apache.org/jira/browse/HBASE-22874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911552#comment-16911552 ] Sean Busbey commented on HBASE-22874: - Anyone interested in raising a hand to get a patch up that takes a stab at this? given the last minute back out of HBASE-22823 from RCs I'd rather get something up while our attention as a community is here. [~gjacoby] you interested? > Canary should not be IA.Public > -- > > Key: HBASE-22874 > URL: https://issues.apache.org/jira/browse/HBASE-22874 > Project: HBase > Issue Type: Bug > Components: canary >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > It can be marked as IA.LimitedPrivate(TOOLS). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22874) Canary should not be IA.Public
[ https://issues.apache.org/jira/browse/HBASE-22874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911550#comment-16911550 ] Andrew Purtell commented on HBASE-22874: I reverted HBASE-22823 and pushed the revert to all branches. We can do the refactor and promotion from Private to LP(tools) here. > Canary should not be IA.Public > -- > > Key: HBASE-22874 > URL: https://issues.apache.org/jira/browse/HBASE-22874 > Project: HBase > Issue Type: Bug > Components: canary >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > It can be marked as IA.LimitedPrivate(TOOLS). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-11062) hbtop
[ https://issues.apache.org/jira/browse/HBASE-11062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911569#comment-16911569 ] Krishna Maheshwari commented on HBASE-11062: This is very nice Toshi. Is it possible to add clients in addition to Region/Namespace/Table/RegionServer? That would help some admins identify which clients are driving load on their HBase so that they can better implement quotas, etc. > hbtop > - > > Key: HBASE-11062 > URL: https://issues.apache.org/jira/browse/HBASE-11062 > Project: HBase > Issue Type: New Feature > Components: hbtop >Reporter: Andrew Purtell >Assignee: Toshihiro Suzuki >Priority: Major > > A top-like monitor could be useful for testing, debugging, operations of > clusters of moderate size, and possibly for diagnosing issues in large > clusters. > Consider a curses interface like the one presented by atop > (http://www.atoptool.nl/images/screenshots/genericw.png) - with aggregate > metrics collected over a monitoring interval in the upper portion of the > pane, and a listing of discrete measurements sorted and filtered by various > criteria in the bottom part of the pane. One might imagine a cluster overview > with cluster aggregate metrics above and a list of regionservers sorted by > utilization below; and a regionserver view with process metrics above and a > list of metrics by operation type below, or a list of client connections, or > a list of threads, sorted by utilization, throughput, or latency. > Generically 'htop' is taken but would be distinctive in the HBase context, a > utility org.apache.hadoop.hbase.HTop > No need necessarily for a curses interface. Could be an external monitor with > a web front end as has been discussed before. I do like the idea of a process > that runs in a terminal because I interact with dev and test HBase clusters > exclusively by SSH. > UPDATE: > The tool name is changed from htop to hbtop. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #496: HBASE-22625 documet use scan snapshot feature
Apache-HBase commented on issue #496: HBASE-22625 documet use scan snapshot feature URL: https://github.com/apache/hbase/pull/496#issuecomment-523119532 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 63 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | 0 | shelldocs | 0 | Shelldocs was not available. | | +1 | @author | 0 | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 | mvninstall | 419 | master passed | | 0 | refguide | 380 | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 389 | the patch passed | | +1 | shellcheck | 0 | There were no new shellcheck issues. | | +1 | whitespace | 0 | The patch has no whitespace issues. | | 0 | refguide | 391 | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Other Tests _ | | +1 | asflicense | 23 | The patch does not generate ASF License warnings. | | | | 1729 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-496/16/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/496 | | Optional Tests | dupname asflicense shellcheck shelldocs refguide | | uname | Linux 5c49ed11075a 4.4.0-154-generic #181-Ubuntu SMP Tue Jun 25 05:29:03 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-496/out/precommit/personality/provided.sh | | git revision | master / ab28f9d8c7 | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-496/16/artifact/out/branch-site/book.html | | refguide | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-496/16/artifact/out/patch-site/book.html | | Max. process+thread count | 76 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-496/16/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) shellcheck=0.4.4 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] shahrs87 commented on issue #509: HBASE-22877 WebHDFS based export snapshot will fail if hfile is in archive directory
shahrs87 commented on issue #509: HBASE-22877 WebHDFS based export snapshot will fail if hfile is in archive directory URL: https://github.com/apache/hbase/pull/509#issuecomment-523121559 > in this case, no exception will be thrown even when the file didn't exist when we are calling @VicoWu Could you elaborate why you think it doesn't throw FileNotfound exception. In the stack trace you pasted, it did throw FileNotFound Exception but it is wrapped in RemoteException. You just need to unwrap RemoteException to see underlying exception. Maybe I am missing something. Please correct me if I am wrong. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22601) Misconfigured addition of peers leads to cluster shutdown.
[
https://issues.apache.org/jira/browse/HBASE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911590#comment-16911590
]
Rushabh S Shah commented on HBASE-22601:
Hi [~apurtell] [~Apache9]
could you guys please review the attached PR. Its very small. Thanks in advance.
> Misconfigured addition of peers leads to cluster shutdown.
> --
>
> Key: HBASE-22601
> URL: https://issues.apache.org/jira/browse/HBASE-22601
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Rushabh S Shah
>Assignee: Rushabh S Shah
>Priority: Major
>
> Recently we added a peer to a production cluster which were in different
> kerberos realm.
> *Steps to reproduce:*
> 1. Add a misconfigured peer which is in different kerberos realm.
> 2. Remove that peer.
> 3. All region servers will start to crash.
> *RCA*
> Enabled trace logging on one Region server for a short amount of time.
> After adding peer, saw the following log lines.
> {noformat}
> 2019-06-18 22:19:20,949 INFO [main-EventThread]
> replication.ReplicationTrackerZKImpl - /hbase/replication/peers znode
> expired, triggering peerListChanged event
> 2019-06-18 22:19:20,992 INFO [main-EventThread]
> replication.ReplicationPeersZKImpl - Added new peer
> cluster=:/hbase
> 2019-06-18 22:19:21,113 INFO [main-EventThread]
> zookeeper.RecoverableZooKeeper - Process identifier=hconnection-0x794a56d6
> connecting to ZooKeeper ensemble=
> 2019-06-18 22:20:01,280 WARN [main-EventThread] zookeeper.ZKUtil -
> hconnection-0x794a56d6-0x16b56265fbebb1b, quorum=,
> baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid)
> org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode =
> AuthFailed for /hbase/hbaseid
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:123)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:220)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:421)
> at
> org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:922)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:706)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:638)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> at
> org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:432)
> at
> org.apache.hadoop.hbase.client.ConnectionManager.createConnectionInternal(ConnectionManager.java:341)
> at
> org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:144)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.init(HBaseInterClusterReplicationEndpoint.java:135)
> at
> com.salesforce.hbase.replication.TenantReplicationEndpoint.init(TenantReplicationEndpoint.java:30)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.getReplicationSource(ReplicationSourceManager.java:517)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.addSource(ReplicationSourceManager.java:273)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:635)
> at
> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:192)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:643)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:544)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:519)
> 2019-06-18 22:20:42,999 WARN [Source,] zookeeper.ZKUtil -
[jira] [Comment Edited] (HBASE-22601) Misconfigured addition of peers leads to cluster shutdown.
[
https://issues.apache.org/jira/browse/HBASE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911590#comment-16911590
]
Rushabh S Shah edited comment on HBASE-22601 at 8/20/19 5:47 PM:
-
Hi [~apurtell] [~Apache9]
could you guys please review the attached PR. Its very small. Thanks in
advance. All the tests are passing also.
was (Author: shahrs87):
Hi [~apurtell] [~Apache9]
could you guys please review the attached PR. Its very small. Thanks in advance.
> Misconfigured addition of peers leads to cluster shutdown.
> --
>
> Key: HBASE-22601
> URL: https://issues.apache.org/jira/browse/HBASE-22601
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.3.2
>Reporter: Rushabh S Shah
>Assignee: Rushabh S Shah
>Priority: Major
>
> Recently we added a peer to a production cluster which were in different
> kerberos realm.
> *Steps to reproduce:*
> 1. Add a misconfigured peer which is in different kerberos realm.
> 2. Remove that peer.
> 3. All region servers will start to crash.
> *RCA*
> Enabled trace logging on one Region server for a short amount of time.
> After adding peer, saw the following log lines.
> {noformat}
> 2019-06-18 22:19:20,949 INFO [main-EventThread]
> replication.ReplicationTrackerZKImpl - /hbase/replication/peers znode
> expired, triggering peerListChanged event
> 2019-06-18 22:19:20,992 INFO [main-EventThread]
> replication.ReplicationPeersZKImpl - Added new peer
> cluster=:/hbase
> 2019-06-18 22:19:21,113 INFO [main-EventThread]
> zookeeper.RecoverableZooKeeper - Process identifier=hconnection-0x794a56d6
> connecting to ZooKeeper ensemble=
> 2019-06-18 22:20:01,280 WARN [main-EventThread] zookeeper.ZKUtil -
> hconnection-0x794a56d6-0x16b56265fbebb1b, quorum=,
> baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid)
> org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode =
> AuthFailed for /hbase/hbaseid
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:123)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:220)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:421)
> at
> org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:922)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:706)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:638)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> at
> org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:432)
> at
> org.apache.hadoop.hbase.client.ConnectionManager.createConnectionInternal(ConnectionManager.java:341)
> at
> org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:144)
> at
> org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint.init(HBaseInterClusterReplicationEndpoint.java:135)
> at
> com.salesforce.hbase.replication.TenantReplicationEndpoint.init(TenantReplicationEndpoint.java:30)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.getReplicationSource(ReplicationSourceManager.java:517)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.addSource(ReplicationSourceManager.java:273)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:635)
> at
> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:192)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:643)
> at
> org.apach
[GitHub] [hbase] Apache-HBase commented on issue #487: HBASE-22844 Fixed Checkstyle violations in client snapshot exceptions
Apache-HBase commented on issue #487: HBASE-22844 Fixed Checkstyle violations in client snapshot exceptions URL: https://github.com/apache/hbase/pull/487#issuecomment-523129513 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 148 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 375 | master passed | | +1 | compile | 32 | master passed | | +1 | checkstyle | 37 | master passed | | +1 | shadedjars | 283 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 24 | master passed | | 0 | spotbugs | 81 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 76 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 335 | the patch passed | | +1 | compile | 30 | the patch passed | | +1 | javac | 30 | the patch passed | | +1 | checkstyle | 31 | hbase-client: The patch generated 0 new + 0 unchanged - 13 fixed = 0 total (was 13) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 278 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 959 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 21 | the patch passed | | +1 | findbugs | 76 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 100 | hbase-client in the patch passed. | | +1 | asflicense | 12 | The patch does not generate ASF License warnings. | | | | 3198 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-487/22/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/487 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 4b061e742c70 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-487/out/precommit/personality/provided.sh | | git revision | master / ab28f9d8c7 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-487/22/testReport/ | | Max. process+thread count | 292 (vs. ulimit of 1) | | modules | C: hbase-client U: hbase-client | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-487/22/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on issue #476: HBASE-11062 hbtop
saintstack commented on issue #476: HBASE-11062 hbtop URL: https://github.com/apache/hbase/pull/476#issuecomment-523129994 > hbase hbtop... I think this feature is useful for hbase administrators, so being able to use it easily may be wanted. Agree. Very useful. Lets chat. Looks like others are thinking it should be hbase-operator-tools going by dev feedback so far but yes, being able to do hbase top is really important hbase script downloads operator tools jar or a hbase version depends on released operator tools so its bundled. On your suggestion that we could come back to the terminal issue later, that sounds good. So answer to my concern #2 could be that we deal w/ terminal in a follow-on. Thanks for putting note up on the dev list. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #511: HBASE-22879 user_permission command failed to show global permission
Apache-HBase commented on issue #511: HBASE-22879 user_permission command failed to show global permission URL: https://github.com/apache/hbase/pull/511#issuecomment-523132048 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 61 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 372 | master passed | | +1 | compile | 24 | master passed | | +1 | checkstyle | 11 | master passed | | +1 | shadedjars | 301 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 15 | master passed | | 0 | spotbugs | 319 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 0 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 336 | the patch passed | | +1 | compile | 22 | the patch passed | | +1 | javac | 22 | the patch passed | | +1 | checkstyle | 10 | the patch passed | | -1 | rubocop | 6 | The patch generated 3 new + 67 unchanged - 3 fixed = 70 total (was 70) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 276 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 943 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 33 | the patch passed | | +1 | findbugs | 0 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 745 | hbase-shell in the patch passed. | | +1 | asflicense | 17 | The patch does not generate ASF License warnings. | | | | 3613 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-511/7/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/511 | | Optional Tests | dupname asflicense javac javadoc unit rubocop spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 90009fb56837 4.4.0-137-generic #163-Ubuntu SMP Mon Sep 24 13:14:43 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-511/out/precommit/personality/provided.sh | | git revision | master / ab28f9d8c7 | | Default Java | 1.8.0_181 | | rubocop | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-511/7/artifact/out/diff-patch-rubocop.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-511/7/testReport/ | | Max. process+thread count | 2528 (vs. ulimit of 1) | | modules | C: hbase-shell U: hbase-shell | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-511/7/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) rubocop=0.74.0 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #387: HBASE-22642 Make move operations of RSGroup idempotent
Apache-HBase commented on issue #387: HBASE-22642 Make move operations of RSGroup idempotent URL: https://github.com/apache/hbase/pull/387#issuecomment-523141982 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 78 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 | mvninstall | 365 | master passed | | +1 | compile | 28 | master passed | | +1 | checkstyle | 13 | master passed | | +1 | shadedjars | 310 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 22 | master passed | | 0 | spotbugs | 53 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 51 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 336 | the patch passed | | +1 | compile | 25 | the patch passed | | +1 | javac | 25 | the patch passed | | +1 | checkstyle | 12 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 301 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1046 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 24 | the patch passed | | +1 | findbugs | 64 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 518 | hbase-rsgroup in the patch passed. | | +1 | asflicense | 12 | The patch does not generate ASF License warnings. | | | | 3617 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-387/105/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/387 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 95870daff2f1 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-387/out/precommit/personality/provided.sh | | git revision | master / ab28f9d8c7 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-387/105/testReport/ | | Max. process+thread count | 4328 (vs. ulimit of 1) | | modules | C: hbase-rsgroup U: hbase-rsgroup | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-387/105/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-11062) hbtop
[ https://issues.apache.org/jira/browse/HBASE-11062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911676#comment-16911676 ] Andrew Purtell commented on HBASE-11062: I am reviewing this. I am also backporting it to branch-1 to try it out at the same time. Feedback after that is done. > hbtop > - > > Key: HBASE-11062 > URL: https://issues.apache.org/jira/browse/HBASE-11062 > Project: HBase > Issue Type: New Feature > Components: hbtop >Reporter: Andrew Purtell >Assignee: Toshihiro Suzuki >Priority: Major > > A top-like monitor could be useful for testing, debugging, operations of > clusters of moderate size, and possibly for diagnosing issues in large > clusters. > Consider a curses interface like the one presented by atop > (http://www.atoptool.nl/images/screenshots/genericw.png) - with aggregate > metrics collected over a monitoring interval in the upper portion of the > pane, and a listing of discrete measurements sorted and filtered by various > criteria in the bottom part of the pane. One might imagine a cluster overview > with cluster aggregate metrics above and a list of regionservers sorted by > utilization below; and a regionserver view with process metrics above and a > list of metrics by operation type below, or a list of client connections, or > a list of threads, sorted by utilization, throughput, or latency. > Generically 'htop' is taken but would be distinctive in the HBase context, a > utility org.apache.hadoop.hbase.HTop > No need necessarily for a curses interface. Could be an external monitor with > a web front end as has been discussed before. I do like the idea of a process > that runs in a terminal because I interact with dev and test HBase clusters > exclusively by SSH. > UPDATE: > The tool name is changed from htop to hbtop. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] shahrs87 edited a comment on issue #509: HBASE-22877 WebHDFS based export snapshot will fail if hfile is in archive directory
shahrs87 edited a comment on issue #509: HBASE-22877 WebHDFS based export snapshot will fail if hfile is in archive directory URL: https://github.com/apache/hbase/pull/509#issuecomment-523121559 > in this case, no exception will be thrown even when the file didn't exist when we are calling @VicoWu Could you elaborate why you think it doesn't throw FileNotfound exception. In the stack trace you pasted, it did throw FileNotFound Exception but it is wrapped in RemoteException. You just need to unwrap RemoteException to see underlying exception. Maybe I am missing something. Please correct me if I am wrong. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22874) Canary should not be IA.Public
[ https://issues.apache.org/jira/browse/HBASE-22874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911688#comment-16911688 ] Rushabh S Shah commented on HBASE-22874: > Anyone interested in raising a hand to get a patch up that takes a stab at >this? [~busbey] I can create a patch in half an hour. > Canary should not be IA.Public > -- > > Key: HBASE-22874 > URL: https://issues.apache.org/jira/browse/HBASE-22874 > Project: HBase > Issue Type: Bug > Components: canary >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > It can be marked as IA.LimitedPrivate(TOOLS). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [hbase] shahrs87 opened a new pull request #514: [HBASE-22874] Mark Canary as LA(Tools)
shahrs87 opened a new pull request #514: [HBASE-22874] Mark Canary as LA(Tools) URL: https://github.com/apache/hbase/pull/514 @busbey please review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] shahrs87 opened a new pull request #515: [HBASE-22874] Mark Canary as LA(Tools)
shahrs87 opened a new pull request #515: [HBASE-22874] Mark Canary as LA(Tools) URL: https://github.com/apache/hbase/pull/515 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] shahrs87 opened a new pull request #516: [HBASE-22874] Mark Canary as LA(Tools)
shahrs87 opened a new pull request #516: [HBASE-22874] Mark Canary as LA(Tools) URL: https://github.com/apache/hbase/pull/516 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] shahrs87 commented on issue #514: [HBASE-22874] Mark Canary as LA(Tools)
shahrs87 commented on issue #514: [HBASE-22874] Mark Canary as LA(Tools) URL: https://github.com/apache/hbase/pull/514#issuecomment-523168453 PR for branch-1 https://github.com/apache/hbase/pull/516 PR for branch-2 https://github.com/apache/hbase/pull/515 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22874) Canary should not be IA.Public
[ https://issues.apache.org/jira/browse/HBASE-22874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911696#comment-16911696 ] Rushabh S Shah commented on HBASE-22874: Below are the PR for all branches: master: [https://github.com/apache/hbase/pull/514] branch-2 [https://github.com/apache/hbase/pull/515|https://github.com/apache/hbase/pull/514] branch-1 [https://github.com/apache/hbase/pull/516|https://github.com/apache/hbase/pull/514] Please let me know if you want me to create reviews for other branches. [~busbey] [~apurtell] [~Apache9] please review. > Canary should not be IA.Public > -- > > Key: HBASE-22874 > URL: https://issues.apache.org/jira/browse/HBASE-22874 > Project: HBase > Issue Type: Bug > Components: canary >Reporter: Duo Zhang >Priority: Blocker > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > It can be marked as IA.LimitedPrivate(TOOLS). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HBASE-22874) Canary should not be IA.Public
[ https://issues.apache.org/jira/browse/HBASE-22874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh S Shah reassigned HBASE-22874: -- Assignee: Rushabh S Shah > Canary should not be IA.Public > -- > > Key: HBASE-22874 > URL: https://issues.apache.org/jira/browse/HBASE-22874 > Project: HBase > Issue Type: Bug > Components: canary >Reporter: Duo Zhang >Assignee: Rushabh S Shah >Priority: Blocker > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > It can be marked as IA.LimitedPrivate(TOOLS). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22874) Canary should not be IA.Public
[ https://issues.apache.org/jira/browse/HBASE-22874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911700#comment-16911700 ] Geoffrey Jacoby commented on HBASE-22874: - [~shahrs87] - if you want to take it, go for it -- if not I can probably get to it sometime in the next week or so. I think it's more than a 30 minute task because it wouldn't just be changing the InterfaceAudience or creating a new interface with the method "run(String[])", which we already have in the Tool interface anyway. I think [~Apache9]'s correct that BulkLoadHFiles / BulkLoadHFilesTool is the right model here -- there's an IA.Public interface suitable for API usage and a LimitedPrivate(Tools) implementation class suitable for command line use. That means refactoring the Canary so that it exposes a true API instead of the arg-parsed String array (and extracting it into Java interface), and then refactoring the Tool portion of the canary to parse the args and call the API instead. Figuring out exactly what that API looks like so that we're comfortable marking the IA as Public will probably take some thought, because the Canary has quite a few command arguments. > Canary should not be IA.Public > -- > > Key: HBASE-22874 > URL: https://issues.apache.org/jira/browse/HBASE-22874 > Project: HBase > Issue Type: Bug > Components: canary >Reporter: Duo Zhang >Assignee: Rushabh S Shah >Priority: Blocker > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > It can be marked as IA.LimitedPrivate(TOOLS). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HBASE-22631) assign failed may make gced parent region appear again !!!
[
https://issues.apache.org/jira/browse/HBASE-22631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911701#comment-16911701
]
HBase QA commented on HBASE-22631:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2.1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
0s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
25s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
18s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s{color} | {color:green} branch-2.1 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} branch-2.1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
20s{color} | {color:red} hbase-server: The patch generated 3 new + 5 unchanged
- 0 fixed = 8 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
24s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
22m 14s{color} | {color:green} Patch does not cause any errors with Hadoop
2.7.7 2.8.5 or 3.0.3 3.1.2. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}198m 57s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}259m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/805/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22631 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978096/HBASE-22631-branch-2.1-02.patch
|
| Optional Tests | dupname asflicense javac javadoc unit findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 88e555f6ad09 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-2.1 / 1769dfc3a3 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.11 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HBASE-Build/805/artifact/patchprocess/diff-checkstyle-hbase-server.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/805/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/80
[jira] [Commented] (HBASE-22852) hbase nightlies leaking gpg-agents
[
https://issues.apache.org/jira/browse/HBASE-22852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911705#comment-16911705
]
Rushabh S Shah commented on HBASE-22852:
[~busbey] I am new to hbase project. If you can guide me on which files to
look, I can try to put up a patch.
> hbase nightlies leaking gpg-agents
> --
>
> Key: HBASE-22852
> URL: https://issues.apache.org/jira/browse/HBASE-22852
> Project: HBase
> Issue Type: Bug
> Components: build
>Reporter: Allen Wittenauer
>Priority: Major
>
> FYI, just triggered yetus master, which includes code to find and kill
> long-running processes still attached to the Jenkins workspace directory. It
> came up with this:
> https://builds.apache.org/view/S-Z/view/Yetus/job/yetus-github-multibranch/job/master/134/console
> {code}
> USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND
> jenkins752 0.0 0.0 93612 584 ?Ss Aug12 0:00 gpg-agent
> --homedir
> /home/jenkins/jenkins-slave/workspace/HBase_Nightly_HBASE-20952/downloads-hadoop-2/.gpg
> --use-standard-socket --daemon
> Killing 752 ***
> {code}
> (repeat 10s of times, which slightly different dates, pids, versions, etc)
> Also, be aware that any other process running on the node (such as the other
> executor) has extremely easy access to whatever gpg creds you are using...
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [hbase] Apache-HBase commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table
Apache-HBase commented on issue #513: HBASE-22867 The ForkJoinPool in CleanerChore will spawn thousands of threads in our cluster with thousands table URL: https://github.com/apache/hbase/pull/513#issuecomment-523177045 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 66 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 | mvninstall | 352 | master passed | | +1 | compile | 53 | master passed | | +1 | checkstyle | 78 | master passed | | +1 | shadedjars | 282 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 38 | master passed | | 0 | spotbugs | 257 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 255 | master passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 308 | the patch passed | | +1 | compile | 55 | the patch passed | | +1 | javac | 55 | the patch passed | | -1 | checkstyle | 78 | hbase-server: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedjars | 305 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 1054 | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 | javadoc | 36 | the patch passed | | +1 | findbugs | 260 | the patch passed | ||| _ Other Tests _ | | -1 | unit | 17630 | hbase-server in the patch failed. | | +1 | asflicense | 39 | The patch does not generate ASF License warnings. | | | | 21333 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | | hadoop.hbase.client.TestFromClientSide3 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/513 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux dbdfabc60310 4.4.0-137-generic #163-Ubuntu SMP Mon Sep 24 13:14:43 UTC 2018 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-513/out/precommit/personality/provided.sh | | git revision | master / a59f7d4ffc | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/4/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/4/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/4/testReport/ | | Max. process+thread count | 5276 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-513/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22601) Misconfigured addition of peers leads to cluster shutdown.
[
https://issues.apache.org/jira/browse/HBASE-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911708#comment-16911708
]
Hadoop QA commented on HBASE-22601:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 26m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1.3 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_222 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_232 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} branch-1.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} branch-1.3 passed with JDK v1.8.0_222 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} branch-1.3 passed with JDK v1.7.0_232 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
42s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
37s{color} | {color:green} branch-1.3 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed with JDK v1.8.0_222 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.7.0_232 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 2m
25s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
8m 54s{color} | {color:green} Patch does not cause any errors with Hadoop 2.4.1
2.5.2 2.6.5 2.7.7. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed with JDK v1.8.0_222 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
37s{color} | {color:green} the patch passed with JDK v1.7.0_232 {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}123m
9s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {c
[GitHub] [hbase] Apache-HBase commented on issue #490: [HBASE-22601] Misconfigured addition of peers leads to cluster shutdown.
Apache-HBase commented on issue #490: [HBASE-22601] Misconfigured addition of peers leads to cluster shutdown. URL: https://github.com/apache/hbase/pull/490#issuecomment-523177830 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 1595 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | hbaseanti | 0 | Patch does not have any anti-patterns. | | +1 | @author | 0 | The patch does not contain any @author tags. | | -0 | test4tests | 0 | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1.3 Compile Tests _ | | +1 | mvninstall | 448 | branch-1.3 passed | | +1 | compile | 35 | branch-1.3 passed with JDK v1.8.0_222 | | +1 | compile | 38 | branch-1.3 passed with JDK v1.7.0_232 | | +1 | checkstyle | 87 | branch-1.3 passed | | +1 | shadedjars | 153 | branch has no errors when building our shaded downstream artifacts. | | +1 | javadoc | 34 | branch-1.3 passed with JDK v1.8.0_222 | | +1 | javadoc | 37 | branch-1.3 passed with JDK v1.7.0_232 | | 0 | spotbugs | 162 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 157 | branch-1.3 passed | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 97 | the patch passed | | +1 | compile | 35 | the patch passed with JDK v1.8.0_222 | | +1 | javac | 35 | the patch passed | | +1 | compile | 37 | the patch passed with JDK v1.7.0_232 | | +1 | javac | 37 | the patch passed | | +1 | checkstyle | 81 | the patch passed | | +1 | whitespace | 1 | The patch has no whitespace issues. | | +1 | shadedjars | 145 | patch has no errors when building our shaded downstream artifacts. | | +1 | hadoopcheck | 534 | Patch does not cause any errors with Hadoop 2.4.1 2.5.2 2.6.5 2.7.7. | | +1 | javadoc | 28 | the patch passed with JDK v1.8.0_222 | | +1 | javadoc | 37 | the patch passed with JDK v1.7.0_232 | | +1 | findbugs | 156 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 7389 | hbase-server in the patch passed. | | +1 | asflicense | 24 | The patch does not generate ASF License warnings. | | | | 11241 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-490/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/490 | | JIRA Issue | HBASE-22601 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 58900342d8bc 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-490/out/precommit/personality/provided.sh | | git revision | branch-1.3 / 3351124 | | Default Java | 1.7.0_232 | | Multi-JDK versions | /usr/lib/jvm/java-8-openjdk-amd64:1.8.0_222 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_232 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-490/8/testReport/ | | Max. process+thread count | 3910 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-490/8/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22874) Canary should not be IA.Public
[ https://issues.apache.org/jira/browse/HBASE-22874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911718#comment-16911718 ] Sean Busbey commented on HBASE-22874: - [~gjacoby] has it exactly right. We need a refactoring first to define a programmatic API we can make IA.Public and then have the Canary tool use that. > Canary should not be IA.Public > -- > > Key: HBASE-22874 > URL: https://issues.apache.org/jira/browse/HBASE-22874 > Project: HBase > Issue Type: Bug > Components: canary >Reporter: Duo Zhang >Assignee: Rushabh S Shah >Priority: Blocker > Fix For: 3.0.0, 1.5.0, 2.3.0, 2.2.1, 2.1.6, 1.3.6, 1.4.11 > > > It can be marked as IA.LimitedPrivate(TOOLS). -- This message was sent by Atlassian Jira (v8.3.2#803003)
