[GitHub] [hbase] thangTang commented on pull request #2259: HBASE-24839 Backport to branch-1 HBASE-20561 The way we stop a ReplicationSource may cause the RS down
thangTang commented on pull request #2259: URL: https://github.com/apache/hbase/pull/2259#issuecomment-700497491 Hi @infraio , could u help me to review this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25017) Attach a design doc to code base
[ https://issues.apache.org/jira/browse/HBASE-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17203736#comment-17203736 ] Duo Zhang commented on HBASE-25017: --- Oh, must be wrong jira number... > Attach a design doc to code base > > > Key: HBASE-25017 > URL: https://issues.apache.org/jira/browse/HBASE-25017 > Project: HBase > Issue Type: Sub-task > Components: documentation > Environment: Ata >Reporter: Duo Zhang >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23970) TestUsersOperationsWithSecureHadoop fails when an existing ticket is present
[
https://issues.apache.org/jira/browse/HBASE-23970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17203738#comment-17203738
]
wenfeiyi666 commented on HBASE-23970:
-
ping [~ndimiduk] and [~stack]
> TestUsersOperationsWithSecureHadoop fails when an existing ticket is present
>
>
> Key: HBASE-23970
> URL: https://issues.apache.org/jira/browse/HBASE-23970
> Project: HBase
> Issue Type: Test
> Components: security, test
>Affects Versions: 3.0.0-alpha-1, 2.3.0
>Reporter: Nick Dimiduk
>Assignee: wenfeiyi666
>Priority: Minor
>
> {{TestUsersOperationsWithSecureHadoop}} makes assumptions about the state of
> the machine environment. When i have an existing ticket issued and active
> (shown via {{klist}}), this test fails. Signing out of all active sessions
> allows the test to again pass.
> {noformat}
> Caused by:
> org.apache.hadoop.security.authentication.util.KerberosName$NoMatchingRule:
> No rules applied to m...@example.com
> at
> org.apache.hadoop.hbase.security.TestUsersOperationsWithSecureHadoop.testAuthUtilLogin(TestUsersOperationsWithSecureHadoop.java:146)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
Reidd commented on pull request #2194: URL: https://github.com/apache/hbase/pull/2194#issuecomment-700505084 One question, i don't get the solution, why this change can avoid all wals roll? the original version `ConcurrentHashMap walNeedsRoll` is also a per wal data structure? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz commented on pull request #2326: HBASE-24967 The table.jsp cost long time to load if the table include…
bsglz commented on pull request #2326: URL: https://github.com/apache/hbase/pull/2326#issuecomment-700508071 > OK, so the solution is to add compaction state to RegionMetrics, so when getting the compaction state, we could use the in memory state instead of doing rpc. > > +1 Yeah, thanks for you time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz edited a comment on pull request #2326: HBASE-24967 The table.jsp cost long time to load if the table include…
bsglz edited a comment on pull request #2326: URL: https://github.com/apache/hbase/pull/2326#issuecomment-700508071 > OK, so the solution is to add compaction state to RegionMetrics, so when getting the compaction state, we could use the in memory state instead of doing rpc. > > +1 Yeah, thanks for your time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bsglz merged pull request #2326: HBASE-24967 The table.jsp cost long time to load if the table include…
bsglz merged pull request #2326: URL: https://github.com/apache/hbase/pull/2326 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-24967) The table.jsp cost long time to load if the table include closed regions
[
https://issues.apache.org/jira/browse/HBASE-24967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17203756#comment-17203756
]
Zheng Wang commented on HBASE-24967:
Thanks for all the comments.
> The table.jsp cost long time to load if the table include closed regions
>
>
> Key: HBASE-24967
> URL: https://issues.apache.org/jira/browse/HBASE-24967
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
>
> The reason is that, in order to get the compaction state of table, we need to
> iterate all regions and call getRegionInfo, the default timeout for rpc is
> 5000ms, so if there exists closed reigons, this line will be blocked for long
> time.
> {code:java}
> CompactionState compactionState =
> admin.getCompactionState(table.getName()).get();{code}
> My proposal is set timeout for it.
> Affect branch-2+, in branch-1 it will display exception stack info
> immediately.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Joseph295 opened a new pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Joseph295 opened a new pull request #2469: URL: https://github.com/apache/hbase/pull/2469 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
Reidd commented on pull request #2194: URL: https://github.com/apache/hbase/pull/2194#issuecomment-700514955 There should be one key point in old version which leads to all WALs rolled, can you point that out. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25114) [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose
Reid Chan created HBASE-25114: - Summary: [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose Key: HBASE-25114 URL: https://issues.apache.org/jira/browse/HBASE-25114 Project: HBase Issue Type: Improvement Reporter: Reid Chan Assignee: Reid Chan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache9 merged pull request #2464: HBASE-25103 Remove ZNodePaths.metaReplicaZNodes
Apache9 merged pull request #2464: URL: https://github.com/apache/hbase/pull/2464 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] nyl3532016 commented on a change in pull request #2450: HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE
nyl3532016 commented on a change in pull request #2450:
URL: https://github.com/apache/hbase/pull/2450#discussion_r496491951
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/rsgroup/RSGroupBasedLoadBalancer.java
##
@@ -203,8 +203,11 @@ public void setMasterServices(MasterServices
masterServices) {
regionList.forEach(r -> currentAssignmentMap.put(r, regions.get(r)));
Map> pairResult =
this.internalBalancer.retainAssignment(currentAssignmentMap,
pair.getSecond());
-pairResult.forEach((server, rs) ->
-assignments.computeIfAbsent(server, s ->
Lists.newArrayList()).addAll(rs));
+if (pairResult == null || pairResult.isEmpty()) {
Review comment:
Yes, Let me refactor retainAssignment and roundRobinAssignment with
nonNull return
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-24967) The table.jsp cost long time to load if the table include closed regions
[
https://issues.apache.org/jira/browse/HBASE-24967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17203765#comment-17203765
]
Zheng Wang commented on HBASE-24967:
Keep the issue open, i will backport it to branch2 later.
> The table.jsp cost long time to load if the table include closed regions
>
>
> Key: HBASE-24967
> URL: https://issues.apache.org/jira/browse/HBASE-24967
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
>
> The reason is that, in order to get the compaction state of table, we need to
> iterate all regions and call getRegionInfo, the default timeout for rpc is
> 5000ms, so if there exists closed reigons, this line will be blocked for long
> time.
> {code:java}
> CompactionState compactionState =
> admin.getCompactionState(table.getName()).get();{code}
> My proposal is set timeout for it.
> Affect branch-2+, in branch-1 it will display exception stack info
> immediately.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-24967) The table.jsp cost long time to load if the table include closed regions
[
https://issues.apache.org/jira/browse/HBASE-24967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zheng Wang updated HBASE-24967:
---
Description:
The reason is that, in order to get the compaction state of table, we need to
iterate all regions and call getRegionInfo, the default timeout for rpc is
5000ms, so if there exists closed reigons, this line will be blocked for long
time.
{code:java}
CompactionState compactionState =
admin.getCompactionState(table.getName()).get();{code}
The solution in this patch is to add compaction state to RegionMetrics, so when
getting the compaction state, we could use the in memory state instead of doing
rpc.
Affect branch-2+, in branch-1 it will display exception stack info immediately.
was:
The reason is that, in order to get the compaction state of table, we need to
iterate all regions and call getRegionInfo, the default timeout for rpc is
5000ms, so if there exists closed reigons, this line will be blocked for long
time.
{code:java}
CompactionState compactionState =
admin.getCompactionState(table.getName()).get();{code}
My proposal is set timeout for it.
Affect branch-2+, in branch-1 it will display exception stack info immediately.
> The table.jsp cost long time to load if the table include closed regions
>
>
> Key: HBASE-24967
> URL: https://issues.apache.org/jira/browse/HBASE-24967
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
>
> The reason is that, in order to get the compaction state of table, we need to
> iterate all regions and call getRegionInfo, the default timeout for rpc is
> 5000ms, so if there exists closed reigons, this line will be blocked for long
> time.
> {code:java}
> CompactionState compactionState =
> admin.getCompactionState(table.getName()).get();{code}
> The solution in this patch is to add compaction state to RegionMetrics, so
> when getting the compaction state, we could use the in memory state instead
> of doing rpc.
> Affect branch-2+, in branch-1 it will display exception stack info
> immediately.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25115) HFilePrettyPrinter can't seek to the row which is the first row of a hfile
Chaoqiang Zhong created HBASE-25115:
---
Summary: HFilePrettyPrinter can't seek to the row which is the
first row of a hfile
Key: HBASE-25115
URL: https://issues.apache.org/jira/browse/HBASE-25115
Project: HBase
Issue Type: Bug
Components: HFile, tooling
Affects Versions: 2.2.6
Reporter: Chaoqiang Zhong
This issue can be reproduced by below steps:
* make a hfile contains two rows '000' and '001';
{code:java}
D:\bin>hbase hfile -p -f
/hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
K: 000/f1:a/1601282789548/Put/vlen=4/seqid=4 V:
K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5 V:
Scanned kv count -> 2{code}
* '001' can be seeked to;
{code:java}
D:\bin>hbase hfile -e -w 001 -f
/hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5
Scanned kv count -> 1{code}
* but '000' can't be seeked to;
{code:java}
D:\bin>hbase hfile -e -w 000 -f
/hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
Scanned kv count -> 0{code}
In HFilePrettyPrinter we use
"scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))" to seek to row.But
this method will retrurn -1 when the row is the first row of hfile.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Reidddddd opened a new pull request #2470: HBASE-25114 [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose
Reidd opened a new pull request #2470: URL: https://github.com/apache/hbase/pull/2470 [HBASE-25114](https://issues.apache.org/jira/browse/HBASE-25114) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25115) HFilePrettyPrinter can't seek to the row which is the first row of a hfile
[
https://issues.apache.org/jira/browse/HBASE-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chaoqiang Zhong updated HBASE-25115:
Description:
This issue can be reproduced by below steps:
* make a hfile contains two rows '000' and '001';
{code:java}
D:\bin>hbase hfile -p -f
/hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
K: 000/f1:a/1601282789548/Put/vlen=4/seqid=4 V:
K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5 V:
Scanned kv count -> 2{code}
* '001' can be seeked to;
{code:java}
D:\bin>hbase hfile -e -w 001 -f
/hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5
Scanned kv count -> 1{code}
* but '000' can't be seeked to;
{code:java}
D:\bin>hbase hfile -e -w 000 -f
/hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
Scanned kv count -> 0{code}
In HFilePrettyPrinter we use
"scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))" to seek to row.But
this method will retrurn -1 when the row is the first row of hfile.
was:
This issue can be reproduced by below steps:
* make a hfile contains two rows '000' and '001';
{code:java}
D:\bin>hbase hfile -p -f
/hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
K: 000/f1:a/1601282789548/Put/vlen=4/seqid=4 V:
K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5 V:
Scanned kv count -> 2{code}
* '001' can be seeked to;
{code:java}
D:\bin>hbase hfile -e -w 001 -f
/hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5
Scanned kv count -> 1{code}
* but '000' can't be seeked to;
{code:java}
D:\bin>hbase hfile -e -w 000 -f
/hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
Scanned kv count -> 0{code}
In HFilePrettyPrinter we use
"scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))" to seek to row.But
this method will retrurn -1 when the row is the first row of hfile.
> HFilePrettyPrinter can't seek to the row which is the first row of a hfile
> --
>
> Key: HBASE-25115
> URL: https://issues.apache.org/jira/browse/HBASE-25115
> Project: HBase
> Issue Type: Bug
> Components: HFile, tooling
>Affects Versions: 2.2.6
>Reporter: Chaoqiang Zhong
>Priority: Major
>
> This issue can be reproduced by below steps:
> * make a hfile contains two rows '000' and '001';
> {code:java}
> D:\bin>hbase hfile -p -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 000/f1:a/1601282789548/Put/vlen=4/seqid=4 V:
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5 V:
> Scanned kv count -> 2{code}
> * '001' can be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 001 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5
> Scanned kv count -> 1{code}
> * but '000' can't be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 000 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> Scanned kv count -> 0{code}
> In HFilePrettyPrinter we use
> "scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))" to seek to
> row.But this method will retrurn -1 when the row is the first row of hfile.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25007) Make HBCK2 work for 'root table'
[
https://issues.apache.org/jira/browse/HBASE-25007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17203770#comment-17203770
]
Duo Zhang commented on HBASE-25007:
---
{quote}
I've not run into case where meta was on the 'unknown server'. Meta has to be
up to make any progress.
{quote}
But I think we should have a way to deal with this? Maybe an HBCK option to
scan meta and fix the inconsistency between meta and in memory state in
assignment manager? And then we just need to schedule SCP, not HBCKSCP.
> Make HBCK2 work for 'root table'
>
>
> Key: HBASE-25007
> URL: https://issues.apache.org/jira/browse/HBASE-25007
> Project: HBase
> Issue Type: Sub-task
> Components: hbck2
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
>
> We will also scan catalog table and fix them in HBCK2, we should add support
> for root too.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25103) Remove ZNodePaths.metaReplicaZNodes
[ https://issues.apache.org/jira/browse/HBASE-25103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25103. --- Fix Version/s: 2.4.0 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Pushed to master and branch-2. Thanks [~huaxiangsun] for reviewing. > Remove ZNodePaths.metaReplicaZNodes > --- > > Key: HBASE-25103 > URL: https://issues.apache.org/jira/browse/HBASE-25103 > Project: HBase > Issue Type: Sub-task >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.4.0 > > > We have already removed the dependency on this field from client side. > Now if we want to remove the META_REPLICAS_NUM field, we must remove the > usage of this field first. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700531420 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 44s | master passed | | +1 :green_heart: | checkstyle | 1m 37s | master passed | | +1 :green_heart: | spotbugs | 3m 14s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 35s | the patch passed | | -0 :warning: | checkstyle | 0m 15s | hbase-hadoop-compat: The patch generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) | | -0 :warning: | checkstyle | 1m 5s | hbase-server: The patch generated 19 new + 9 unchanged - 0 fixed = 28 total (was 9) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 11m 38s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | -1 :x: | spotbugs | 0m 42s | hbase-hadoop-compat generated 6 new + 0 unchanged - 0 fixed = 6 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. | | | | 38m 51s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-hadoop-compat | | | Dead store to assignMetricsOnMasterUI rather than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:[line 94] | | | Dead store to closeMetricsOnMasterUI rather than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:[line 99] | | | Dead store to moveMetricsOnMasterUI rather than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:[line 96] | | | Dead store to openMetricsOnMasterUI rather than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:[line 98] | | | Dead store to reopenMetricsOnMasterUI rather than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:[line 97] | | | Dead store to unassignMetricsOnMasterUI rather than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:than field with same name in org.apache.hadoop.hbase.master.MetricsAssignmentManagerSourceImpl.init() At MetricsAssignmentManagerSourceImpl.java:[line 95] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 86da489d73b3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5d926627ae | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-hadoop-compat.txt | | checkstyle |
[jira] [Assigned] (HBASE-25115) HFilePrettyPrinter can't seek to the row which is the first row of a hfile
[
https://issues.apache.org/jira/browse/HBASE-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
niuyulin reassigned HBASE-25115:
Assignee: niuyulin
> HFilePrettyPrinter can't seek to the row which is the first row of a hfile
> --
>
> Key: HBASE-25115
> URL: https://issues.apache.org/jira/browse/HBASE-25115
> Project: HBase
> Issue Type: Bug
> Components: HFile, tooling
>Affects Versions: 2.2.6
>Reporter: Chaoqiang Zhong
>Assignee: niuyulin
>Priority: Major
>
> This issue can be reproduced by below steps:
> * make a hfile contains two rows '000' and '001';
> {code:java}
> D:\bin>hbase hfile -p -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 000/f1:a/1601282789548/Put/vlen=4/seqid=4 V:
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5 V:
> Scanned kv count -> 2{code}
> * '001' can be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 001 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5
> Scanned kv count -> 1{code}
> * but '000' can't be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 000 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> Scanned kv count -> 0{code}
> In HFilePrettyPrinter we use
> "scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))" to seek to
> row.But this method will retrurn -1 when the row is the first row of hfile.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2468: HBASE-25109 Add MR Counters to WALPlayer; currently hard to tell if i…
Apache-HBase commented on pull request #2468: URL: https://github.com/apache/hbase/pull/2468#issuecomment-700532920 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 7s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 40s | branch-2 passed | | +1 :green_heart: | compile | 3m 11s | branch-2 passed | | +1 :green_heart: | shadedjars | 8m 6s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 29s | hbase-mapreduce in branch-2 failed. | | -0 :warning: | javadoc | 0m 26s | root in branch-2 failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 36s | the patch passed | | +1 :green_heart: | compile | 4m 57s | the patch passed | | +1 :green_heart: | javac | 4m 57s | the patch passed | | +1 :green_heart: | shadedjars | 9m 25s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 25s | hbase-mapreduce in the patch failed. | | -0 :warning: | javadoc | 0m 19s | root in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 190m 29s | root in the patch passed. | | | | 233m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2468 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 739b8e65b930 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 5218774cf6 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-mapreduce.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-root.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-mapreduce.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-root.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/3/testReport/ | | Max. process+thread count | 6660 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700535881 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 40s | master passed | | +1 :green_heart: | compile | 2m 1s | master passed | | +1 :green_heart: | shadedjars | 7m 6s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 25s | hbase-hadoop-compat in master failed. | | -0 :warning: | javadoc | 0m 20s | hbase-procedure in master failed. | | -0 :warning: | javadoc | 0m 45s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 30s | the patch passed | | +1 :green_heart: | compile | 2m 1s | the patch passed | | +1 :green_heart: | javac | 2m 1s | the patch passed | | +1 :green_heart: | shadedjars | 7m 18s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 21s | hbase-hadoop-compat in the patch failed. | | -0 :warning: | javadoc | 0m 19s | hbase-procedure in the patch failed. | | -0 :warning: | javadoc | 0m 46s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 41s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 1m 52s | hbase-procedure in the patch passed. | | -1 :x: | unit | 11m 3s | hbase-server in the patch failed. | | | | 47m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 72dd8158b570 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5d926627ae | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/testReport/ | | Max. process+thread count | 1353 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700537843 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 4s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 26s | master passed | | +1 :green_heart: | compile | 1m 46s | master passed | | +1 :green_heart: | shadedjars | 7m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 18s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 15s | the patch passed | | +1 :green_heart: | compile | 1m 49s | the patch passed | | +1 :green_heart: | javac | 1m 49s | the patch passed | | +1 :green_heart: | shadedjars | 7m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 18s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 45s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 2m 11s | hbase-procedure in the patch passed. | | -1 :x: | unit | 14m 2s | hbase-server in the patch failed. | | | | 50m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2373a0969cf5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5d926627ae | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/testReport/ | | Max. process+thread count | 1140 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/1/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2450: HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE
Apache-HBase commented on pull request #2450: URL: https://github.com/apache/hbase/pull/2450#issuecomment-700565946 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 50s | master passed | | +1 :green_heart: | checkstyle | 1m 27s | master passed | | +1 :green_heart: | spotbugs | 2m 54s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 29s | the patch passed | | -0 :warning: | checkstyle | 1m 21s | hbase-server: The patch generated 2 new + 65 unchanged - 0 fixed = 67 total (was 65) | | -0 :warning: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | hadoopcheck | 13m 37s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 4m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 43m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2450 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 5ba3573bbae0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b268b1f621 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt | | whitespace | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/artifact/yetus-general-check/output/whitespace-eol.txt | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil commented on pull request #2400: HBASE-24877 addendum: additional checks to avoid one extra possible race control in the initialize loop
wchevreuil commented on pull request #2400: URL: https://github.com/apache/hbase/pull/2400#issuecomment-700567778 Latest failures seems unrelated. Merging it now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] wchevreuil merged pull request #2400: HBASE-24877 addendum: additional checks to avoid one extra possible race control in the initialize loop
wchevreuil merged pull request #2400: URL: https://github.com/apache/hbase/pull/2400 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-25116) RegionMonitor support RegionTask count normalize
niuyulin created HBASE-25116: Summary: RegionMonitor support RegionTask count normalize Key: HBASE-25116 URL: https://issues.apache.org/jira/browse/HBASE-25116 Project: HBase Issue Type: Improvement Reporter: niuyulin Assignee: niuyulin -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25115) HFilePrettyPrinter can't seek to the row which is the first row of a hfile
[
https://issues.apache.org/jira/browse/HBASE-25115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17203799#comment-17203799
]
Chaoqiang Zhong commented on HBASE-25115:
-
This maybe cased by HFileBlockIndex#rootBlockContainingKey().
When we use HFileBlockIndex#rootBlockContainingKey() with a cell created by
PrivateCellUtil.createFirstOnRow(), it will return -1.
> HFilePrettyPrinter can't seek to the row which is the first row of a hfile
> --
>
> Key: HBASE-25115
> URL: https://issues.apache.org/jira/browse/HBASE-25115
> Project: HBase
> Issue Type: Bug
> Components: HFile, tooling
>Affects Versions: 2.2.6
>Reporter: Chaoqiang Zhong
>Assignee: niuyulin
>Priority: Major
>
> This issue can be reproduced by below steps:
> * make a hfile contains two rows '000' and '001';
> {code:java}
> D:\bin>hbase hfile -p -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 000/f1:a/1601282789548/Put/vlen=4/seqid=4 V:
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5 V:
> Scanned kv count -> 2{code}
> * '001' can be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 001 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> K: 001/f1:a/1601282792779/Put/vlen=4/seqid=5
> Scanned kv count -> 1{code}
> * but '000' can't be seeked to;
> {code:java}
> D:\bin>hbase hfile -e -w 000 -f
> /hbase/data/default/test2/df76e4acab5398e70be332f6807ec3ba/f1/fda213c556d540a58d29d6bd85931dcd
> Scanned kv count -> 0{code}
> In HFilePrettyPrinter we use
> "scanner.seekTo(PrivateCellUtil.createFirstOnRow(this.row))" to seek to
> row.But this method will retrurn -1 when the row is the first row of hfile.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25034) Table regions details on master GUI display slowly.
[ https://issues.apache.org/jira/browse/HBASE-25034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17203803#comment-17203803 ] DingSheng Li commented on HBASE-25034: -- After pagination，the table regions details are shown as the following figure. !pagination.jpg|width=991,height=436! > Table regions details on master GUI display slowly. > --- > > Key: HBASE-25034 > URL: https://issues.apache.org/jira/browse/HBASE-25034 > Project: HBase > Issue Type: Improvement >Reporter: DingSheng Li >Priority: Major > Labels: newbie > > When a table has a large number of regions (e.g.,a single table contains more > than 100,000 regions), it takes about 20 to 30 minutes to display the table > regions on the master GUI, which is unacceptable to users. After testing, we > find that web page rendering takes up the most time,and this can be solved by > pagination query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Issue Comment Deleted] (HBASE-25034) Table regions details on master GUI display slowly.
[ https://issues.apache.org/jira/browse/HBASE-25034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DingSheng Li updated HBASE-25034: - Comment: was deleted (was: After pagination，the table regions details are shown as the following figure. !pagination.jpg|width=991,height=436!) > Table regions details on master GUI display slowly. > --- > > Key: HBASE-25034 > URL: https://issues.apache.org/jira/browse/HBASE-25034 > Project: HBase > Issue Type: Improvement >Reporter: DingSheng Li >Priority: Major > Labels: newbie > > When a table has a large number of regions (e.g.,a single table contains more > than 100,000 regions), it takes about 20 to 30 minutes to display the table > regions on the master GUI, which is unacceptable to users. After testing, we > find that web page rendering takes up the most time,and this can be solved by > pagination query. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25116) RegionMonitor support RegionTask count normalize
[ https://issues.apache.org/jira/browse/HBASE-25116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] niuyulin updated HBASE-25116: - Description: large count of region task from canary may affect user normal request, meanwhile if region task is few, the availability monitoring may shake for occasional exception > RegionMonitor support RegionTask count normalize > > > Key: HBASE-25116 > URL: https://issues.apache.org/jira/browse/HBASE-25116 > Project: HBase > Issue Type: Improvement >Reporter: niuyulin >Assignee: niuyulin >Priority: Minor > > large count of region task from canary may affect user normal request, > meanwhile if region task is few, the availability monitoring may shake for > occasional exception -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25116) RegionMonitor support RegionTask count normalize
[ https://issues.apache.org/jira/browse/HBASE-25116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] niuyulin updated HBASE-25116: - Description: large count of region task from canary may affect user normal request, meanwhile if region task is few, the availability monitoring may shake for occasional exception. so , if the task count is large , we will randomly trim tasks for each table, according to the raito of the table region count in whole tasks region count. if the task count is few, we will repeat tasks was:large count of region task from canary may affect user normal request, meanwhile if region task is few, the availability monitoring may shake for occasional exception > RegionMonitor support RegionTask count normalize > > > Key: HBASE-25116 > URL: https://issues.apache.org/jira/browse/HBASE-25116 > Project: HBase > Issue Type: Improvement >Reporter: niuyulin >Assignee: niuyulin >Priority: Minor > > large count of region task from canary may affect user normal request, > meanwhile if region task is few, the availability monitoring may shake for > occasional exception. > so , if the task count is large , we will randomly trim tasks for each table, > according to the raito of the table region count in whole tasks region count. > if the task count is few, we will repeat tasks -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700579690 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 34s | master passed | | +1 :green_heart: | checkstyle | 1m 36s | master passed | | +1 :green_heart: | spotbugs | 3m 8s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 0s | root in the patch failed. | | -0 :warning: | checkstyle | 0m 11s | hbase-hadoop-compat: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -0 :warning: | checkstyle | 0m 12s | hbase-procedure: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | -1 :x: | hadoopcheck | 1m 21s | The patch causes 11 errors with Hadoop v3.1.2. | | -1 :x: | hadoopcheck | 2m 36s | The patch causes 11 errors with Hadoop v3.2.1. | | -1 :x: | spotbugs | 0m 38s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | asflicense | 0m 30s | The patch generated 1 ASF License warnings. | | | | 18m 50s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 0b673d1406e7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-general-check/output/patch-mvninstall-root.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-hadoop-compat.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-procedure.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-general-check/output/patch-javac-3.1.2.txt | | hadoopcheck | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-general-check/output/patch-javac-3.2.1.txt | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-general-check/output/patch-spotbugs-hbase-server.txt | | asflicense | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-general-check/output/patch-asflicense-problems.txt | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700583801 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 10s | master passed | | +1 :green_heart: | compile | 1m 46s | master passed | | +1 :green_heart: | shadedjars | 6m 38s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 22s | hbase-hadoop-compat in master failed. | | -0 :warning: | javadoc | 0m 19s | hbase-procedure in master failed. | | -0 :warning: | javadoc | 0m 40s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 16s | root in the patch failed. | | -1 :x: | compile | 0m 44s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 44s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 2m 53s | patch has 11 errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 19s | hbase-hadoop-compat in the patch failed. | | -0 :warning: | javadoc | 0m 18s | hbase-procedure in the patch failed. | | -0 :warning: | javadoc | 0m 39s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 0m 21s | hbase-hadoop-compat in the patch failed. | | +1 :green_heart: | unit | 1m 39s | hbase-procedure in the patch passed. | | -1 :x: | unit | 0m 43s | hbase-server in the patch failed. | | | | 26m 9s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux edb18e752900 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/patch-shadedjars.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-hadoop-compat.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-Pre
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700587135 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 9s | master passed | | +1 :green_heart: | compile | 1m 49s | master passed | | +1 :green_heart: | shadedjars | 8m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 32s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 1m 25s | root in the patch failed. | | -1 :x: | compile | 0m 52s | hbase-server in the patch failed. | | -0 :warning: | javac | 0m 52s | hbase-server in the patch failed. | | -1 :x: | shadedjars | 3m 34s | patch has 11 errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 21s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 0m 25s | hbase-hadoop-compat in the patch failed. | | +1 :green_heart: | unit | 2m 24s | hbase-procedure in the patch passed. | | -1 :x: | unit | 0m 43s | hbase-server in the patch failed. | | | | 31m 33s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f3a695a5d02d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 1.8.0_232 | | mvninstall | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk8-hadoop3-check/output/patch-mvninstall-root.txt | | compile | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | javac | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk8-hadoop3-check/output/patch-compile-hbase-server.txt | | shadedjars | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk8-hadoop3-check/output/patch-shadedjars.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-hadoop-compat.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/testReport/ | | Max. process+thread count | 247 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/2/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-25116) RegionMonitor support RegionTask count normalize
[ https://issues.apache.org/jira/browse/HBASE-25116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] niuyulin updated HBASE-25116: - Description: large count of region task from canary may affect user normal request, meanwhile if region task is few, the availability monitoring may shake for occasional exception. so , if the task count is large , we will randomly trim tasks for each table, according to the raito of the table region count in whole tasks region count. If the task count is few, we will repeat tasks was: large count of region task from canary may affect user normal request, meanwhile if region task is few, the availability monitoring may shake for occasional exception. so , if the task count is large , we will randomly trim tasks for each table, according to the raito of the table region count in whole tasks region count. if the task count is few, we will repeat tasks > RegionMonitor support RegionTask count normalize > > > Key: HBASE-25116 > URL: https://issues.apache.org/jira/browse/HBASE-25116 > Project: HBase > Issue Type: Improvement >Reporter: niuyulin >Assignee: niuyulin >Priority: Minor > > large count of region task from canary may affect user normal request, > meanwhile if region task is few, the availability monitoring may shake for > occasional exception. > so , if the task count is large , we will randomly trim tasks for each table, > according to the raito of the table region count in whole tasks region count. > If the task count is few, we will repeat tasks -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25117) ReplicationSourceShipper thread can not be finished
Sun Xin created HBASE-25117: --- Summary: ReplicationSourceShipper thread can not be finished Key: HBASE-25117 URL: https://issues.apache.org/jira/browse/HBASE-25117 Project: HBase Issue Type: Bug Reporter: Sun Xin Assignee: Sun Xin See [Flaky Tests|https://ci-hadoop.apache.org/job/HBase/job/HBase-Flaky-Tests/job/master/161/console], some UTs about replication failed cause timeout. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
Apache-HBase commented on pull request #2194: URL: https://github.com/apache/hbase/pull/2194#issuecomment-700628756 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 8m 2s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 45s | branch-1 passed | | +1 :green_heart: | compile | 1m 3s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 53s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 2m 24s | branch-1 passed | | +1 :green_heart: | shadedjars | 5m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 16s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 1m 6s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 4m 39s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 32s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 7s | the patch passed | | +1 :green_heart: | compile | 1m 8s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 1m 8s | the patch passed | | +1 :green_heart: | compile | 1m 11s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 1m 11s | the patch passed | | +1 :green_heart: | checkstyle | 2m 26s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 24s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 6m 46s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 44s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 1m 3s | the patch passed with JDK v1.7.0_272 | | -1 :x: | findbugs | 4m 22s | hbase-server generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 :green_heart: | unit | 162m 33s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. | | | | 229m 31s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Sequence of calls to java.util.concurrent.ConcurrentHashMap may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason) At LogRoller.java:may not be atomic in org.apache.hadoop.hbase.regionserver.LogRoller$1.logRollRequested(WALActionsListener$RollRequestReason) At LogRoller.java:[line 81] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2194 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux f4e8d0bd9fb5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2194/out/precommit/personality/provided.sh | | git revision | branch-1 / 93b76fd | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | findbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/6/artifact/out/new-findbugs-hbase-server.html | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/6/testReport/ | | Max. process+thread count | 4580 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2194/6/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. -
[GitHub] [hbase] Reidddddd commented on pull request #2194: HBASE-24849 Branch-1 Backport : HBASE-24665 MultiWAL : Avoid rolling of ALL WALs when one of the WAL needs a roll
Reidd commented on pull request #2194: URL: https://github.com/apache/hbase/pull/2194#issuecomment-700629560 Please take a look at the `FindBugs` warning. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700631938 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 26s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 33s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 55s | master passed | | +1 :green_heart: | checkstyle | 2m 6s | master passed | | +1 :green_heart: | spotbugs | 4m 8s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 9s | the patch passed | | -0 :warning: | checkstyle | 0m 17s | hbase-hadoop-compat: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | -0 :warning: | checkstyle | 0m 18s | hbase-procedure: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 17m 44s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 5m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 52s | The patch does not generate ASF License warnings. | | | | 58m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 1daac378438e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-hadoop-compat.txt | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-procedure.txt | | Max. process+thread count | 85 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2450: HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE
Apache-HBase commented on pull request #2450: URL: https://github.com/apache/hbase/pull/2450#issuecomment-700637532 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 50s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 30s | master passed | | +1 :green_heart: | compile | 1m 39s | master passed | | +1 :green_heart: | shadedjars | 9m 23s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 1m 5s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 6s | the patch passed | | +1 :green_heart: | compile | 1m 34s | the patch passed | | +1 :green_heart: | javac | 1m 34s | the patch passed | | +1 :green_heart: | shadedjars | 10m 8s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 47s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 146m 44s | hbase-server in the patch passed. | | | | 187m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2450 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d21c8aa90d75 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b268b1f621 | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/testReport/ | | Max. process+thread count | 3898 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2450: HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE
Apache-HBase commented on pull request #2450: URL: https://github.com/apache/hbase/pull/2450#issuecomment-700640396 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 55s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 58s | master passed | | +1 :green_heart: | compile | 1m 10s | master passed | | +1 :green_heart: | shadedjars | 7m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 3s | the patch passed | | +1 :green_heart: | compile | 1m 5s | the patch passed | | +1 :green_heart: | javac | 1m 5s | the patch passed | | +1 :green_heart: | shadedjars | 8m 55s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 161m 14s | hbase-server in the patch passed. | | | | 193m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2450 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fb7ee3dd00e9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / b268b1f621 | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/testReport/ | | Max. process+thread count | 4794 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
virajjasani commented on a change in pull request #2454:
URL: https://github.com/apache/hbase/pull/2454#discussion_r496654064
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizerWorkQueue.java
##
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.LinkedHashSet;
+import java.util.Queue;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.locks.Condition;
+import java.util.concurrent.locks.ReentrantLock;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * A specialized collection that holds pending work for the {@link
RegionNormalizerWorker}. It is
+ * an ordered collection class that has the following properties:
+ *
+ * Guarantees uniqueness of elements, as a {@link Set}.
+ * Consumers retrieve objects from the head, as a {@link Queue}, via
{@link #take()}.
+ * Work is retrieved on a FIFO policy.
+ * Work retrieval blocks the calling thread until new work is available,
as a
+ * {@link BlockingQueue}.
+ * Allows a producer to insert an item at the head of the queue, if
desired.
+ *
+ * Assumes low-frequency and low-parallelism concurrent access, so protects
state using a
+ * simplistic synchronization strategy.
+ */
+@InterfaceAudience.Private
+class RegionNormalizerWorkQueue {
+
+ /** Underlying storage structure that gives us the Set behavior and FIFO
retrieval policy. */
+ private LinkedHashSet delegate;
+
+ // the locking structure used here follows the example found in
LinkedBlockingQueue. The
+ // difference is that our locks guard access to `delegate` rather than the
head node.
+
+ /** Lock held by take, poll, etc */
+ private final ReentrantLock takeLock;
+
+ /** Wait queue for waiting takes */
+ private final Condition notEmpty;
+
+ /** Lock held by put, offer, etc */
+ private final ReentrantLock putLock;
+
+ public RegionNormalizerWorkQueue() {
+delegate = new LinkedHashSet<>();
+takeLock = new ReentrantLock();
+notEmpty = takeLock.newCondition();
+putLock = new ReentrantLock();
+ }
+
+ /**
+ * Signals a waiting take. Called only from put/offer (which do not
+ * otherwise ordinarily lock takeLock.)
+ */
+ private void signalNotEmpty() {
+final ReentrantLock takeLock = this.takeLock;
Review comment:
Glad to know this!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2459: HBASE-24999 Master manages ReplicationServers
Apache-HBase commented on pull request #2459: URL: https://github.com/apache/hbase/pull/2459#issuecomment-700654495 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | prototool | 0m 0s | prototool was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-24666 Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 59s | HBASE-24666 passed | | +1 :green_heart: | checkstyle | 1m 37s | HBASE-24666 passed | | -1 :x: | spotbugs | 2m 29s | hbase-server in HBASE-24666 has 3 extant spotbugs warnings. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 58s | the patch passed | | +1 :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-protocol-shaded | | +1 :green_heart: | checkstyle | 1m 39s | hbase-server: The patch generated 0 new + 147 unchanged - 3 fixed = 147 total (was 150) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 20s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | hbaseprotoc | 2m 33s | the patch passed | | +1 :green_heart: | spotbugs | 5m 30s | hbase-protocol-shaded in the patch passed. | | +1 :green_heart: | spotbugs | 3m 6s | hbase-server generated 0 new + 0 unchanged - 3 fixed = 0 total (was 3) | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 60m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2459/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2459 | | Optional Tests | dupname asflicense cc hbaseprotoc prototool spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux abd3f7b2a48c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-24666 / 827ac96a21 | | spotbugs | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2459/4/artifact/yetus-general-check/output/branch-spotbugs-hbase-server-warnings.html | | Max. process+thread count | 84 (vs. ulimit of 3) | | modules | C: hbase-protocol-shaded hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2459/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2470: HBASE-25114 [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose
Apache-HBase commented on pull request #2470: URL: https://github.com/apache/hbase/pull/2470#issuecomment-700663289 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 39s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 31s | branch-1 passed | | +1 :green_heart: | compile | 0m 57s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 55s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 2m 9s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 49s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 0s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 53s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 4m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 2s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 30s | the patch passed | | +1 :green_heart: | compile | 0m 53s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 53s | the patch passed | | +1 :green_heart: | compile | 0m 56s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 56s | the patch passed | | +1 :green_heart: | checkstyle | 2m 6s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 3m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 6m 7s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 46s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 57s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 4m 1s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 197m 34s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 51s | The patch does not generate ASF License warnings. | | | | 250m 55s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestAdmin2 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2470/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2470 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2d46ced5cf1d 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2470/out/precommit/personality/provided.sh | | git revision | branch-1 / c7891d8 | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2470/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2470/1/testReport/ | | Max. process+thread count | 3981 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2470/1/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
infraio commented on a change in pull request #2469:
URL: https://github.com/apache/hbase/pull/2469#discussion_r496678196
##
File path:
hbase-hadoop-compat/src/main/java/org/apache/hadoop/hbase/metrics/OperationMetricsWithMasterUI.java
##
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hbase.metrics;
+
+import org.apache.hadoop.hbase.metrics.impl.HistogramImpl;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * Container class for commonly collected metrics for most operations.
Instantiate this class to
+ * collect submitted count, failed count and time histogram for an
operation.Furthermore, there are
+ * some cases where we should keep the history of some metrics for master UI,
+ */
+@InterfaceAudience.Private
+public class OperationMetricsWithMasterUI extends OperationMetrics {
Review comment:
OperationMetricsWithMasterUI => OperationMetricsForMasterUI?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
infraio commented on a change in pull request #2469: URL: https://github.com/apache/hbase/pull/2469#discussion_r496678358 ## File path: hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/ProcedureMetrics.java ## @@ -49,4 +49,10 @@ * @return Total number of instances failed for a type of a procedure */ Counter getFailedCounter(); + + /** + * @return Histogram of runtimes for all successfully completed instances of a type of a procedure + * on master UI only + */ + Histogram getTimeHistoOnMasterUI(); Review comment: No need to add to interface? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] infraio commented on a change in pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
infraio commented on a change in pull request #2469:
URL: https://github.com/apache/hbase/pull/2469#discussion_r496678918
##
File path:
hbase-procedure/src/main/java/org/apache/hadoop/hbase/procedure2/Procedure.java
##
@@ -439,6 +439,10 @@ protected void updateMetricsOnFinish(TEnvironment env,
long runtime, boolean suc
if (timeHisto != null) {
timeHisto.update(runtime);
}
+ Histogram timeHistoOnMasterUI = metrics.getTimeHistoOnMasterUI();
Review comment:
You can use "instance of" and cast to OperationMetricsForMasterUI. Then
no need to add the method to interface?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-25017) Attach a design doc to code base
[
https://issues.apache.org/jira/browse/HBASE-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17203913#comment-17203913
]
Hudson commented on HBASE-25017:
Results for branch master
[build #72 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/72/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/72/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/72/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/72/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Attach a design doc to code base
>
>
> Key: HBASE-25017
> URL: https://issues.apache.org/jira/browse/HBASE-25017
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
> Environment: Ata
>Reporter: Duo Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2450: HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE
Apache-HBase commented on pull request #2450: URL: https://github.com/apache/hbase/pull/2450#issuecomment-700711182 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 46s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 3s | master passed | | +1 :green_heart: | checkstyle | 1m 24s | master passed | | +1 :green_heart: | spotbugs | 2m 41s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 31s | the patch passed | | +1 :green_heart: | checkstyle | 1m 22s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 20s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 41m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2450 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux c01b84d6b177 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700720385 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 24s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 25s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 6m 23s | master passed | | +1 :green_heart: | compile | 2m 32s | master passed | | +1 :green_heart: | shadedjars | 10m 23s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 25s | hbase-hadoop-compat in master failed. | | -0 :warning: | javadoc | 0m 22s | hbase-procedure in master failed. | | -0 :warning: | javadoc | 0m 56s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 6m 12s | the patch passed | | +1 :green_heart: | compile | 2m 31s | the patch passed | | +1 :green_heart: | javac | 2m 31s | the patch passed | | +1 :green_heart: | shadedjars | 10m 13s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 24s | hbase-hadoop-compat in the patch failed. | | -0 :warning: | javadoc | 0m 21s | hbase-procedure in the patch failed. | | -0 :warning: | javadoc | 0m 56s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 45s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 2m 7s | hbase-procedure in the patch passed. | | -1 :x: | unit | 171m 8s | hbase-server in the patch failed. | | | | 222m 19s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 39feee6859f9 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/testReport/ | | Max. process+thread count | 3704 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700742771 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 22s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 17s | master passed | | +1 :green_heart: | compile | 1m 44s | master passed | | +1 :green_heart: | shadedjars | 7m 56s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 29s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 46s | the patch passed | | +1 :green_heart: | compile | 1m 56s | the patch passed | | +1 :green_heart: | javac | 1m 56s | the patch passed | | +1 :green_heart: | shadedjars | 8m 28s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 43s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 2m 24s | hbase-procedure in the patch passed. | | -1 :x: | unit | 216m 30s | hbase-server in the patch failed. | | | | 255m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7ffcc9f431e9 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/testReport/ | | Max. process+thread count | 3243 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700778068 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 16s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 35s | master passed | | +1 :green_heart: | checkstyle | 1m 58s | master passed | | +1 :green_heart: | spotbugs | 3m 47s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 36s | the patch passed | | -0 :warning: | checkstyle | 0m 15s | hbase-hadoop-compat: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 14m 35s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 4m 27s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 49m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux ae62134e6f04 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-hadoop-compat.txt | | Max. process+thread count | 85 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25107) Migrate flaky reporting jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204072#comment-17204072
]
Hudson commented on HBASE-25107:
Results for branch branch-2
[build #62 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Migrate flaky reporting jenkins job from Hadoop to hbase
>
>
> Key: HBASE-25107
> URL: https://issues.apache.org/jira/browse/HBASE-25107
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25103) Remove ZNodePaths.metaReplicaZNodes
[
https://issues.apache.org/jira/browse/HBASE-25103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204073#comment-17204073
]
Hudson commented on HBASE-25103:
Results for branch branch-2
[build #62 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove ZNodePaths.metaReplicaZNodes
> ---
>
> Key: HBASE-25103
> URL: https://issues.apache.org/jira/browse/HBASE-25103
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> We have already removed the dependency on this field from client side.
> Now if we want to remove the META_REPLICAS_NUM field, we must remove the
> usage of this field first.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25017) Attach a design doc to code base
[
https://issues.apache.org/jira/browse/HBASE-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204071#comment-17204071
]
Hudson commented on HBASE-25017:
Results for branch branch-2
[build #62 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(x) {color:red}-1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/62/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Attach a design doc to code base
>
>
> Key: HBASE-25017
> URL: https://issues.apache.org/jira/browse/HBASE-25017
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
> Environment: Ata
>Reporter: Duo Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2450: HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE
Apache-HBase commented on pull request #2450: URL: https://github.com/apache/hbase/pull/2450#issuecomment-700811688 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 5s | master passed | | +1 :green_heart: | compile | 1m 25s | master passed | | +1 :green_heart: | shadedjars | 8m 3s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 47s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 51s | the patch passed | | +1 :green_heart: | compile | 1m 17s | the patch passed | | +1 :green_heart: | javac | 1m 17s | the patch passed | | +1 :green_heart: | shadedjars | 8m 15s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 53s | hbase-server in the patch failed. | ||| _ Other Tests _ | | -1 :x: | unit | 144m 27s | hbase-server in the patch failed. | | | | 179m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2450 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7f0403980b56 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/testReport/ | | Max. process+thread count | 4153 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25107) Migrate flaky reporting jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204103#comment-17204103
]
Hudson commented on HBASE-25107:
Results for branch branch-1
[build #33 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Migrate flaky reporting jenkins job from Hadoop to hbase
>
>
> Key: HBASE-25107
> URL: https://issues.apache.org/jira/browse/HBASE-25107
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25017) Attach a design doc to code base
[
https://issues.apache.org/jira/browse/HBASE-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204102#comment-17204102
]
Hudson commented on HBASE-25017:
Results for branch branch-1
[build #33 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Attach a design doc to code base
>
>
> Key: HBASE-25017
> URL: https://issues.apache.org/jira/browse/HBASE-25017
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
> Environment: Ata
>Reporter: Duo Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25025) [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp
[
https://issues.apache.org/jira/browse/HBASE-25025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204101#comment-17204101
]
Hudson commented on HBASE-25025:
Results for branch branch-1
[build #33 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/33//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> [Flaky Test][branch-1] TestFromClientSide#testCheckAndDeleteWithCompareOp
> -
>
> Key: HBASE-25025
> URL: https://issues.apache.org/jira/browse/HBASE-25025
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Reid Chan
>Assignee: Reid Chan
>Priority: Minor
> Fix For: 1.7.0
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Reidddddd merged pull request #2470: HBASE-25114 [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose
Reidd merged pull request #2470: URL: https://github.com/apache/hbase/pull/2470 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-25114) [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose
[ https://issues.apache.org/jira/browse/HBASE-25114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reid Chan resolved HBASE-25114. --- Fix Version/s: 1.7.0 Hadoop Flags: Reviewed Resolution: Fixed > [Flake Test][branch-1] TestFromClientSide#testCacheOnWriteEvictOnClose > -- > > Key: HBASE-25114 > URL: https://issues.apache.org/jira/browse/HBASE-25114 > Project: HBase > Issue Type: Improvement >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.7.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25017) Attach a design doc to code base
[
https://issues.apache.org/jira/browse/HBASE-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204127#comment-17204127
]
Hudson commented on HBASE-25017:
Results for branch branch-2.3
[build #65 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/65/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/65/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/65/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/65/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/65/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Attach a design doc to code base
>
>
> Key: HBASE-25017
> URL: https://issues.apache.org/jira/browse/HBASE-25017
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
> Environment: Ata
>Reporter: Duo Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25107) Migrate flaky reporting jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204128#comment-17204128
]
Hudson commented on HBASE-25107:
Results for branch branch-2.3
[build #65 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/65/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/65/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/65/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/65/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/65/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Migrate flaky reporting jenkins job from Hadoop to hbase
>
>
> Key: HBASE-25107
> URL: https://issues.apache.org/jira/browse/HBASE-25107
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700864543 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 30s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 30s | master passed | | +1 :green_heart: | compile | 2m 0s | master passed | | +1 :green_heart: | shadedjars | 7m 10s | branch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 23s | hbase-hadoop-compat in master failed. | | -0 :warning: | javadoc | 0m 19s | hbase-procedure in master failed. | | -0 :warning: | javadoc | 0m 45s | hbase-server in master failed. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 18s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 23s | the patch passed | | +1 :green_heart: | compile | 2m 1s | the patch passed | | +1 :green_heart: | javac | 2m 1s | the patch passed | | +1 :green_heart: | shadedjars | 7m 9s | patch has no errors when building our shaded downstream artifacts. | | -0 :warning: | javadoc | 0m 22s | hbase-hadoop-compat in the patch failed. | | -0 :warning: | javadoc | 0m 19s | hbase-procedure in the patch failed. | | -0 :warning: | javadoc | 0m 46s | hbase-server in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 41s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 1m 44s | hbase-procedure in the patch passed. | | -1 :x: | unit | 141m 14s | hbase-server in the patch failed. | | | | 177m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux e338ccfd535f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 2020-01-14 | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-procedure.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-jdk11-hadoop3-check/output/branch-javadoc-hbase-server.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-hadoop-compat.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-procedure.txt | | javadoc | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-jdk11-hadoop3-check/output/patch-javadoc-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/testReport/ | | Max. process+thread count | 3657 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-23287) WALs not aged off of HDFS because LogCleaner is not added to choreService
[
https://issues.apache.org/jira/browse/HBASE-23287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204160#comment-17204160
]
Maziar Mirzazad commented on HBASE-23287:
-
[~DeanZ] could you please fix 1.5.0 too?
> WALs not aged off of HDFS because LogCleaner is not added to choreService
> -
>
> Key: HBASE-23287
> URL: https://issues.apache.org/jira/browse/HBASE-23287
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.5.0, 1.6.0, 1.4.11
>Reporter: Baiqiang Zhao
>Assignee: Baiqiang Zhao
>Priority: Blocker
> Fix For: 1.6.0, 1.4.12
>
> Attachments: HBASE-23287.branch-1.0001.patch
>
>
> We found /hbase/oldWALs keeps growing, and we never use replication feature.
> Then we found OldWALsCleaner thread is always in WAITTING state :
> {code:java}
> Thread 176 (OldWALsCleaner-1): State: WAITING Blocked count: 0 Waited count:
> 1 Waiting on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2ef05d36
> Stack: sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.deleteFile(LogCleaner.java:185)
>
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.access$000(LogCleaner.java:48)
> org.apache.hadoop.hbase.master.cleaner.LogCleaner$1.run(LogCleaner.java:161)
> java.lang.Thread.run(Thread.java:745)
> {code}
> Finally, we found that LogCleaner is not added to choreService, so LogCleaner
> doesn't run periodically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25107) Migrate flaky reporting jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204170#comment-17204170
]
Hudson commented on HBASE-25107:
Results for branch branch-1.4
[build #6 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/6/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/6//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/6//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/6//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Migrate flaky reporting jenkins job from Hadoop to hbase
>
>
> Key: HBASE-25107
> URL: https://issues.apache.org/jira/browse/HBASE-25107
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25017) Attach a design doc to code base
[
https://issues.apache.org/jira/browse/HBASE-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204169#comment-17204169
]
Hudson commented on HBASE-25017:
Results for branch branch-1.4
[build #6 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/6/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/6//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/6//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1.4/6//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 source release artifact{color}
-- See build output for details.
> Attach a design doc to code base
>
>
> Key: HBASE-25017
> URL: https://issues.apache.org/jira/browse/HBASE-25017
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
> Environment: Ata
>Reporter: Duo Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2450: HBASE-25093 the RSGroupBasedLoadBalancer#retainAssignment throws NPE
Apache-HBase commented on pull request #2450: URL: https://github.com/apache/hbase/pull/2450#issuecomment-700880544 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 7s | master passed | | +1 :green_heart: | compile | 1m 23s | master passed | | +1 :green_heart: | shadedjars | 8m 43s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 51s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 50s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | +1 :green_heart: | shadedjars | 8m 45s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 250m 55s | hbase-server in the patch passed. | | | | 286m 6s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2450 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux f59803f04456 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 1.8.0_232 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/testReport/ | | Max. process+thread count | 3280 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2450/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
virajjasani commented on a change in pull request #2454:
URL: https://github.com/apache/hbase/pull/2454#discussion_r496771793
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizerWorker.java
##
@@ -0,0 +1,254 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import java.io.IOException;
+import java.time.Duration;
+import java.util.Collections;
+import java.util.List;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.master.HMaster;
+import org.apache.hadoop.hbase.master.MasterServices;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import
org.apache.hbase.thirdparty.com.google.common.util.concurrent.RateLimiter;
+import
org.apache.hbase.thirdparty.org.apache.commons.collections4.CollectionUtils;
+
+/**
+ * Consumes normalization request targets ({@link TableName}s) off the
+ * {@link RegionNormalizerWorkQueue}, dispatches them to the {@link
RegionNormalizer},
+ * and executes the resulting {@link NormalizationPlan}s.
+ */
+@InterfaceAudience.Private
+class RegionNormalizerWorker implements Runnable {
+ private static final Logger LOG =
LoggerFactory.getLogger(RegionNormalizerWorker.class);
+ private static final String RATE_LIMIT_BYTES_PER_SEC_KEY =
+"hbase.normalizer.throughput.max_bytes_per_sec";
+ private static final long RATE_UNLIMITED_BYTES = 1_000_000_000_000L; //
1TB/sec
+
+ private final MasterServices masterServices;
+ private final RegionNormalizer regionNormalizer;
+ private final RegionNormalizerWorkQueue workQueue;
+ private final RateLimiter rateLimiter;
+
+ private final long[] skippedCount;
+ private long splitPlanCount;
+ private long mergePlanCount;
+
+ public RegionNormalizerWorker(
+final Configuration configuration,
+final MasterServices masterServices,
+final RegionNormalizer regionNormalizer,
+final RegionNormalizerWorkQueue workQueue
+ ) {
+this.masterServices = masterServices;
+this.regionNormalizer = regionNormalizer;
+this.workQueue = workQueue;
+this.skippedCount = new long[NormalizationPlan.PlanType.values().length];
+this.splitPlanCount = 0;
+this.mergePlanCount = 0;
+this.rateLimiter = loadRateLimiter(configuration);
+ }
+
+ private static RateLimiter loadRateLimiter(final Configuration
configuration) {
+long rateLimitBytes =
+ configuration.getLongBytes(RATE_LIMIT_BYTES_PER_SEC_KEY,
RATE_UNLIMITED_BYTES);
+long rateLimitMbs = rateLimitBytes / 1_000_000L;
+if (rateLimitMbs <= 0) {
+ LOG.warn("Configured value {}={} is <= 1MB. Falling back to default.",
+RATE_LIMIT_BYTES_PER_SEC_KEY, rateLimitBytes);
+ rateLimitBytes = RATE_UNLIMITED_BYTES;
+ rateLimitMbs = RATE_UNLIMITED_BYTES / 1_000_000L;
+}
+LOG.info("Normalizer rate limit set to {}",
+ rateLimitBytes == RATE_UNLIMITED_BYTES ? "unlimited" : rateLimitMbs + "
MB/sec");
+return RateLimiter.create(rateLimitMbs);
+ }
+
+ /**
+ * @see RegionNormalizerManager#planSkipped(NormalizationPlan.PlanType)
+ */
+ void planSkipped(NormalizationPlan.PlanType type) {
+synchronized (skippedCount) {
+ // updates come here via procedure threads, so synchronize access to
this counter.
+ skippedCount[type.ordinal()]++;
+}
+ }
+
+ /**
+ * @see RegionNormalizerManager#getSkippedCount(NormalizationPlan.PlanType)
+ */
+ long getSkippedCount(NormalizationPlan.PlanType type) {
+return skippedCount[type.ordinal()];
+ }
+
+ /**
+ * @see HMaster#getSplitPlanCount()
+ */
+ long getSplitPlanCount() {
+return splitPlanCount;
+ }
+
+ /**
+ * @see HMaster#getMergePlanCount()
+ */
+ long getMergePlanCount() {
+return mergePlanCount;
+ }
+
+ @Override
+ public void run() {
+while (true) {
+ if (Thread.interrupted()) {
+LOG.debug("interrupt detected. terminating.");
+break;
+ }
+
[jira] [Commented] (HBASE-23287) WALs not aged off of HDFS because LogCleaner is not added to choreService
[
https://issues.apache.org/jira/browse/HBASE-23287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204194#comment-17204194
]
Viraj Jasani commented on HBASE-23287:
--
[~m.mirzazad], 1.5 does not have maintenance release planned. If you are
already on 1.5.0, it is definitely recommended to upgrade to 1.6.0. Hope this
helps.
> WALs not aged off of HDFS because LogCleaner is not added to choreService
> -
>
> Key: HBASE-23287
> URL: https://issues.apache.org/jira/browse/HBASE-23287
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.5.0, 1.6.0, 1.4.11
>Reporter: Baiqiang Zhao
>Assignee: Baiqiang Zhao
>Priority: Blocker
> Fix For: 1.6.0, 1.4.12
>
> Attachments: HBASE-23287.branch-1.0001.patch
>
>
> We found /hbase/oldWALs keeps growing, and we never use replication feature.
> Then we found OldWALsCleaner thread is always in WAITTING state :
> {code:java}
> Thread 176 (OldWALsCleaner-1): State: WAITING Blocked count: 0 Waited count:
> 1 Waiting on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2ef05d36
> Stack: sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.deleteFile(LogCleaner.java:185)
>
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.access$000(LogCleaner.java:48)
> org.apache.hadoop.hbase.master.cleaner.LogCleaner$1.run(LogCleaner.java:161)
> java.lang.Thread.run(Thread.java:745)
> {code}
> Finally, we found that LogCleaner is not added to choreService, so LogCleaner
> doesn't run periodically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
virajjasani commented on a change in pull request #2454:
URL: https://github.com/apache/hbase/pull/2454#discussion_r496771793
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizerWorker.java
##
@@ -0,0 +1,254 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import java.io.IOException;
+import java.time.Duration;
+import java.util.Collections;
+import java.util.List;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.master.HMaster;
+import org.apache.hadoop.hbase.master.MasterServices;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import
org.apache.hbase.thirdparty.com.google.common.util.concurrent.RateLimiter;
+import
org.apache.hbase.thirdparty.org.apache.commons.collections4.CollectionUtils;
+
+/**
+ * Consumes normalization request targets ({@link TableName}s) off the
+ * {@link RegionNormalizerWorkQueue}, dispatches them to the {@link
RegionNormalizer},
+ * and executes the resulting {@link NormalizationPlan}s.
+ */
+@InterfaceAudience.Private
+class RegionNormalizerWorker implements Runnable {
+ private static final Logger LOG =
LoggerFactory.getLogger(RegionNormalizerWorker.class);
+ private static final String RATE_LIMIT_BYTES_PER_SEC_KEY =
+"hbase.normalizer.throughput.max_bytes_per_sec";
+ private static final long RATE_UNLIMITED_BYTES = 1_000_000_000_000L; //
1TB/sec
+
+ private final MasterServices masterServices;
+ private final RegionNormalizer regionNormalizer;
+ private final RegionNormalizerWorkQueue workQueue;
+ private final RateLimiter rateLimiter;
+
+ private final long[] skippedCount;
+ private long splitPlanCount;
+ private long mergePlanCount;
+
+ public RegionNormalizerWorker(
+final Configuration configuration,
+final MasterServices masterServices,
+final RegionNormalizer regionNormalizer,
+final RegionNormalizerWorkQueue workQueue
+ ) {
+this.masterServices = masterServices;
+this.regionNormalizer = regionNormalizer;
+this.workQueue = workQueue;
+this.skippedCount = new long[NormalizationPlan.PlanType.values().length];
+this.splitPlanCount = 0;
+this.mergePlanCount = 0;
+this.rateLimiter = loadRateLimiter(configuration);
+ }
+
+ private static RateLimiter loadRateLimiter(final Configuration
configuration) {
+long rateLimitBytes =
+ configuration.getLongBytes(RATE_LIMIT_BYTES_PER_SEC_KEY,
RATE_UNLIMITED_BYTES);
+long rateLimitMbs = rateLimitBytes / 1_000_000L;
+if (rateLimitMbs <= 0) {
+ LOG.warn("Configured value {}={} is <= 1MB. Falling back to default.",
+RATE_LIMIT_BYTES_PER_SEC_KEY, rateLimitBytes);
+ rateLimitBytes = RATE_UNLIMITED_BYTES;
+ rateLimitMbs = RATE_UNLIMITED_BYTES / 1_000_000L;
+}
+LOG.info("Normalizer rate limit set to {}",
+ rateLimitBytes == RATE_UNLIMITED_BYTES ? "unlimited" : rateLimitMbs + "
MB/sec");
+return RateLimiter.create(rateLimitMbs);
+ }
+
+ /**
+ * @see RegionNormalizerManager#planSkipped(NormalizationPlan.PlanType)
+ */
+ void planSkipped(NormalizationPlan.PlanType type) {
+synchronized (skippedCount) {
+ // updates come here via procedure threads, so synchronize access to
this counter.
+ skippedCount[type.ordinal()]++;
+}
+ }
+
+ /**
+ * @see RegionNormalizerManager#getSkippedCount(NormalizationPlan.PlanType)
+ */
+ long getSkippedCount(NormalizationPlan.PlanType type) {
+return skippedCount[type.ordinal()];
+ }
+
+ /**
+ * @see HMaster#getSplitPlanCount()
+ */
+ long getSplitPlanCount() {
+return splitPlanCount;
+ }
+
+ /**
+ * @see HMaster#getMergePlanCount()
+ */
+ long getMergePlanCount() {
+return mergePlanCount;
+ }
+
+ @Override
+ public void run() {
+while (true) {
+ if (Thread.interrupted()) {
+LOG.debug("interrupt detected. terminating.");
+break;
+ }
+
[GitHub] [hbase] virajjasani commented on a change in pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
virajjasani commented on a change in pull request #2454:
URL: https://github.com/apache/hbase/pull/2454#discussion_r496959173
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizerWorkQueue.java
##
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Iterator;
+import java.util.LinkedHashSet;
+import java.util.Queue;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.locks.Condition;
+import java.util.concurrent.locks.ReentrantLock;
+import org.apache.yetus.audience.InterfaceAudience;
+
+/**
+ * A specialized collection that holds pending work for the {@link
RegionNormalizerWorker}. It is
+ * an ordered collection class that has the following properties:
+ *
+ * Guarantees uniqueness of elements, as a {@link Set}.
+ * Consumers retrieve objects from the head, as a {@link Queue}, via
{@link #take()}.
+ * Work is retrieved on a FIFO policy.
+ * Work retrieval blocks the calling thread until new work is available,
as a
+ * {@link BlockingQueue}.
+ * Allows a producer to insert an item at the head of the queue, if
desired.
+ *
+ * Assumes low-frequency and low-parallelism concurrent access, so protects
state using a
+ * simplistic synchronization strategy.
+ */
+@InterfaceAudience.Private
+class RegionNormalizerWorkQueue {
+
+ /** Underlying storage structure that gives us the Set behavior and FIFO
retrieval policy. */
+ private LinkedHashSet delegate;
+
+ // the locking structure used here follows the example found in
LinkedBlockingQueue. The
+ // difference is that our locks guard access to `delegate` rather than the
head node.
+
+ /** Lock held by take, poll, etc */
+ private final ReentrantLock takeLock;
+
+ /** Wait queue for waiting takes */
+ private final Condition notEmpty;
+
+ /** Lock held by put, offer, etc */
+ private final ReentrantLock putLock;
+
+ public RegionNormalizerWorkQueue() {
+delegate = new LinkedHashSet<>();
+takeLock = new ReentrantLock();
+notEmpty = takeLock.newCondition();
+putLock = new ReentrantLock();
+ }
+
+ /**
+ * Signals a waiting take. Called only from put/offer (which do not
+ * otherwise ordinarily lock takeLock.)
+ */
+ private void signalNotEmpty() {
+final ReentrantLock takeLock = this.takeLock;
+takeLock.lock();
+try {
+ notEmpty.signal();
+} finally {
+ takeLock.unlock();
+}
+ }
+
+ /**
+ * Locks to prevent both puts and takes.
+ */
+ private void fullyLock() {
+putLock.lock();
+takeLock.lock();
+ }
+
+ /**
+ * Unlocks to allow both puts and takes.
+ */
+ private void fullyUnlock() {
+takeLock.unlock();
+putLock.unlock();
+ }
+
+ /**
+ * Inserts the specified element at the tail of the queue, if it's not
already present.
+ *
+ * @param e the element to add
+ */
+ public void put(E e) {
+if (e == null) {
+ throw new NullPointerException();
+}
+
+putLock.lock();
+try {
+ delegate.add(e);
+} finally {
+ putLock.unlock();
+}
+
+if (!delegate.isEmpty()) {
+ signalNotEmpty();
+}
+ }
+
+ /**
+ * Inserts the specified element at the head of the queue.
+ *
+ * @param e the element to add
+ */
+ public void putFirst(E e) {
+if (e == null) {
+ throw new NullPointerException();
+}
+putAllFirst(Collections.singleton(e));
+ }
+
+ /**
+ * Inserts the specified elements at the tail of the queue. Any elements
already present in
+ * the queue are ignored.
+ *
+ * @param c the elements to add
+ */
+ public void putAll(Collection c) {
+if (c == null) {
+ throw new NullPointerException();
+}
+
+putLock.lock();
+try {
+ delegate.addAll(c);
+} finally {
+ putLock.unlock();
+}
+
+if (!delegate.isEmpty()) {
+ signalNotEmpty();
+}
+ }
+
+ /**
+ * Inserts the specified elements at the head of the queue.
+ *
+ * @param c the elements to add
+ */
+ public void putAllFirst(Collection c) {
+if (
[GitHub] [hbase] ndimiduk commented on a change in pull request #1915: HBASE-23970 TestUsersOperationsWithSecureHadoop fails when an existing ticket is present
ndimiduk commented on a change in pull request #1915:
URL: https://github.com/apache/hbase/pull/1915#discussion_r496959374
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/security/TestUsersOperationsWithSecureHadoop.java
##
@@ -64,7 +66,15 @@
private static String CLIENT_NAME;
@BeforeClass
- public static void setUp() throws Exception {
+ public static void checkAndSetUp() throws Exception {
+// check localhost kerberos users
+Process process = Runtime.getRuntime().exec(new String[]{"bash", "-c",
"klist"});
+boolean timeout = process.waitFor(2, TimeUnit.SECONDS);
+assertTrue("localhost exec klist timeout!", timeout);
+int ret = process.exitValue();
+assertNotEquals("localhost have an existing ticket!", 0, ret);
Review comment:
If you want to handle it like this, it would be better to skip/ignore
the test than to fail it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
ndimiduk commented on a change in pull request #2454:
URL: https://github.com/apache/hbase/pull/2454#discussion_r496987197
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -371,7 +358,11 @@ private boolean skipForMerge(final RegionStates
regionStates, final RegionInfo r
final long nextSizeMb = getRegionSizeMB(next);
// always merge away empty regions when they present themselves.
if (currentSizeMb == 0 || nextSizeMb == 0 || currentSizeMb + nextSizeMb
< avgRegionSizeMb) {
-plans.add(new MergeNormalizationPlan(current, next));
+final MergeNormalizationPlan plan = new
MergeNormalizationPlan.Builder()
+ .addTarget(current, currentSizeMb)
+ .addTarget(next, nextSizeMb)
Review comment:
Yes, merging more than two regions at a time is coming in
[HBASE-24419](https://issues.apache.org/jira/browse/HBASE-24419). I started
working on it yesterday. And yes, this constructor API for
MergeNormalizationPlan will likely change because of it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] ndimiduk commented on a change in pull request #2454: HBASE-24628 Region normalizer now respects a rate limit
ndimiduk commented on a change in pull request #2454:
URL: https://github.com/apache/hbase/pull/2454#discussion_r496989734
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizerWorker.java
##
@@ -0,0 +1,254 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.master.normalizer;
+
+import java.io.IOException;
+import java.time.Duration;
+import java.util.Collections;
+import java.util.List;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HConstants;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.RegionInfo;
+import org.apache.hadoop.hbase.client.TableDescriptor;
+import org.apache.hadoop.hbase.master.HMaster;
+import org.apache.hadoop.hbase.master.MasterServices;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import
org.apache.hbase.thirdparty.com.google.common.util.concurrent.RateLimiter;
+import
org.apache.hbase.thirdparty.org.apache.commons.collections4.CollectionUtils;
+
+/**
+ * Consumes normalization request targets ({@link TableName}s) off the
+ * {@link RegionNormalizerWorkQueue}, dispatches them to the {@link
RegionNormalizer},
+ * and executes the resulting {@link NormalizationPlan}s.
+ */
+@InterfaceAudience.Private
+class RegionNormalizerWorker implements Runnable {
+ private static final Logger LOG =
LoggerFactory.getLogger(RegionNormalizerWorker.class);
+ private static final String RATE_LIMIT_BYTES_PER_SEC_KEY =
+"hbase.normalizer.throughput.max_bytes_per_sec";
+ private static final long RATE_UNLIMITED_BYTES = 1_000_000_000_000L; //
1TB/sec
+
+ private final MasterServices masterServices;
+ private final RegionNormalizer regionNormalizer;
+ private final RegionNormalizerWorkQueue workQueue;
+ private final RateLimiter rateLimiter;
+
+ private final long[] skippedCount;
+ private long splitPlanCount;
+ private long mergePlanCount;
+
+ public RegionNormalizerWorker(
+final Configuration configuration,
+final MasterServices masterServices,
+final RegionNormalizer regionNormalizer,
+final RegionNormalizerWorkQueue workQueue
+ ) {
+this.masterServices = masterServices;
+this.regionNormalizer = regionNormalizer;
+this.workQueue = workQueue;
+this.skippedCount = new long[NormalizationPlan.PlanType.values().length];
+this.splitPlanCount = 0;
+this.mergePlanCount = 0;
+this.rateLimiter = loadRateLimiter(configuration);
+ }
+
+ private static RateLimiter loadRateLimiter(final Configuration
configuration) {
+long rateLimitBytes =
+ configuration.getLongBytes(RATE_LIMIT_BYTES_PER_SEC_KEY,
RATE_UNLIMITED_BYTES);
+long rateLimitMbs = rateLimitBytes / 1_000_000L;
+if (rateLimitMbs <= 0) {
+ LOG.warn("Configured value {}={} is <= 1MB. Falling back to default.",
+RATE_LIMIT_BYTES_PER_SEC_KEY, rateLimitBytes);
+ rateLimitBytes = RATE_UNLIMITED_BYTES;
+ rateLimitMbs = RATE_UNLIMITED_BYTES / 1_000_000L;
+}
+LOG.info("Normalizer rate limit set to {}",
+ rateLimitBytes == RATE_UNLIMITED_BYTES ? "unlimited" : rateLimitMbs + "
MB/sec");
+return RateLimiter.create(rateLimitMbs);
+ }
+
+ /**
+ * @see RegionNormalizerManager#planSkipped(NormalizationPlan.PlanType)
+ */
+ void planSkipped(NormalizationPlan.PlanType type) {
+synchronized (skippedCount) {
+ // updates come here via procedure threads, so synchronize access to
this counter.
+ skippedCount[type.ordinal()]++;
+}
+ }
+
+ /**
+ * @see RegionNormalizerManager#getSkippedCount(NormalizationPlan.PlanType)
+ */
+ long getSkippedCount(NormalizationPlan.PlanType type) {
+return skippedCount[type.ordinal()];
+ }
+
+ /**
+ * @see HMaster#getSplitPlanCount()
+ */
+ long getSplitPlanCount() {
+return splitPlanCount;
+ }
+
+ /**
+ * @see HMaster#getMergePlanCount()
+ */
+ long getMergePlanCount() {
+return mergePlanCount;
+ }
+
+ @Override
+ public void run() {
+while (true) {
+ if (Thread.interrupted()) {
+LOG.debug("interrupt detected. terminating.");
+break;
+ }
+
[jira] [Commented] (HBASE-25017) Attach a design doc to code base
[
https://issues.apache.org/jira/browse/HBASE-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204233#comment-17204233
]
Hudson commented on HBASE-25017:
Results for branch branch-2.2
[build #74 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/74/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/74//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/74//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/74//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/74//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Attach a design doc to code base
>
>
> Key: HBASE-25017
> URL: https://issues.apache.org/jira/browse/HBASE-25017
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
> Environment: Ata
>Reporter: Duo Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25107) Migrate flaky reporting jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204234#comment-17204234
]
Hudson commented on HBASE-25107:
Results for branch branch-2.2
[build #74 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/74/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/74//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/74//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/74//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/74//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Migrate flaky reporting jenkins job from Hadoop to hbase
>
>
> Key: HBASE-25107
> URL: https://issues.apache.org/jira/browse/HBASE-25107
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2469: HBASE-25112 Add new procedure metrics for master UI only
Apache-HBase commented on pull request #2469: URL: https://github.com/apache/hbase/pull/2469#issuecomment-700940945 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 30s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 1s | master passed | | +1 :green_heart: | compile | 2m 9s | master passed | | +1 :green_heart: | shadedjars | 8m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 31s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 6s | the patch passed | | +1 :green_heart: | compile | 2m 22s | the patch passed | | +1 :green_heart: | javac | 2m 22s | the patch passed | | +1 :green_heart: | shadedjars | 8m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 51s | hbase-hadoop-compat in the patch passed. | | +1 :green_heart: | unit | 2m 37s | hbase-procedure in the patch passed. | | -1 :x: | unit | 263m 20s | hbase-server in the patch failed. | | | | 306m 44s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2469 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8f66c6ebadc1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 9fc29c4cbf | | Default Java | 1.8.0_232 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/testReport/ | | Max. process+thread count | 3095 (vs. ulimit of 3) | | modules | C: hbase-hadoop-compat hbase-procedure hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2469/5/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-25103) Remove ZNodePaths.metaReplicaZNodes
[
https://issues.apache.org/jira/browse/HBASE-25103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204255#comment-17204255
]
Hudson commented on HBASE-25103:
Results for branch master
[build #73 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove ZNodePaths.metaReplicaZNodes
> ---
>
> Key: HBASE-25103
> URL: https://issues.apache.org/jira/browse/HBASE-25103
> Project: HBase
> Issue Type: Sub-task
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0
>
>
> We have already removed the dependency on this field from client side.
> Now if we want to remove the META_REPLICAS_NUM field, we must remove the
> usage of this field first.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25017) Attach a design doc to code base
[
https://issues.apache.org/jira/browse/HBASE-25017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204252#comment-17204252
]
Hudson commented on HBASE-25017:
Results for branch master
[build #73 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Attach a design doc to code base
>
>
> Key: HBASE-25017
> URL: https://issues.apache.org/jira/browse/HBASE-25017
> Project: HBase
> Issue Type: Sub-task
> Components: documentation
> Environment: Ata
>Reporter: Duo Zhang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24877) Add option to avoid aborting RS process upon uncaught exceptions happen on replication source
[
https://issues.apache.org/jira/browse/HBASE-24877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204256#comment-17204256
]
Hudson commented on HBASE-24877:
Results for branch master
[build #73 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add option to avoid aborting RS process upon uncaught exceptions happen on
> replication source
> -
>
> Key: HBASE-24877
> URL: https://issues.apache.org/jira/browse/HBASE-24877
> Project: HBase
> Issue Type: Improvement
> Components: Replication
>Affects Versions: 3.0.0-alpha-1
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
>
> Currently, we abort entire RS process if any uncaught exceptions happens on
> ReplicationSource initialization. This may be too extreme on certain
> deployments, where custom replication endpoint implementations may choose to
> do so when remote peers are unavailable, but source cluster shouldn't be
> brought down entirely. Similarly, source reader and shipper threads would
> cause RS to abort on any runtime exception occurrence while running.
> This patch adds configuration option (false by default, to keep the original
> behaviour), to avoid aborting entire RS processes under these conditions.
> Instead, if ReplicationSource initialization fails with a RuntimeException,
> it keeps retrying the source startup. In the case of readers/shippers runtime
> errors, it refreshes the replication source, terminating current source and
> its readers/shippers and creating new ones.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25107) Migrate flaky reporting jenkins job from Hadoop to hbase
[
https://issues.apache.org/jira/browse/HBASE-25107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204253#comment-17204253
]
Hudson commented on HBASE-25107:
Results for branch master
[build #73 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Migrate flaky reporting jenkins job from Hadoop to hbase
>
>
> Key: HBASE-25107
> URL: https://issues.apache.org/jira/browse/HBASE-25107
> Project: HBase
> Issue Type: Sub-task
> Components: scripts
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.3.3, 1.7.0, 2.4.0, 1.4.14, 2.2.7
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-24967) The table.jsp cost long time to load if the table include closed regions
[
https://issues.apache.org/jira/browse/HBASE-24967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204254#comment-17204254
]
Hudson commented on HBASE-24967:
Results for branch master
[build #73 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/73/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The table.jsp cost long time to load if the table include closed regions
>
>
> Key: HBASE-24967
> URL: https://issues.apache.org/jira/browse/HBASE-24967
> Project: HBase
> Issue Type: Improvement
> Components: UI
>Reporter: Zheng Wang
>Assignee: Zheng Wang
>Priority: Major
>
> The reason is that, in order to get the compaction state of table, we need to
> iterate all regions and call getRegionInfo, the default timeout for rpc is
> 5000ms, so if there exists closed reigons, this line will be blocked for long
> time.
> {code:java}
> CompactionState compactionState =
> admin.getCompactionState(table.getName()).get();{code}
> The solution in this patch is to add compaction state to RegionMetrics, so
> when getting the compaction state, we could use the in memory state instead
> of doing rpc.
> Affect branch-2+, in branch-1 it will display exception stack info
> immediately.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204279#comment-17204279
]
Sean Busbey commented on HBASE-23887:
-
I don't want to keep pushing work on you, but if this change is adopted as an
opt-in feature for a kind of cache then having that google doc as a markdown or
pdf file in the {{dev-support/design-docs}} area will help a lot when folks
need to reason about if using this feature is worthwhile.
I think this an interesting approach to skewed key reads. I would expect it to
help with zipfian workloads (like YCSB is supposed to do) because the "should
we bother to cache a new block" is essentially trying to approximate the
likelihood that a new read is from the tail of the distribution rather than the
set of frequent items.
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png,
> evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100 && buf.getBlockType().isData())
> if (cacheKey.getOffset() % 100 >= cacheDataBlockPercent)
> return;
> ...
> // the same code as usual
> }
> {code}
>
> Other parameters help to control when this logic will be enabled. It means it
> will work only while heavy reading going on.
> hbase.lru.cache.heavy.eviction.count.limit - set how many times have to run
> eviction process that start to avoid of putting data to BlockCache
> hbase.lru.cache.heavy.eviction.bytes.size.limit - set how many bytes have to
> evicted each time that start to avoid of putting data to BlockCache
> By default: if 10 times (100 secunds) evicted more than 10 MB (each time)
> then we start to skip 50% of data blocks.
> When heavy evitions process end then new logic off and will put into
> BlockCache all blocks again.
>
>
[jira] [Comment Edited] (HBASE-23887) BlockCache performance improve by reduce eviction rate
[
https://issues.apache.org/jira/browse/HBASE-23887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204279#comment-17204279
]
Sean Busbey edited comment on HBASE-23887 at 9/29/20, 8:44 PM:
---
I don't want to keep pushing work on you, but if this change is adopted as an
opt-in feature for a kind of cache then having that google doc as a markdown or
pdf file in the {{dev-support/design-docs}} area will help a lot when folks
need to reason about if using this feature is worthwhile.
I think this an interesting approach to skewed key reads. I would expect it to
help with zipfian workloads (like YCSB is supposed to do) because the "should
we bother to cache a new block" is essentially trying to approximate the
likelihood that a new read is from the tail of the distribution rather than the
set of frequent items. if something is from the tail then it's not worth
thrashing trying to chase a cache hit that is very unlikely to come later.
was (Author: busbey):
I don't want to keep pushing work on you, but if this change is adopted as an
opt-in feature for a kind of cache then having that google doc as a markdown or
pdf file in the {{dev-support/design-docs}} area will help a lot when folks
need to reason about if using this feature is worthwhile.
I think this an interesting approach to skewed key reads. I would expect it to
help with zipfian workloads (like YCSB is supposed to do) because the "should
we bother to cache a new block" is essentially trying to approximate the
likelihood that a new read is from the tail of the distribution rather than the
set of frequent items.
> BlockCache performance improve by reduce eviction rate
> --
>
> Key: HBASE-23887
> URL: https://issues.apache.org/jira/browse/HBASE-23887
> Project: HBase
> Issue Type: Improvement
> Components: BlockCache, Performance
>Reporter: Danil Lipovoy
>Assignee: Danil Lipovoy
>Priority: Minor
> Attachments: 1582787018434_rs_metrics.jpg,
> 1582801838065_rs_metrics_new.png, BC_LongRun.png,
> BlockCacheEvictionProcess.gif, BlockCacheEvictionProcess.gif, cmp.png,
> evict_BC100_vs_BC23.png, eviction_100p.png, eviction_100p.png,
> eviction_100p.png, gc_100p.png, graph.png, image-2020-06-07-08-11-11-929.png,
> image-2020-06-07-08-19-00-922.png, image-2020-06-07-12-07-24-903.png,
> image-2020-06-07-12-07-30-307.png, image-2020-06-08-17-38-45-159.png,
> image-2020-06-08-17-38-52-579.png, image-2020-06-08-18-35-48-366.png,
> image-2020-06-14-20-51-11-905.png, image-2020-06-22-05-57-45-578.png,
> image-2020-09-23-09-48-59-714.png, image-2020-09-23-10-06-11-189.png,
> ratio.png, ratio2.png, read_requests_100pBC_vs_23pBC.png, requests_100p.png,
> requests_100p.png, requests_new2_100p.png, requests_new_100p.png, scan.png,
> scan_and_gets.png, scan_and_gets2.png, wave.png, ycsb_logs.zip
>
>
> Hi!
> I first time here, correct me please if something wrong.
> All latest information is here:
> [https://docs.google.com/document/d/1X8jVnK_3lp9ibpX6lnISf_He-6xrHZL0jQQ7hoTV0-g/edit?usp=sharing]
> I want propose how to improve performance when data in HFiles much more than
> BlockChache (usual story in BigData). The idea - caching only part of DATA
> blocks. It is good becouse LruBlockCache starts to work and save huge amount
> of GC.
> Sometimes we have more data than can fit into BlockCache and it is cause a
> high rate of evictions. In this case we can skip cache a block N and insted
> cache the N+1th block. Anyway we would evict N block quite soon and that why
> that skipping good for performance.
> ---
> Some information below isn't actual
> ---
>
>
> Example:
> Imagine we have little cache, just can fit only 1 block and we are trying to
> read 3 blocks with offsets:
> 124
> 198
> 223
> Current way - we put the block 124, then put 198, evict 124, put 223, evict
> 198. A lot of work (5 actions).
> With the feature - last few digits evenly distributed from 0 to 99. When we
> divide by modulus we got:
> 124 -> 24
> 198 -> 98
> 223 -> 23
> It helps to sort them. Some part, for example below 50 (if we set
> *hbase.lru.cache.data.block.percent* = 50) go into the cache. And skip
> others. It means we will not try to handle the block 198 and save CPU for
> other job. In the result - we put block 124, then put 223, evict 124 (3
> actions).
> See the picture in attachment with test below. Requests per second is higher,
> GC is lower.
>
> The key point of the code:
> Added the parameter: *hbase.lru.cache.data.block.percent* which by default =
> 100
>
> But if we set it 1-99, then will work the next logic:
>
>
> {code:java}
> public void cacheBlock(BlockCacheKey cacheKey, Cacheable buf, boolean
> inMemory) {
> if (cacheDataBlockPercent != 100
[GitHub] [hbase] busbey commented on a change in pull request #2453: HBASE-24813 ReplicationSource should clear buffer usage on Replicatio…
busbey commented on a change in pull request #2453:
URL: https://github.com/apache/hbase/pull/2453#discussion_r497049373
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceShipper.java
##
@@ -331,4 +333,53 @@ public boolean sleepForRetries(String msg, int
sleepMultiplier) {
}
return sleepMultiplier < maxRetriesMultiplier;
}
+
+ /**
+ * Attempts to properly update
ReplicationSourceManager.totalBufferUser,
+ * in case there were unprocessed entries batched by the reader to the
shipper,
+ * but the shipper didn't manage to ship those because the replication
source is being terminated.
+ * In that case, it iterates through the batched entries and decrease the
pending
+ * entries size from ReplicationSourceManager.totalBufferUser
+ *
+ * NOTES
+ * 1) This method should only be called upon replication source termination.
+ * It blocks waiting for both shipper and reader threads termination,
+ * to make sure no race conditions
+ * when updating ReplicationSourceManager.totalBufferUser.
+ *
+ * 2) It does not attempt to terminate reader and shipper threads.
Those must
+ * have been triggered interruption/termination prior to calling this method.
+ */
+ void clearWALEntryBatch() {
+long timeout = System.currentTimeMillis() + this.shipEditsTimeout;
Review comment:
this should be using the environment to get the time I think? or is
there not one accessible from here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-23287) WALs not aged off of HDFS because LogCleaner is not added to choreService
[
https://issues.apache.org/jira/browse/HBASE-23287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204306#comment-17204306
]
Maziar Mirzazad commented on HBASE-23287:
-
thanks [~vjasani]
> WALs not aged off of HDFS because LogCleaner is not added to choreService
> -
>
> Key: HBASE-23287
> URL: https://issues.apache.org/jira/browse/HBASE-23287
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 1.5.0, 1.6.0, 1.4.11
>Reporter: Baiqiang Zhao
>Assignee: Baiqiang Zhao
>Priority: Blocker
> Fix For: 1.6.0, 1.4.12
>
> Attachments: HBASE-23287.branch-1.0001.patch
>
>
> We found /hbase/oldWALs keeps growing, and we never use replication feature.
> Then we found OldWALsCleaner thread is always in WAITTING state :
> {code:java}
> Thread 176 (OldWALsCleaner-1): State: WAITING Blocked count: 0 Waited count:
> 1 Waiting on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2ef05d36
> Stack: sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.deleteFile(LogCleaner.java:185)
>
> org.apache.hadoop.hbase.master.cleaner.LogCleaner.access$000(LogCleaner.java:48)
> org.apache.hadoop.hbase.master.cleaner.LogCleaner$1.run(LogCleaner.java:161)
> java.lang.Thread.run(Thread.java:745)
> {code}
> Finally, we found that LogCleaner is not added to choreService, so LogCleaner
> doesn't run periodically.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25118) Extend Cell Tags to Delete object.
Rushabh Shah created HBASE-25118: Summary: Extend Cell Tags to Delete object. Key: HBASE-25118 URL: https://issues.apache.org/jira/browse/HBASE-25118 Project: HBase Issue Type: Improvement Reporter: Rushabh Shah Assignee: Rushabh Shah We want to track the source of mutations (especially Deletes) via Phoenix. We have multiple use cases which does the deletes namely: customer deleting the data, internal process like GDPR compliance, Phoenix TTL MR jobs. For every mutations we want to track the source of operation which initiated the deletes. At my day job, we have custom Backup/Restore tool. For example: During GDPR compliance cleanup (lets say at time t0), we mistakenly deleted some customer data and it were possible that customer also deleted some data from their side (at time t1). To recover mistakenly deleted data, we restore from the backup at time (t0 - 1). By doing this, we also recovered the data that customer intentionally deleted. We need a way for Restore tool to selectively recover data. We want to leverage Cell Tag feature for Delete mutations to store these metadata. Currently Delete object doesn't support Tag feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-25118) Extend Cell Tags to Delete object.
[ https://issues.apache.org/jira/browse/HBASE-25118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rushabh Shah updated HBASE-25118: - Fix Version/s: 2.4.0 1.7.0 3.0.0-alpha-1 > Extend Cell Tags to Delete object. > -- > > Key: HBASE-25118 > URL: https://issues.apache.org/jira/browse/HBASE-25118 > Project: HBase > Issue Type: Improvement >Reporter: Rushabh Shah >Assignee: Rushabh Shah >Priority: Major > Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0 > > > We want to track the source of mutations (especially Deletes) via Phoenix. We > have multiple use cases which does the deletes namely: customer deleting the > data, internal process like GDPR compliance, Phoenix TTL MR jobs. For every > mutations we want to track the source of operation which initiated the > deletes. > At my day job, we have custom Backup/Restore tool. > For example: During GDPR compliance cleanup (lets say at time t0), we > mistakenly deleted some customer data and it were possible that customer also > deleted some data from their side (at time t1). To recover mistakenly deleted > data, we restore from the backup at time (t0 - 1). By doing this, we also > recovered the data that customer intentionally deleted. > We need a way for Restore tool to selectively recover data. > We want to leverage Cell Tag feature for Delete mutations to store these > metadata. Currently Delete object doesn't support Tag feature. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2451: HBASE-25055 Add ReplicationSource for meta WALs; add enable/disable w…
Apache-HBase commented on pull request #2451: URL: https://github.com/apache/hbase/pull/2451#issuecomment-701056437 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ HBASE-18070 Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 10s | HBASE-18070 passed | | +1 :green_heart: | checkstyle | 1m 28s | HBASE-18070 passed | | +1 :green_heart: | spotbugs | 2m 54s | HBASE-18070 passed | | -0 :warning: | patch | 3m 4s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 9s | the patch passed | | +1 :green_heart: | checkstyle | 1m 45s | hbase-server: The patch generated 0 new + 42 unchanged - 11 fixed = 42 total (was 53) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 12m 10s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 2m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 43m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2451 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle | | uname | Linux 02c9af434ec1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | HBASE-18070 / 03e8e5b517 | | Max. process+thread count | 94 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2451/3/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #2468: HBASE-25109 Add MR Counters to WALPlayer; currently hard to tell if i…
saintstack commented on a change in pull request #2468:
URL: https://github.com/apache/hbase/pull/2468#discussion_r497163970
##
File path:
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/WALPlayer.java
##
@@ -225,8 +245,10 @@ public void map(WALKey key, WALEdit value, Context context)
}
}
if (CellUtil.isDelete(cell)) {
+context.getCounter(Counter.CELLS_WRITTEN).increment(1);
del.add(cell);
} else {
+context.getCounter(Counter.CELLS_WRITTEN).increment(1);
put.add(cell);
Review comment:
Makes sense.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-24764) Add support of adding base peer configs via hbase-site.xml for all replication peers.
[ https://issues.apache.org/jira/browse/HBASE-24764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Jain updated HBASE-24764: --- Release Note: Adds a new configuration parameter "hbase.replication.peer.base.config" which accepts a semi-colon separated key=CSV pairs (example: k1=v1;k2=v2_1,v3...). When this configuration is set on the server side, these kv pairs are added to every peer configuration if not already set. Peer specific configuration overrides have precedence over the above default configuration. This is useful in cases when some configuration has to be set for all the peers by default and one does not want to add to every peer definition. was: Adds a new configuration parameter "hbase.replication.peer.default.config" which accepts a semi-colon separated key=CSV pairs (example: k1=v1;k2=v2_1,v3...). When this configuration is set on the server side, these kv pairs are added to every peer configuration if not already set. Peer specific configuration overrides have precedence over the above default configuration. This is useful in cases when some configuration has to be set for all the peers by default and one does not want to add to every peer definition. > Add support of adding base peer configs via hbase-site.xml for all > replication peers. > - > > Key: HBASE-24764 > URL: https://issues.apache.org/jira/browse/HBASE-24764 > Project: HBase > Issue Type: Improvement > Components: Replication >Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.4.0 >Reporter: Ankit Jain >Assignee: Ankit Jain >Priority: Minor > Fix For: 3.0.0-alpha-1, 1.7.0, 2.4.0 > > Attachments: HBASE-24764-master.patch > > > Today, if a user needs to apply some common base peer configs to all the > replication peers on a given cluster, the only way is to execute > update_peer_config via CLI which requires manual intervention and can be > tedious in case of large deployment fleet. > As part of this JIRA, we plan to add the support to have base replication > peer configs as part of hbase-site.xml like > hbase.replication.peer.base.config="k1=v1;k2=v2.." which can be easily > updated and applied as part of a rolling restart. Example below: > > hbase.replication.peer.base.configs > > hbase.replication.source.custom.walentryfilters=x,y,z;hbase.rpc.protection=abc;hbase.xxx.custom_property=123 > > This will be empty by default, but one can override to have base configs in > place. > The final peer configuration would be a merge of this newly added base config > + whatever users override during the peer creation/update (if any). > Related Jira: https://issues.apache.org/jira/browse/HBASE-17543. HBASE-17543 > added the support to add the WALEntryFilters to default endpoint via peer > configuration. By this new Jira we are extending the support to update peer > configs via hbase-site.xml. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25119) Fix property name added as part of HBASE-24764 in branch-1
Ankit Jain created HBASE-25119: -- Summary: Fix property name added as part of HBASE-24764 in branch-1 Key: HBASE-25119 URL: https://issues.apache.org/jira/browse/HBASE-25119 Project: HBase Issue Type: Improvement Components: Replication Reporter: Ankit Jain Assignee: Ankit Jain A new property was added as part of this https://issues.apache.org/jira/browse/HBASE-24764. Master branch has the name as "hbase.replication.peer.base.config" while branch-1 has the name as "hbase.replication.peer.default.config" As part of this Jira fix the name in branch-1 to be consistent. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2468: HBASE-25109 Add MR Counters to WALPlayer; currently hard to tell if i…
Apache-HBase commented on pull request #2468: URL: https://github.com/apache/hbase/pull/2468#issuecomment-701102135 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 59s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +0 :ok: | mvndep | 0m 16s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 30s | branch-2 passed | | +1 :green_heart: | checkstyle | 2m 1s | branch-2 passed | | +0 :ok: | refguide | 7m 4s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | spotbugs | 8m 45s | branch-2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 13s | the patch passed | | -0 :warning: | checkstyle | 2m 3s | root: The patch generated 4 new + 18 unchanged - 4 fixed = 22 total (was 22) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 6m 57s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | | +1 :green_heart: | hadoopcheck | 11m 39s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1. | | +1 :green_heart: | spotbugs | 8m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 29s | The patch does not generate ASF License warnings. | | | | 66m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/4/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2468 | | Optional Tests | dupname asflicense spotbugs hadoopcheck hbaseanti checkstyle refguide | | uname | Linux 9735ea8968ae 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / f973a8087f | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/4/artifact/yetus-general-check/output/branch-site/book.html | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/4/artifact/yetus-general-check/output/diff-checkstyle-root.txt | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/4/artifact/yetus-general-check/output/patch-site/book.html | | Max. process+thread count | 137 (vs. ulimit of 12500) | | modules | C: hbase-mapreduce . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2468/4/console | | versions | git=2.17.1 maven=(cecedd343002696d0abb50b32b541b8a6ba2883f) spotbugs=3.1.12 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] ankitjain64 opened a new pull request #2471: HBASE-25119: Fix property name added as part of HBASE-24764 in branch-1
ankitjain64 opened a new pull request #2471: URL: https://github.com/apache/hbase/pull/2471 A new property was added as part of this https://issues.apache.org/jira/browse/HBASE-24764. Master branch has the name as "hbase.replication.peer.base.config" while branch-1 has the name as "hbase.replication.peer.default.config" As part of this Jira fix the name in branch-1 to be consistent. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Work started] (HBASE-25119) Fix property name added as part of HBASE-24764 in branch-1
[ https://issues.apache.org/jira/browse/HBASE-25119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-25119 started by Ankit Jain. -- > Fix property name added as part of HBASE-24764 in branch-1 > -- > > Key: HBASE-25119 > URL: https://issues.apache.org/jira/browse/HBASE-25119 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Ankit Jain >Assignee: Ankit Jain >Priority: Minor > > A new property was added as part of this > https://issues.apache.org/jira/browse/HBASE-24764. > Master branch has the name as "hbase.replication.peer.base.config" > while branch-1 has the name as "hbase.replication.peer.default.config" > > As part of this Jira fix the name in branch-1 to be consistent. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on pull request #2471: HBASE-25119: Fix property name added as part of HBASE-24764 in branch-1
Apache-HBase commented on pull request #2471: URL: https://github.com/apache/hbase/pull/2471#issuecomment-701115501 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 21s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 52s | branch-1 passed | | +1 :green_heart: | compile | 0m 25s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | compile | 0m 32s | branch-1 passed with JDK v1.7.0_272 | | +1 :green_heart: | checkstyle | 1m 0s | branch-1 passed | | +1 :green_heart: | shadedjars | 4m 30s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 43s | branch-1 passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 36s | branch-1 passed with JDK v1.7.0_272 | | +0 :ok: | spotbugs | 2m 23s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 18s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 1s | the patch passed | | +1 :green_heart: | compile | 0m 25s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javac | 0m 25s | the patch passed | | +1 :green_heart: | compile | 0m 32s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | javac | 0m 32s | the patch passed | | +1 :green_heart: | checkstyle | 0m 50s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 15s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 6m 46s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 26s | the patch passed with JDK v1.8.0_262 | | +1 :green_heart: | javadoc | 0m 34s | the patch passed with JDK v1.7.0_272 | | +1 :green_heart: | findbugs | 2m 21s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 16s | hbase-client in the patch passed. | | +1 :green_heart: | asflicense | 0m 24s | The patch does not generate ASF License warnings. | | | | 52m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.13 Server=19.03.13 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2471/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2471 | | JIRA Issue | HBASE-25119 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c029d87a8a07 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-2471/out/precommit/personality/provided.sh | | git revision | branch-1 / e719a5b | | Default Java | 1.7.0_272 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_262 /usr/lib/jvm/zulu-7-amd64:1.7.0_272 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2471/1/testReport/ | | Max. process+thread count | 148 (vs. ulimit of 1) | | modules | C: hbase-client U: hbase-client | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2471/1/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
