[GitHub] [hbase] binlijin merged pull request #202: HBASE-22280 Separate read/write handler for priority request(especial…
binlijin merged pull request #202: HBASE-22280 Separate read/write handler for priority request(especial… URL: https://github.com/apache/hbase/pull/202 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-22280) Separate read/write handler for priority request(especially for meta).
[ https://issues.apache.org/jira/browse/HBASE-22280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijin Bin updated HBASE-22280: -- Fix Version/s: 2.2.3 2.3.0 3.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Separate read/write handler for priority request(especially for meta). > -- > > Key: HBASE-22280 > URL: https://issues.apache.org/jira/browse/HBASE-22280 > Project: HBase > Issue Type: New Feature > Components: Scheduler >Reporter: Lijin Bin >Assignee: Lijin Bin >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.3 > > Attachments: HBASE-22280.patch > > > Client may give too many read pressure on meta, so blocking master write meta > for region open. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22280) Separate read/write handler for priority request(especially for meta).
[ https://issues.apache.org/jira/browse/HBASE-22280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991233#comment-16991233 ] Lijin Bin commented on HBASE-22280: --- Push to master/branch-2/branch-2.2 [~liyu] Thanks for the review. > Separate read/write handler for priority request(especially for meta). > -- > > Key: HBASE-22280 > URL: https://issues.apache.org/jira/browse/HBASE-22280 > Project: HBase > Issue Type: New Feature > Components: Scheduler >Reporter: Lijin Bin >Assignee: Lijin Bin >Priority: Major > Fix For: 3.0.0, 2.3.0, 2.2.3 > > Attachments: HBASE-22280.patch > > > Client may give too many read pressure on meta, so blocking master write meta > for region open. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23376) NPE happens while replica region is moving
[
https://issues.apache.org/jira/browse/HBASE-23376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991234#comment-16991234
]
HBase QA commented on HBASE-23376:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
7s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
58s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
41s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
40s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
1s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 8s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}226m 2s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}286m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.quotas.TestClusterScopeQuotaThrottle |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/1066/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-23376 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12988287/HBASE-23376.master.v01.dummy.patch
|
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 6266c612b08c 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1
05:24:09 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 978546b2f2 |
|
[GitHub] [hbase] bea0113 commented on a change in pull request #880: HBASE-18382 add transport type info into Thrift UI
bea0113 commented on a change in pull request #880: HBASE-18382 add transport
type info into Thrift UI
URL: https://github.com/apache/hbase/pull/880#discussion_r355314132
##
File path: hbase-thrift/src/main/resources/hbase-webapps/thrift/thrift.jsp
##
@@ -31,9 +31,11 @@ String serverType =
(String)getServletContext().getAttribute("hbase.thrift.serve
long startcode = conf.getLong("startcode", System.currentTimeMillis());
String listenPort = conf.get("hbase.regionserver.thrift.port", "9090");
ImplType implType = ImplType.getServerImpl(conf);
-String framed = implType.isAlwaysFramed()
-? "true" : conf.get("hbase.regionserver.thrift.framed", "false");
-String compact = conf.get("hbase.regionserver.thrift.compact", "false");
+
+String transport = implType.isAlwaysFramed()
||conf.getBoolean("hbase.regionserver.thrift.framed", false) ? "Framed" :
"Standard";
Review comment:
I corrected these, I hope it looks good now :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #880: HBASE-18382 add transport type info into Thrift UI
Apache-HBase commented on issue #880: HBASE-18382 add transport type info into Thrift UI URL: https://github.com/apache/hbase/pull/880#issuecomment-563129715 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 3m 18s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 56s | master passed | | +1 :green_heart: | javadoc | 0m 33s | master passed | | -0 :warning: | patch | 6m 41s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 35s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | javadoc | 0m 33s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 4m 41s | hbase-thrift in the patch passed. | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 22m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-880/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/880 | | Optional Tests | dupname asflicense javac javadoc unit | | uname | Linux 18aaa459d957 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-880/out/precommit/personality/provided.sh | | git revision | master / dff7fffe85 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-880/5/testReport/ | | Max. process+thread count | 1808 (vs. ulimit of 1) | | modules | C: hbase-thrift U: hbase-thrift | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-880/5/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23376) NPE happens while replica region is moving
[
https://issues.apache.org/jira/browse/HBASE-23376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991395#comment-16991395
]
Duo Zhang commented on HBASE-23376:
---
Emmm...
What I mean, is that based on your original patch, adding a dummy modification
in the hbase-server module, not only a dummy modification in the hbase-server
module...
> NPE happens while replica region is moving
> --
>
> Key: HBASE-23376
> URL: https://issues.apache.org/jira/browse/HBASE-23376
> Project: HBase
> Issue Type: Bug
> Components: read replicas
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Minor
> Attachments: HBASE-23376.branch-2.001.patch,
> HBASE-23376.master.v01.dummy.patch
>
>
> The following code is from AsyncNonMetaRegionLocator#addToCache
> {code:java}
> private RegionLocations addToCache(TableCache tableCache, RegionLocations
> locs) {
> LOG.trace("Try adding {} to cache", locs);
> byte[] startKey = locs.getDefaultRegionLocation().getRegion().getStartKey();
> ...
> }{code}
> we will get a NPE if the locs is without the default region.
>
> The following code is from
> AsyncRegionLocatorHelper#updateCachedLocationOnError
> {code:java}
> ...
> if (cause instanceof RegionMovedException) {
> RegionMovedException rme = (RegionMovedException) cause;
> HRegionLocation newLoc =
> new HRegionLocation(loc.getRegion(), rme.getServerName(),
> rme.getLocationSeqNum());
> LOG.debug("Try updating {} with the new location {} constructed by {}",
> loc, newLoc,
> rme.toString());
> addToCache.accept(newLoc);
> ...{code}
> If the replica region is moving, we will get a RegionMovedException and add
> the HRegionLocation of replica region to cache. And finally NPE happens.
>
> {code:java}
> java.lang.NullPointerExceptionjava.lang.NullPointerException at
> org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.addToCache(AsyncNonMetaRegionLocator.java:240)
> at
> org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.addLocationToCache(AsyncNonMetaRegionLocator.java:596)
> at
> org.apache.hadoop.hbase.client.AsyncRegionLocatorHelper.updateCachedLocationOnError(AsyncRegionLocatorHelper.java:80)
> at
> org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.updateCachedLocationOnError(AsyncNonMetaRegionLocator.java:610)
> at
> org.apache.hadoop.hbase.client.AsyncRegionLocator.updateCachedLocationOnError(AsyncRegionLocator.java:153)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] wchevreuil merged pull request #880: HBASE-18382 add transport type info into Thrift UI
wchevreuil merged pull request #880: HBASE-18382 add transport type info into Thrift UI URL: https://github.com/apache/hbase/pull/880 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-18382) [Thrift] Add transport type info to info server
[
https://issues.apache.org/jira/browse/HBASE-18382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991407#comment-16991407
]
Wellington Chevreuil commented on HBASE-18382:
--
Merged into master. Thanks for the contribution, [~bea0113]!
> [Thrift] Add transport type info to info server
>
>
> Key: HBASE-18382
> URL: https://issues.apache.org/jira/browse/HBASE-18382
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Lars George
>Priority: Minor
> Labels: beginner
>
> It would be really helpful to know if the Thrift server was started using the
> HTTP or binary transport. Any additional info, like QOP settings for SASL
> etc. would be great too. Right now the UI is very limited and shows
> {{true/false}} for, for example, {{Compact Transport}}. It'd suggest to
> change this to show something more useful like this:
> {noformat}
> Thrift Impl Type: non-blocking
> Protocol: Binary
> Transport: Framed
> QOP: Authentication & Confidential
> {noformat}
> or
> {noformat}
> Protocol: Binary + HTTP
> Transport: Standard
> QOP: none
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-18382) [Thrift] Add transport type info to info server
[
https://issues.apache.org/jira/browse/HBASE-18382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil resolved HBASE-18382.
--
Resolution: Fixed
> [Thrift] Add transport type info to info server
>
>
> Key: HBASE-18382
> URL: https://issues.apache.org/jira/browse/HBASE-18382
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Lars George
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0
>
>
> It would be really helpful to know if the Thrift server was started using the
> HTTP or binary transport. Any additional info, like QOP settings for SASL
> etc. would be great too. Right now the UI is very limited and shows
> {{true/false}} for, for example, {{Compact Transport}}. It'd suggest to
> change this to show something more useful like this:
> {noformat}
> Thrift Impl Type: non-blocking
> Protocol: Binary
> Transport: Framed
> QOP: Authentication & Confidential
> {noformat}
> or
> {noformat}
> Protocol: Binary + HTTP
> Transport: Standard
> QOP: none
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-18382) [Thrift] Add transport type info to info server
[
https://issues.apache.org/jira/browse/HBASE-18382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil updated HBASE-18382:
-
Fix Version/s: 3.0.0
> [Thrift] Add transport type info to info server
>
>
> Key: HBASE-18382
> URL: https://issues.apache.org/jira/browse/HBASE-18382
> Project: HBase
> Issue Type: Improvement
> Components: Thrift
>Reporter: Lars George
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0
>
>
> It would be really helpful to know if the Thrift server was started using the
> HTTP or binary transport. Any additional info, like QOP settings for SASL
> etc. would be great too. Right now the UI is very limited and shows
> {{true/false}} for, for example, {{Compact Transport}}. It'd suggest to
> change this to show something more useful like this:
> {noformat}
> Thrift Impl Type: non-blocking
> Protocol: Binary
> Transport: Framed
> QOP: Authentication & Confidential
> {noformat}
> or
> {noformat}
> Protocol: Binary + HTTP
> Transport: Standard
> QOP: none
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23376) NPE happens while replica region is moving
[
https://issues.apache.org/jira/browse/HBASE-23376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sun Xin updated HBASE-23376:
Attachment: HBASE-23376.master.v02.dummy.patch
> NPE happens while replica region is moving
> --
>
> Key: HBASE-23376
> URL: https://issues.apache.org/jira/browse/HBASE-23376
> Project: HBase
> Issue Type: Bug
> Components: read replicas
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Minor
> Attachments: HBASE-23376.branch-2.001.patch,
> HBASE-23376.master.v01.dummy.patch, HBASE-23376.master.v02.dummy.patch
>
>
> The following code is from AsyncNonMetaRegionLocator#addToCache
> {code:java}
> private RegionLocations addToCache(TableCache tableCache, RegionLocations
> locs) {
> LOG.trace("Try adding {} to cache", locs);
> byte[] startKey = locs.getDefaultRegionLocation().getRegion().getStartKey();
> ...
> }{code}
> we will get a NPE if the locs is without the default region.
>
> The following code is from
> AsyncRegionLocatorHelper#updateCachedLocationOnError
> {code:java}
> ...
> if (cause instanceof RegionMovedException) {
> RegionMovedException rme = (RegionMovedException) cause;
> HRegionLocation newLoc =
> new HRegionLocation(loc.getRegion(), rme.getServerName(),
> rme.getLocationSeqNum());
> LOG.debug("Try updating {} with the new location {} constructed by {}",
> loc, newLoc,
> rme.toString());
> addToCache.accept(newLoc);
> ...{code}
> If the replica region is moving, we will get a RegionMovedException and add
> the HRegionLocation of replica region to cache. And finally NPE happens.
>
> {code:java}
> java.lang.NullPointerExceptionjava.lang.NullPointerException at
> org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.addToCache(AsyncNonMetaRegionLocator.java:240)
> at
> org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.addLocationToCache(AsyncNonMetaRegionLocator.java:596)
> at
> org.apache.hadoop.hbase.client.AsyncRegionLocatorHelper.updateCachedLocationOnError(AsyncRegionLocatorHelper.java:80)
> at
> org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.updateCachedLocationOnError(AsyncNonMetaRegionLocator.java:610)
> at
> org.apache.hadoop.hbase.client.AsyncRegionLocator.updateCachedLocationOnError(AsyncRegionLocator.java:153)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23376) NPE happens while replica region is moving
[
https://issues.apache.org/jira/browse/HBASE-23376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991424#comment-16991424
]
Sun Xin commented on HBASE-23376:
-
Thanks [~zhangduo] for reminding me, I have uploaded the patch which includes
original modification and a dummy modification.
> NPE happens while replica region is moving
> --
>
> Key: HBASE-23376
> URL: https://issues.apache.org/jira/browse/HBASE-23376
> Project: HBase
> Issue Type: Bug
> Components: read replicas
>Reporter: Sun Xin
>Assignee: Sun Xin
>Priority: Minor
> Attachments: HBASE-23376.branch-2.001.patch,
> HBASE-23376.master.v01.dummy.patch, HBASE-23376.master.v02.dummy.patch
>
>
> The following code is from AsyncNonMetaRegionLocator#addToCache
> {code:java}
> private RegionLocations addToCache(TableCache tableCache, RegionLocations
> locs) {
> LOG.trace("Try adding {} to cache", locs);
> byte[] startKey = locs.getDefaultRegionLocation().getRegion().getStartKey();
> ...
> }{code}
> we will get a NPE if the locs is without the default region.
>
> The following code is from
> AsyncRegionLocatorHelper#updateCachedLocationOnError
> {code:java}
> ...
> if (cause instanceof RegionMovedException) {
> RegionMovedException rme = (RegionMovedException) cause;
> HRegionLocation newLoc =
> new HRegionLocation(loc.getRegion(), rme.getServerName(),
> rme.getLocationSeqNum());
> LOG.debug("Try updating {} with the new location {} constructed by {}",
> loc, newLoc,
> rme.toString());
> addToCache.accept(newLoc);
> ...{code}
> If the replica region is moving, we will get a RegionMovedException and add
> the HRegionLocation of replica region to cache. And finally NPE happens.
>
> {code:java}
> java.lang.NullPointerExceptionjava.lang.NullPointerException at
> org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.addToCache(AsyncNonMetaRegionLocator.java:240)
> at
> org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.addLocationToCache(AsyncNonMetaRegionLocator.java:596)
> at
> org.apache.hadoop.hbase.client.AsyncRegionLocatorHelper.updateCachedLocationOnError(AsyncRegionLocatorHelper.java:80)
> at
> org.apache.hadoop.hbase.client.AsyncNonMetaRegionLocator.updateCachedLocationOnError(AsyncNonMetaRegionLocator.java:610)
> at
> org.apache.hadoop.hbase.client.AsyncRegionLocator.updateCachedLocationOnError(AsyncRegionLocator.java:153)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #820: HBASE-23286 Improve MTTR: Split WAL to HFile
Apache-HBase commented on issue #820: HBASE-23286 Improve MTTR: Split WAL to HFile URL: https://github.com/apache/hbase/pull/820#issuecomment-563188160 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 40s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 15s | master passed | | +1 :green_heart: | compile | 1m 19s | master passed | | +1 :green_heart: | checkstyle | 1m 52s | master passed | | +1 :green_heart: | shadedjars | 4m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | | +0 :ok: | spotbugs | 4m 12s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 59s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 59s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | -1 :x: | checkstyle | 1m 25s | hbase-server: The patch generated 4 new + 324 unchanged - 0 fixed = 328 total (was 324) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 33s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 2s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | | +1 :green_heart: | findbugs | 5m 10s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 7s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 153m 10s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 1m 5s | The patch does not generate ASF License warnings. | | | | 219m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/820 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 49e3a00e17cf 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-820/out/precommit/personality/provided.sh | | git revision | master / 978546b2f2 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/4/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/4/testReport/ | | Max. process+thread count | 4123 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #908: HBASE-23377 Balancer should skip disabled tables's regions
Apache-HBase commented on issue #908: HBASE-23377 Balancer should skip disabled tables's regions URL: https://github.com/apache/hbase/pull/908#issuecomment-563196985 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 6m 9s | master passed | | +1 :green_heart: | compile | 0m 58s | master passed | | +1 :green_heart: | checkstyle | 1m 33s | master passed | | +1 :green_heart: | shadedjars | 5m 1s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | | +0 :ok: | spotbugs | 4m 33s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 32s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 34s | the patch passed | | +1 :green_heart: | compile | 0m 59s | the patch passed | | +1 :green_heart: | javac | 0m 59s | the patch passed | | +1 :green_heart: | checkstyle | 1m 31s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 5m 5s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 17m 29s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 34s | the patch passed | | +1 :green_heart: | findbugs | 4m 39s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 254m 16s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 317m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-908/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/908 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e3121d4ba5be 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-908/out/precommit/personality/provided.sh | | git revision | master / 978546b2f2 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-908/3/testReport/ | | Max. process+thread count | 5060 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-908/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #919: HBASE-23066 Allow cache on write during compactions when prefetching is
Apache-HBase commented on issue #919: HBASE-23066 Allow cache on write during compactions when prefetching is URL: https://github.com/apache/hbase/pull/919#issuecomment-563212406 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 13s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 29s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | checkstyle | 1m 20s | master passed | | +1 :green_heart: | shadedjars | 4m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 37s | master passed | | +0 :ok: | spotbugs | 4m 12s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 10s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 2s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | -1 :x: | checkstyle | 1m 18s | hbase-server: The patch generated 3 new + 46 unchanged - 0 fixed = 49 total (was 46) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 36s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 38s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | | +1 :green_heart: | findbugs | 4m 14s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 289m 24s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. | | | | 347m 29s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.util.compaction.TestMajorCompactorTTL | | | hadoop.hbase.master.procedure.TestSCPWithReplicas | | | hadoop.hbase.master.procedure.TestSCPWithReplicasWithoutZKCoordinated | | | hadoop.hbase.client.TestFromClientSide | | | hadoop.hbase.io.hfile.TestCacheOnWrite | | | hadoop.hbase.client.TestFromClientSideWithCoprocessor | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-919/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/919 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux f521e5f59ea1 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-919/out/precommit/personality/provided.sh | | git revision | master / 978546b2f2 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-919/1/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-919/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-919/1/testReport/ | | Max. process+thread count | 4423 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-919/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #884: HBASE-23347 Allowable custom authentication methods for RPCs
wchevreuil commented on a change in pull request #884: HBASE-23347 Allowable
custom authentication methods for RPCs
URL: https://github.com/apache/hbase/pull/884#discussion_r355398083
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/ServerRpcConnection.java
##
@@ -762,18 +750,17 @@ protected final boolean processPreamble(ByteBuffer
preambleBuffer) throws IOExce
return false;
}
}
-if (!this.rpcServer.isSecurityEnabled && authMethod != AuthMethod.SIMPLE) {
- doRawSaslReply(SaslStatus.SUCCESS, new
IntWritable(SaslUtil.SWITCH_TO_SIMPLE_AUTH), null,
-null);
- authMethod = AuthMethod.SIMPLE;
- // client has already sent the initial Sasl message and we
- // should ignore it. Both client and server should fall back
- // to simple auth from now on.
- skipInitialSaslHandshake = true;
-}
-if (authMethod != AuthMethod.SIMPLE) {
- useSasl = true;
-}
+// TODO can we remove this fallback? Is this even a good idea?
+//if (!this.rpcServer.isSecurityEnabled && authMethod !=
AuthMethod.SIMPLE) {
+// doRawSaslReply(SaslStatus.SUCCESS, new
IntWritable(SaslUtil.SWITCH_TO_SIMPLE_AUTH), null,
+//null);
+// authMethod = AuthMethod.SIMPLE;
+// // client has already sent the initial Sasl message and we
+// // should ignore it. Both client and server should fall back
+// // to simple auth from now on.
+// skipInitialSaslHandshake = true;
+//}
Review comment:
Maybe we should keep the current fallback approach and re-think it on a
separate jira? There are few other scenarios where we might want to connect to
an unsecure cluster from a secured environment. Some of the inter-cluster tools
such as CopyTable, ExportSnapshot, HashTable/SyncTable, when dealing with a
secure/unsecure pair, wouldn't be able to have "simple" as the authentication
config value.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #884: HBASE-23347 Allowable custom authentication methods for RPCs
wchevreuil commented on a change in pull request #884: HBASE-23347 Allowable
custom authentication methods for RPCs
URL: https://github.com/apache/hbase/pull/884#discussion_r355419209
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/security/provider/SimpleSaslServerAuthenticationProvider.java
##
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.security.provider;
+
+import java.io.IOException;
+import java.util.Map;
+
+import javax.security.sasl.SaslServer;
+
+import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.token.SecretManager;
+import org.apache.hadoop.security.token.TokenIdentifier;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.AUTHENTICATION)
+@InterfaceStability.Evolving
+public class SimpleSaslServerAuthenticationProvider extends
SimpleSaslClientAuthenticationProvider
+implements SaslServerAuthenticationProvider {
+
+ @Override
+ public SaslServer createServer(SecretManager secretManager,
+ Map saslProps) throws IOException {
+return null;
Review comment:
Maybe we should rather throw an error here? My reading is that under
`SIMPLE`, we should never create an `HBaseSaslRpcServer` instance, and even if
we do, returning null here would just crash things further down in the code
path. At least throwing an error here would explicitly show something is wrong.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] wchevreuil commented on a change in pull request #884: HBASE-23347 Allowable custom authentication methods for RPCs
wchevreuil commented on a change in pull request #884: HBASE-23347 Allowable custom authentication methods for RPCs URL: https://github.com/apache/hbase/pull/884#discussion_r355410032 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcServer.java ## @@ -55,43 +48,10 @@ private UserGroupInformation attemptingUser; // user name before auth Review comment: Looks like this is not set anywhere in the code now. Maybe not needed? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-17512) Optimize ServerName.parsePort() to reduce objects creation
[
https://issues.apache.org/jira/browse/HBASE-17512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991556#comment-16991556
]
Beata Sudi commented on HBASE-17512:
Hi!
[~samglover] are you still working on this issue? If not, I could finish it.
And it seems like parsePort(), parseStartcode() and parseHostname() has been
marked deprecated since 2.0, but still being used in BackupUtils.
> Optimize ServerName.parsePort() to reduce objects creation
> --
>
> Key: HBASE-17512
> URL: https://issues.apache.org/jira/browse/HBASE-17512
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.2.4
>Reporter: Jean-Marc Spaggiari
>Assignee: Samuel David Glover
>Priority: Minor
> Labels: beginner
>
> ServerName.parsePort() calls the split method on a string. This string format
> is like "www.example.org,1234,1212121212" where we try to get only 1234. Each
> time the split() method is called, it creates 4 objets. 3 strings and an
> array. And we just us one of those strings.
> This this method is called in a single place. In the constructor:
> {code}
> private ServerName(final String serverName) {
> this(parseHostname(serverName), parsePort(serverName),
> parseStartcode(serverName));
> }
> {code}
> So parsePort creates 3 string. Te hostname, the port, the startcode, but
> returns only the port, while we call 2 other methods to redo the exact same
> work.
> This constructor is called only there:
> {code}
> /**
>* Retrieve an instance of ServerName.
>* Callers should use the equals method to compare returned instances,
> though we may return
>* a shared immutable object as an internal optimization.
>*/
> public static ServerName valueOf(final String serverName) {
> return new ServerName(serverName);
> }
> {code}
> and this is called here and there. Not intensively, but still, should be
> cleaned.
> Instead of using split, something like this might do better:
> {code}
> public static int parsePort(final String serverName) {
> int indexStart = serverName.indexOf(SERVERNAME_SEPARATOR);
> int indexStop = serverName.lastIndexOf(SERVERNAME_SEPARATOR);
> return Integer.parseInt(serverName.substring(indexStart + 1, indexStop));
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] symat commented on issue #394: HBASE-22716:upgrade zookeeper version to 3.5.5
symat commented on issue #394: HBASE-22716:upgrade zookeeper version to 3.5.5 URL: https://github.com/apache/hbase/pull/394#issuecomment-563247466 FYI: Hadoop trunk already uses Curator 4.2 and ZooKeeper 3.5.6 (see https://issues.apache.org/jira/browse/HADOOP-16579) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23548) Idm Crack Free Download
ellis560 created HBASE-23548: Summary: Idm Crack Free Download Key: HBASE-23548 URL: https://issues.apache.org/jira/browse/HBASE-23548 Project: HBase Issue Type: Bug Reporter: ellis560 [IDM Crack|[http://idm-crack.net/] is the most powerful tools that allows to register your Internet download manager. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23303) Add security headers to REST server/info page
[
https://issues.apache.org/jira/browse/HBASE-23303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991639#comment-16991639
]
Hudson commented on HBASE-23303:
Results for branch master
[build #1562 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1562/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1562//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1562//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1562//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Add security headers to REST server/info page
> -
>
> Key: HBASE-23303
> URL: https://issues.apache.org/jira/browse/HBASE-23303
> Project: HBase
> Issue Type: Improvement
> Components: REST
>Affects Versions: 3.0.0, 2.0.6, 2.1.7, 2.2.2
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3, 2.1.9
>
>
> Vulnerability scanners suggest that the following extra headers should be
> added to both Info/Rest server endpoints which are exposed by {{hbase-rest}}
> project.
> * X-Frame-Options: SAMEORIGIN
> * X-Xss-Protection: 1; mode=block
> * X-Content-Type-Options: nosniff
> * Strict-Transport-Security: “max-age=63072000;includeSubDomains;preload”
> * Content-Security-Policy: default-src https: data: 'unsafe-inline'
> 'unsafe-eval'
> Info server already has "X-Frame-Options: DENY" which is more restrictive
> than "SAMEORIGIN", so it's probably fine. All of three headers are missing
> from REST responses.
> I'll put together a patch to resolve this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22280) Separate read/write handler for priority request(especially for meta).
[
https://issues.apache.org/jira/browse/HBASE-22280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991640#comment-16991640
]
Hudson commented on HBASE-22280:
Results for branch master
[build #1562 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1562/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1562//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1562//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1562//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Separate read/write handler for priority request(especially for meta).
> --
>
> Key: HBASE-22280
> URL: https://issues.apache.org/jira/browse/HBASE-22280
> Project: HBase
> Issue Type: New Feature
> Components: Scheduler
>Reporter: Lijin Bin
>Assignee: Lijin Bin
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.3
>
> Attachments: HBASE-22280.patch
>
>
> Client may give too many read pressure on meta, so blocking master write meta
> for region open.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Deleted] (HBASE-23548) Idm Crack Free Download
[ https://issues.apache.org/jira/browse/HBASE-23548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Thistlethwaite deleted HBASE-23548: - > Idm Crack Free Download > --- > > Key: HBASE-23548 > URL: https://issues.apache.org/jira/browse/HBASE-23548 > Project: HBase > Issue Type: Bug >Reporter: ellis560 >Priority: Major > > [IDM Crack|[http://idm-crack.net/] is the most powerful tools that allows to > register your Internet download manager. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] joshelser commented on issue #722: HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs
joshelser commented on issue #722: HBASE-23065 [hbtop] Top-N heavy hitter user and client drill downs URL: https://github.com/apache/hbase/pull/722#issuecomment-563274555 > Can I merge this OK by me. I know Ankit was traveling with intermittent access. I think he might be back soon, but I don't think you need to wait for him if you think this is good. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23376) NPE happens while replica region is moving
[
https://issues.apache.org/jira/browse/HBASE-23376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991666#comment-16991666
]
HBase QA commented on HBASE-23376:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
9s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
0s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
26s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
2s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 4s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
42s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}230m 47s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 3m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}301m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/1067/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-23376 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12988301/HBASE-23376.master.v02.dummy.patch
|
| Optional Tests | d
[GitHub] [hbase] Apache-HBase commented on issue #820: HBASE-23286 Improve MTTR: Split WAL to HFile
Apache-HBase commented on issue #820: HBASE-23286 Improve MTTR: Split WAL to HFile URL: https://github.com/apache/hbase/pull/820#issuecomment-563277815 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 51s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 23s | master passed | | +1 :green_heart: | compile | 1m 18s | master passed | | +1 :green_heart: | checkstyle | 1m 54s | master passed | | +1 :green_heart: | shadedjars | 4m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | | +0 :ok: | spotbugs | 4m 5s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 52s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 1s | the patch passed | | +1 :green_heart: | compile | 1m 20s | the patch passed | | +1 :green_heart: | javac | 1m 20s | the patch passed | | -1 :x: | checkstyle | 1m 32s | hbase-server: The patch generated 4 new + 324 unchanged - 0 fixed = 328 total (was 324) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 38s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 33s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 59s | the patch passed | | +1 :green_heart: | findbugs | 5m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 9s | hbase-common in the patch passed. | | -1 :x: | unit | 159m 16s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 1m 6s | The patch does not generate ASF License warnings. | | | | 225m 22s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestAsyncTableGetMultiThreaded | | | hadoop.hbase.quotas.TestClusterScopeQuotaThrottle | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/820 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 4e818f13f62f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-820/out/precommit/personality/provided.sh | | git revision | master / 82e155eb26 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/5/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/5/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/5/testReport/ | | Max. process+thread count | 4217 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/5/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] wchevreuil commented on issue #45: HBASE-23371 [HBCK2] Provide client side method for removing ghost regions in meta.
wchevreuil commented on issue #45: HBASE-23371 [HBCK2] Provide client side method for removing ghost regions in meta. URL: https://github.com/apache/hbase-operator-tools/pull/45#issuecomment-563294564 New commit pushed to this PR. Changes: 1) Made a single command "extraRegionsInMeta" with a "-f/-fix" option as suggested by Stack; 2) Added remarks about "assigns" command in the help text, as pointed out by Daisuke; This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-operator-tools] asf-ci commented on issue #45: HBASE-23371 [HBCK2] Provide client side method for removing ghost regions in meta.
asf-ci commented on issue #45: HBASE-23371 [HBCK2] Provide client side method for removing ghost regions in meta. URL: https://github.com/apache/hbase-operator-tools/pull/45#issuecomment-563297675 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/PreCommit-HBASE-OPERATOR-TOOLS-Build/121/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] wchevreuil merged pull request #894: HBASE-23073 Add an optional costFunction to balance regions according to a capacity rule
wchevreuil merged pull request #894: HBASE-23073 Add an optional costFunction to balance regions according to a capacity rule URL: https://github.com/apache/hbase/pull/894 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (HBASE-23073) Add an optional costFunction to balance regions according to a capacity rule
[ https://issues.apache.org/jira/browse/HBASE-23073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil resolved HBASE-23073. -- Resolution: Fixed Merged into branch-1. Thanks for the contribution, [~PierreZ]! > Add an optional costFunction to balance regions according to a capacity rule > > > Key: HBASE-23073 > URL: https://issues.apache.org/jira/browse/HBASE-23073 > Project: HBase > Issue Type: New Feature > Components: master >Affects Versions: 3.0.0 >Reporter: Pierre Zemb >Assignee: Pierre Zemb >Priority: Minor > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-23073.branch-1.0002.patch, > HBASE-23073.branch-1.001.patch > > > Based on the work in > [HBASE-22618|https://issues.apache.org/jira/browse/HBASE-22618], users can > now load custom costFunctions inside the main balancer used by HBase. As an > example, we like like to add upstream an optional cost function called > HeterogeneousRegionCountCostFunction that will deal with our issue: how to > balance regions according to the capacity of a RS instead of using the > RegionCountSkewCostFunction that is trying to avoid skew. > A rule file is loaded from HDFS before balancing. It contains lines of rules. > A rule is composed of a regexp for hostname, and a limit. For example, we > could have: > * rs[0-9] 200 > * rs1[0-9] 50 > RegionServers with hostname matching the first rules will have a limit of > 200, and the others 50. If there's no match, a default is set. > Thanks to the rule, we have two informations: the max number of regions for > this cluster, and the rules for each servers. HeterogeneousBalancer will try > to balance regions according to their capacity. > Let's take an example. Let's say that we have 20 RS: > 10 RS, named through rs0 to rs9 loaded with 60 regions each, and each can > handle 200 regions. > 10 RS, named through rs10 to rs19 loaded with 60 regions each, and each > can support 50 regions. > Based on the following rules: > rs[0-9] 200 > rs1[0-9] 50 > The second group is overloaded, whereas the first group has plenty of space. > Moving a region from the first group to the second should provide a lower > cost. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23073) Add an optional costFunction to balance regions according to a capacity rule
[ https://issues.apache.org/jira/browse/HBASE-23073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-23073: - Fix Version/s: 1.6.0 > Add an optional costFunction to balance regions according to a capacity rule > > > Key: HBASE-23073 > URL: https://issues.apache.org/jira/browse/HBASE-23073 > Project: HBase > Issue Type: New Feature > Components: master >Affects Versions: 3.0.0 >Reporter: Pierre Zemb >Assignee: Pierre Zemb >Priority: Minor > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-23073.branch-1.0002.patch, > HBASE-23073.branch-1.001.patch > > > Based on the work in > [HBASE-22618|https://issues.apache.org/jira/browse/HBASE-22618], users can > now load custom costFunctions inside the main balancer used by HBase. As an > example, we like like to add upstream an optional cost function called > HeterogeneousRegionCountCostFunction that will deal with our issue: how to > balance regions according to the capacity of a RS instead of using the > RegionCountSkewCostFunction that is trying to avoid skew. > A rule file is loaded from HDFS before balancing. It contains lines of rules. > A rule is composed of a regexp for hostname, and a limit. For example, we > could have: > * rs[0-9] 200 > * rs1[0-9] 50 > RegionServers with hostname matching the first rules will have a limit of > 200, and the others 50. If there's no match, a default is set. > Thanks to the rule, we have two informations: the max number of regions for > this cluster, and the rules for each servers. HeterogeneousBalancer will try > to balance regions according to their capacity. > Let's take an example. Let's say that we have 20 RS: > 10 RS, named through rs0 to rs9 loaded with 60 regions each, and each can > handle 200 regions. > 10 RS, named through rs10 to rs19 loaded with 60 regions each, and each > can support 50 regions. > Based on the following rules: > rs[0-9] 200 > rs1[0-9] 50 > The second group is overloaded, whereas the first group has plenty of space. > Moving a region from the first group to the second should provide a lower > cost. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23549) Document steps to disable MOB for a column family
Sean Busbey created HBASE-23549: --- Summary: Document steps to disable MOB for a column family Key: HBASE-23549 URL: https://issues.apache.org/jira/browse/HBASE-23549 Project: HBase Issue Type: Improvement Components: documentation, mob Reporter: Sean Busbey Assignee: Sean Busbey Ref guide has steps for enabling MOB on a column family but has no corresponding instructions on safely turning the feature off. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-23549) Document steps to disable MOB for a column family
[ https://issues.apache.org/jira/browse/HBASE-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-23549 started by Sean Busbey. --- > Document steps to disable MOB for a column family > - > > Key: HBASE-23549 > URL: https://issues.apache.org/jira/browse/HBASE-23549 > Project: HBase > Issue Type: Improvement > Components: documentation, mob >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > > Ref guide has steps for enabling MOB on a column family but has no > corresponding instructions on safely turning the feature off. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23549) Document steps to disable MOB for a column family
[ https://issues.apache.org/jira/browse/HBASE-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991735#comment-16991735 ] Sean Busbey commented on HBASE-23549: - there is no missing step 4, I just flubbed the numbering. > Document steps to disable MOB for a column family > - > > Key: HBASE-23549 > URL: https://issues.apache.org/jira/browse/HBASE-23549 > Project: HBase > Issue Type: Improvement > Components: documentation, mob >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Minor > > Ref guide has steps for enabling MOB on a column family but has no > corresponding instructions on safely turning the feature off. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23549) Document steps to disable MOB for a column family
[
https://issues.apache.org/jira/browse/HBASE-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991734#comment-16991734
]
Sean Busbey commented on HBASE-23549:
-
confirmed this works, does not require an outage but will have a brief blip of
client retries when the regions are reloaded.
1) Ensure the mob compaction chore is off by having
hbase.mob.file.compaction.chore.period set to 0 for the master. (requires
rolling restart of masters) You should also ensure you do not issue any mob
compactions via the hbase shell.
2) In the hbase shell change the MOB size threshold for the column family
you’re moving out of MOB to be larger then the largest cell e.g. table named
‘table_1’ and column named ‘example_cf’ and we'll use a max cell size of 1GB
{code}
hbase(main):011:0> alter 'table_1', {NAME => 'example_cf', MOB_THRESHOLD =>
'10'}
Updating all regions with the new schema...
9/25 regions updated.
25/25 regions updated.
Done.
0 row(s) in 3.4940 seconds
{code}
Note that if you are still ingesting data you must ensure this threshold is
larger than any cell value. MAX_INT is a safe choice.
3) Perform a major compaction on the table (a normal major compaction, not a
mob compaction)
{code}
hbase(main):012:0> major_compact 'table_1'
0 row(s) in 0.2600 seconds
{code}
5) Since that command is asynchronous, you’ll have to monitor for the end.
{code}
hbase(main):015:0> @hbase.admin(@formatter).instance_eval do p
@admin.get_compaction_state('table_1').to_string end
“MAJOR”
{code}
When it’s done this command will say “NONE”
{code}
hbase(main):016:0> @hbase.admin(@formatter).instance_eval do p
@admin.get_compaction_state('table_1').to_string end
“NONE”
{code}
6) Run the MobRefReporter job to ensure there are no mob cells. In particular
the job should show 0 input records
{code}
...
Map-Reduce Framework
Map input records=0
...
{code}
7) Alter the column family configs so that the MOB feature is disabled
{code}
hbase(main):017:0> alter 'table_1', {NAME => 'example_cf', IS_MOB => 'false'}
Updating all regions with the new schema...
8/25 regions updated.
25/25 regions updated.
Done.
0 row(s) in 2.9370 seconds
{code}
8 ) There will be residual data in the mobdir area.
{code}
$ hdfs dfs -count /hbase/mobdir/data/default/table_1
4 54 9063269081 /hbase/mobdir/data/default/table_1
{code}
This data is spurious and may be reclaimed. You should sideline it, verify your
application’s view of the table, and then delete it.
> Document steps to disable MOB for a column family
> -
>
> Key: HBASE-23549
> URL: https://issues.apache.org/jira/browse/HBASE-23549
> Project: HBase
> Issue Type: Improvement
> Components: documentation, mob
>Reporter: Sean Busbey
>Assignee: Sean Busbey
>Priority: Minor
>
> Ref guide has steps for enabling MOB on a column family but has no
> corresponding instructions on safely turning the feature off.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk merged pull request #918: HBASE-23373 Log `RetriesExhaustedException` context with full time precision (#903)
ndimiduk merged pull request #918: HBASE-23373 Log `RetriesExhaustedException` context with full time precision (#903) URL: https://github.com/apache/hbase/pull/918 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23373) Log `RetriesExhaustedException` context with full time precision
[
https://issues.apache.org/jira/browse/HBASE-23373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-23373:
-
Fix Version/s: 2.1.8
> Log `RetriesExhaustedException` context with full time precision
>
>
> Key: HBASE-23373
> URL: https://issues.apache.org/jira/browse/HBASE-23373
> Project: HBase
> Issue Type: Improvement
> Components: asyncclient, Client
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.1.8, 2.2.3
>
>
> {{RetriesExhaustedException}} will print out a list of the exceptions its
> accumulated while make its attempts. These exceptions are prefixed with a
> timestamp indicating when the exception was collected. We currently print
> this out using {{Date.toString()}}, which only has second granularity and is
> inconsistent by locale. Normalize the locale and include full precision.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-23373) Log `RetriesExhaustedException` context with full time precision
[
https://issues.apache.org/jira/browse/HBASE-23373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk resolved HBASE-23373.
--
Resolution: Fixed
> Log `RetriesExhaustedException` context with full time precision
>
>
> Key: HBASE-23373
> URL: https://issues.apache.org/jira/browse/HBASE-23373
> Project: HBase
> Issue Type: Improvement
> Components: asyncclient, Client
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
>Priority: Minor
> Fix For: 3.0.0, 2.3.0, 2.1.8, 2.2.3
>
>
> {{RetriesExhaustedException}} will print out a list of the exceptions its
> accumulated while make its attempts. These exceptions are prefixed with a
> timestamp indicating when the exception was collected. We currently print
> this out using {{Date.toString()}}, which only has second granularity and is
> inconsistent by locale. Normalize the locale and include full precision.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on issue #872: HBASE-23333 Provide call context around timeouts and other failure scenarios
ndimiduk commented on issue #872: HBASE-2 Provide call context around timeouts and other failure scenarios URL: https://github.com/apache/hbase/pull/872#issuecomment-563323767 Still have objections @Apache9 ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23073) Add an optional costFunction to balance regions according to a capacity rule
[ https://issues.apache.org/jira/browse/HBASE-23073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991772#comment-16991772 ] Pierre Zemb commented on HBASE-23073: - Thank you [~wchevreuil] for your patience ! > Add an optional costFunction to balance regions according to a capacity rule > > > Key: HBASE-23073 > URL: https://issues.apache.org/jira/browse/HBASE-23073 > Project: HBase > Issue Type: New Feature > Components: master >Affects Versions: 3.0.0 >Reporter: Pierre Zemb >Assignee: Pierre Zemb >Priority: Minor > Fix For: 3.0.0, 2.3.0, 1.6.0 > > Attachments: HBASE-23073.branch-1.0002.patch, > HBASE-23073.branch-1.001.patch > > > Based on the work in > [HBASE-22618|https://issues.apache.org/jira/browse/HBASE-22618], users can > now load custom costFunctions inside the main balancer used by HBase. As an > example, we like like to add upstream an optional cost function called > HeterogeneousRegionCountCostFunction that will deal with our issue: how to > balance regions according to the capacity of a RS instead of using the > RegionCountSkewCostFunction that is trying to avoid skew. > A rule file is loaded from HDFS before balancing. It contains lines of rules. > A rule is composed of a regexp for hostname, and a limit. For example, we > could have: > * rs[0-9] 200 > * rs1[0-9] 50 > RegionServers with hostname matching the first rules will have a limit of > 200, and the others 50. If there's no match, a default is set. > Thanks to the rule, we have two informations: the max number of regions for > this cluster, and the rules for each servers. HeterogeneousBalancer will try > to balance regions according to their capacity. > Let's take an example. Let's say that we have 20 RS: > 10 RS, named through rs0 to rs9 loaded with 60 regions each, and each can > handle 200 regions. > 10 RS, named through rs10 to rs19 loaded with 60 regions each, and each > can support 50 regions. > Based on the following rules: > rs[0-9] 200 > rs1[0-9] 50 > The second group is overloaded, whereas the first group has plenty of space. > Moving a region from the first group to the second should provide a lower > cost. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23550) FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length (8) exceed the capacity of the array: 41
Alex Batyrshin created HBASE-23550:
--
Summary: FSHLog crash with java.lang.IllegalArgumentException:
offset (40) + length (8) exceed the capacity of the array: 41
Key: HBASE-23550
URL: https://issues.apache.org/jira/browse/HBASE-23550
Project: HBase
Issue Type: Bug
Affects Versions: 1.4.10
Reporter: Alex Batyrshin
{code:java}
$ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
file:///tmp/hbase06-wal
...
Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
timestamp=Mon Dec 09 18:46:30 MSK 2019
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:_0
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:apd
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:st
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:pt
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:gt
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:d:o
cell total size sum: 128
edit heap size: 784
position: 49623
Exception in thread "main" java.lang.IllegalArgumentException: offset (40) +
length (8) exceed the capacity of the array: 41
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:779)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:753)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:738)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1569)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1560)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.toStringMap(WALPrettyPrinter.java:352)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:297)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:438)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.main(FSHLog.java:2028){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23550) FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length (8) exceed the capacity of the array: 41
[
https://issues.apache.org/jira/browse/HBASE-23550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Batyrshin updated HBASE-23550:
---
Description:
{code:java}
{code}
{code:java}
$ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
file:///tmp/hbase06-wal
...
Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
timestamp=Mon Dec 09 18:46:30 MSK 2019
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:_0
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:apd
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:st
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:pt
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:gt
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:d:o
cell total size sum: 128
edit heap size: 784
position: 49623
Exception in thread "main" java.lang.IllegalArgumentException: offset (40) +
length (8) exceed the capacity of the array: 41
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:779)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:753)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:738)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1569)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1560)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.toStringMap(WALPrettyPrinter.java:352)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:297)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:438)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.main(FSHLog.java:2028){code}
was:
{code:java}
$ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
file:///tmp/hbase06-wal
...
Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
timestamp=Mon Dec 09 18:46:30 MSK 2019
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:_0
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:apd
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:st
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:pt
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:gt
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:d:o
cell total size sum: 128
edit heap size: 784
position: 49623
Exception in thread "main" java.lang.IllegalArgumentException: offset (40) +
length (8) exceed the capacity of the array: 41
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:779)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:753)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:738)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1569)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1560)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.toStringMap(WALPrettyPrinter.java:352)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:297)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:438)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.main(FSHLog.java:2028){code}
> FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length
> (8) exceed the capacity of the array: 41
> ---
>
> Key: HBASE-23550
> URL: https://issues.apache.org/jira/browse/HBASE-23550
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.10
>Reporter: Alex Batyrshin
>Priority: Major
>
> {code:java}
> {code}
> {code:java}
> $ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
> file:///tmp/hbase06-wal
> ...
> Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
> timestamp=Mon Dec 09 18:46:30 MSK 2019
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:_0
> cell total size sum: 120
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:apd
> cell total size sum: 128
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:st
> cell total size sum: 120
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:pt
[jira] [Updated] (HBASE-22869) JavaDoc for NullComparator is the same as BinaryComparator
[
https://issues.apache.org/jira/browse/HBASE-22869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Clay B. updated HBASE-22869:
Attachment: 0002-HBASE-22869-JavaDoc-for-NullComparator-is-the-same-a.patch
> JavaDoc for NullComparator is the same as BinaryComparator
> --
>
> Key: HBASE-22869
> URL: https://issues.apache.org/jira/browse/HBASE-22869
> Project: HBase
> Issue Type: Improvement
> Components: documentation
>Reporter: Clay B.
>Assignee: Clay B.
>Priority: Trivial
> Attachments:
> 0001-HBASE-22869-JavaDoc-for-NullComparator-is-the-same-a.patch,
> 0001-HBASE-22869-JavaDoc-for-NullComparator-is-the-same-a.patch,
> 0002-HBASE-22869-JavaDoc-for-NullComparator-is-the-same-a.patch
>
>
> Looking at the list of default HBase
> [filters|https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/package-summary.html]
> one can not see what is null about {{NullComparator}} as its JavaDoc is
> identical to
> [{{BinaryComparator}}|https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/filter/BinaryComparator.html].
> It would be nice to know how it is "null" for easier adoption.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23550) FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length (8) exceed the capacity of the array: 41
[
https://issues.apache.org/jira/browse/HBASE-23550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Batyrshin updated HBASE-23550:
---
Description:
{code:java}
{code}
{code:java}
$ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
file:///tmp/hbase06-wal
...
Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
timestamp=Mon Dec 09 18:46:30 MSK 2019
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:_0
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:apd
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:st
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:pt
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:gt
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:d:o
cell total size sum: 128
edit heap size: 784
position: 49623
Exception in thread "main" java.lang.IllegalArgumentException: offset (40) +
length (8) exceed the capacity of the array: 41
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:779)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:753)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:738)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1569)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1560)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.toStringMap(WALPrettyPrinter.java:352)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:297)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:438)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.main(FSHLog.java:2028){code}
was:
{code:java}
{code}
{code:java}
$ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
file:///tmp/hbase06-wal
...
Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
timestamp=Mon Dec 09 18:46:30 MSK 2019
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:_0
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:apd
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:st
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:pt
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:gt
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:d:o
cell total size sum: 128
edit heap size: 784
position: 49623
Exception in thread "main" java.lang.IllegalArgumentException: offset (40) +
length (8) exceed the capacity of the array: 41
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:779)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:753)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:738)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1569)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1560)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.toStringMap(WALPrettyPrinter.java:352)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:297)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:438)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.main(FSHLog.java:2028){code}
> FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length
> (8) exceed the capacity of the array: 41
> ---
>
> Key: HBASE-23550
> URL: https://issues.apache.org/jira/browse/HBASE-23550
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.10
>Reporter: Alex Batyrshin
>Priority: Major
>
> {code:java}
> {code}
> {code:java}
> $ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
> file:///tmp/hbase06-wal
> ...
> Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
> timestamp=Mon Dec 09 18:46:30 MSK 2019
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:_0
> cell total size sum: 120
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:apd
> cell total size sum: 128
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:st
> cell total size sum: 120
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jT
[jira] [Updated] (HBASE-23550) FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length (8) exceed the capacity of the array: 41
[
https://issues.apache.org/jira/browse/HBASE-23550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Batyrshin updated HBASE-23550:
---
Description:
{code:java}
{code}
{code:java}
$ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
file:///tmp/hbase06-wal
...
Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
timestamp=Mon Dec 09 18:46:30 MSK 2019
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:_0
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:apd
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:st
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:pt
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:gt
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:d:o
cell total size sum: 128
edit heap size: 784
position: 49623
Exception in thread "main" java.lang.IllegalArgumentException: offset (40) +
length (8) exceed the capacity of the array: 41
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:779)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:753)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:738)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1569)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1560)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.toStringMap(WALPrettyPrinter.java:352)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:297)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:438)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.main(FSHLog.java:2028){code}
was:
{code:java}
{code}
{code:java}
$ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
file:///tmp/hbase06-wal
...
Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
timestamp=Mon Dec 09 18:46:30 MSK 2019
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:_0
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:apd
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:st
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:pt
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:gt
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:d:o
cell total size sum: 128
edit heap size: 784
position: 49623
Exception in thread "main" java.lang.IllegalArgumentException: offset (40) +
length (8) exceed the capacity of the array: 41
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:779)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:753)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:738)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1569)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1560)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.toStringMap(WALPrettyPrinter.java:352)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:297)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:438)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.main(FSHLog.java:2028){code}
> FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length
> (8) exceed the capacity of the array: 41
> ---
>
> Key: HBASE-23550
> URL: https://issues.apache.org/jira/browse/HBASE-23550
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.10
>Reporter: Alex Batyrshin
>Priority: Major
>
> {code:java}
> {code}
> {code:java}
> $ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
> file:///tmp/hbase06-wal
> ...
> Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
> timestamp=Mon Dec 09 18:46:30 MSK 2019
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:_0
> cell total size sum: 120
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:apd
> cell total size sum: 128
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:st
> cell total size sum: 120
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?j
[jira] [Updated] (HBASE-23550) FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length (8) exceed the capacity of the array: 41
[
https://issues.apache.org/jira/browse/HBASE-23550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Batyrshin updated HBASE-23550:
---
Description:
We are using HBase-1.4.10 with Phoenix-4.14.2, so config file looks like:
{code:java}
hbase.regionserver.wal.codec
org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec
{code}
Trying to dump wal
{code:java}
$ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
file:///tmp/hbase06-wal
...
Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
timestamp=Mon Dec 09 18:46:30 MSK 2019
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:_0
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:apd
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:st
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:pt
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:gt
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:d:o
cell total size sum: 128
edit heap size: 784
position: 49623
Exception in thread "main" java.lang.IllegalArgumentException: offset (40) +
length (8) exceed the capacity of the array: 41
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:779)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:753)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:738)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1569)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1560)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.toStringMap(WALPrettyPrinter.java:352)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:297)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:438)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.main(FSHLog.java:2028){code}
was:
{code:java}
{code}
{code:java}
$ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
file:///tmp/hbase06-wal
...
Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
timestamp=Mon Dec 09 18:46:30 MSK 2019
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:_0
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:apd
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:st
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:pt
cell total size sum: 120
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
column=d:d:gt
cell total size sum: 128
row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u, column=d:d:o
cell total size sum: 128
edit heap size: 784
position: 49623
Exception in thread "main" java.lang.IllegalArgumentException: offset (40) +
length (8) exceed the capacity of the array: 41
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:779)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:753)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:738)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1569)
at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1560)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.toStringMap(WALPrettyPrinter.java:352)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:297)
at
org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:438)
at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.main(FSHLog.java:2028){code}
> FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length
> (8) exceed the capacity of the array: 41
> ---
>
> Key: HBASE-23550
> URL: https://issues.apache.org/jira/browse/HBASE-23550
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.10
>Reporter: Alex Batyrshin
>Priority: Major
>
> We are using HBase-1.4.10 with Phoenix-4.14.2, so config file looks like:
> {code:java}
>
> hbase.regionserver.wal.codec
>
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec
> {code}
> Trying to dump wal
> {code:java}
> $ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
> file:///tmp/hbase06-wal
> ...
> Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
> timestamp=Mon Dec 09 18:46:30 MSK 2019
> row=\x1
[jira] [Commented] (HBASE-22869) JavaDoc for NullComparator is the same as BinaryComparator
[
https://issues.apache.org/jira/browse/HBASE-22869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991828#comment-16991828
]
HBase QA commented on HBASE-22869:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
17s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
19s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
20s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
18m 25s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
57s{color} | {color:green} hbase-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/PreCommit-HBASE-Build/1068/artifact/patchprocess/Dockerfile
|
| JIRA Issue | HBASE-22869 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12988356/0002-HBASE-22869-JavaDoc-for-NullComparator-is-the-same-a.patch
|
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux fbcbd7cfcfa6 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1
05:24:09 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 82e155eb26 |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.ap
[jira] [Created] (HBASE-23551) random high ports not reported properly in the UI
Allan Espinosa created HBASE-23551: -- Summary: random high ports not reported properly in the UI Key: HBASE-23551 URL: https://issues.apache.org/jira/browse/HBASE-23551 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 2.2.2 Reporter: Allan Espinosa We want to run HBase on kubernetes and want to investigate hostNetworking for performance and host purposes. We basically set hbase.*.port,hbase.*.info.port=0 in our hbase-site.xml . This will allow us to skip kubernetes networking since hbase-related services will bind to random high ports. We encountered the following bugs in the HBase UI Active HBase Master: * Links to Backup masters points to http://: instead of http://:. The http://: endpoint responds with a proper HTTP 404 from the Jetty instance in the backup master process Backup HBase master * Links to Active Master points to http://: instead of http://:. The http://: endpoint responds with a proper HTTP 404 from the Jetty instance in the backup master process Regionservers * Regionserver running in the same host network as the active hbase master can register * Regionserver on different hosts (using host networking) connects to a different RPC port in the active HBase master. I can’t figure out which port is this yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23551) random high ports not reported properly in the UI
[ https://issues.apache.org/jira/browse/HBASE-23551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allan Espinosa updated HBASE-23551: --- Description: We want to run HBase on kubernetes and want to investigate hostNetworking for performance and host purposes. We basically set hbase.<>.port,hbase.<>.info.port=0 in our hbase-site.xml . This will allow us to skip kubernetes networking since hbase-related services will bind to random high ports. We encountered the following bugs in the HBase UI Active HBase Master: * Links to Backup masters points to http://: instead of http://:. The http://: endpoint responds with a proper HTTP 404 from the Jetty instance in the backup master process Backup HBase master * Links to Active Master points to http://: instead of http://:. The http://: endpoint responds with a proper HTTP 404 from the Jetty instance in the backup master process Regionservers * Regionserver running in the same host network as the active hbase master can register * Regionserver on different hosts (using host networking) connects to a different RPC port in the active HBase master. I can’t figure out which port is this yet. was: We want to run HBase on kubernetes and want to investigate hostNetworking for performance and host purposes. We basically set hbase.*.port,hbase.*.info.port=0 in our hbase-site.xml . This will allow us to skip kubernetes networking since hbase-related services will bind to random high ports. We encountered the following bugs in the HBase UI Active HBase Master: * Links to Backup masters points to http://: instead of http://:. The http://: endpoint responds with a proper HTTP 404 from the Jetty instance in the backup master process Backup HBase master * Links to Active Master points to http://: instead of http://:. The http://: endpoint responds with a proper HTTP 404 from the Jetty instance in the backup master process Regionservers * Regionserver running in the same host network as the active hbase master can register * Regionserver on different hosts (using host networking) connects to a different RPC port in the active HBase master. I can’t figure out which port is this yet. > random high ports not reported properly in the UI > - > > Key: HBASE-23551 > URL: https://issues.apache.org/jira/browse/HBASE-23551 > Project: HBase > Issue Type: Bug > Components: master, regionserver >Affects Versions: 2.2.2 >Reporter: Allan Espinosa >Priority: Major > > We want to run HBase on kubernetes and want to investigate hostNetworking for > performance and host purposes. We basically set > hbase.<>.port,hbase.<>.info.port=0 in our hbase-site.xml . This will allow > us to skip kubernetes networking since hbase-related services will bind to > random high ports. > > We encountered the following bugs in the HBase UI > Active HBase Master: > * Links to Backup masters points to > http://: instead of > http://:. The > http://: endpoint responds with a proper > HTTP 404 from the Jetty instance in the backup master process > Backup HBase master > * Links to Active Master points to http://: > instead of http://:. The > http://: endpoint responds with a proper > HTTP 404 from the Jetty instance in the backup master process > Regionservers > * Regionserver running in the same host network as the active hbase master > can register > * Regionserver on different hosts (using host networking) connects to a > different RPC port in the active HBase master. I can’t figure out which port > is this yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #820: HBASE-23286 Improve MTTR: Split WAL to HFile
Apache-HBase commented on issue #820: HBASE-23286 Improve MTTR: Split WAL to HFile URL: https://github.com/apache/hbase/pull/820#issuecomment-563374426 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 11s | master passed | | +1 :green_heart: | compile | 1m 20s | master passed | | +1 :green_heart: | checkstyle | 1m 51s | master passed | | +1 :green_heart: | shadedjars | 4m 33s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | master passed | | +0 :ok: | spotbugs | 4m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 57s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 3s | the patch passed | | +1 :green_heart: | compile | 1m 18s | the patch passed | | +1 :green_heart: | javac | 1m 18s | the patch passed | | +1 :green_heart: | checkstyle | 1m 50s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 35s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 39s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 56s | the patch passed | | +1 :green_heart: | findbugs | 5m 6s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 10s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 155m 5s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 1m 6s | The patch does not generate ASF License warnings. | | | | 220m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/820 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e0bdf781ff11 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-820/out/precommit/personality/provided.sh | | git revision | master / 82e155eb26 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/6/testReport/ | | Max. process+thread count | 4126 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-820/6/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] belugabehr commented on a change in pull request #912: HBASE-23380: General Cleanup of FSUtil
belugabehr commented on a change in pull request #912: HBASE-23380: General
Cleanup of FSUtil
URL: https://github.com/apache/hbase/pull/912#discussion_r355617474
##
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
##
@@ -227,12 +232,8 @@ public static FSDataOutputStream create(Configuration
conf, FileSystem fs, Path
throw new IOException(ite.getCause());
} catch (NoSuchMethodException e) {
LOG.debug("DFS Client does not support most favored nodes create;
using default create");
- if (LOG.isTraceEnabled()) LOG.trace("Ignoring; use default create",
e);
-} catch (IllegalArgumentException e) {
- LOG.debug("Ignoring (most likely Reflection related exception) " +
e);
-} catch (SecurityException e) {
- LOG.debug("Ignoring (most likely Reflection related exception) " +
e);
-} catch (IllegalAccessException e) {
+ LOG.trace("Ignoring; use default create", e);
Review comment:
For such a trivial case, there is not need to check for trace being enabled.
The trace method itself will check internally if trace is enabled. Checking
for trace logging here would only be required if there was some sort of
complicated computation that would simply be thrown away every time. Passing a
static string and a reference to an Exception does not require any computation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] belugabehr commented on issue #912: HBASE-23380: General Cleanup of FSUtil
belugabehr commented on issue #912: HBASE-23380: General Cleanup of FSUtil URL: https://github.com/apache/hbase/pull/912#issuecomment-563375849 @HorizonNet Thanks for the review! Much appreciated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] belugabehr commented on issue #912: HBASE-23380: General Cleanup of FSUtil
belugabehr commented on issue #912: HBASE-23380: General Cleanup of FSUtil URL: https://github.com/apache/hbase/pull/912#issuecomment-563376670 Rebased on master This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23552) Cleanup javadocs on ITBLL
Nick Dimiduk created HBASE-23552: Summary: Cleanup javadocs on ITBLL Key: HBASE-23552 URL: https://issues.apache.org/jira/browse/HBASE-23552 Project: HBase Issue Type: Test Components: integration tests Reporter: Nick Dimiduk We have this nice description in the java doc on ITBLL but it's unformatted and thus illegible. Clean it up to it can be read by humans. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23552) Format Javadocs on ITBLL
[ https://issues.apache.org/jira/browse/HBASE-23552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23552: - Summary: Format Javadocs on ITBLL (was: Cleanup javadocs on ITBLL) > Format Javadocs on ITBLL > > > Key: HBASE-23552 > URL: https://issues.apache.org/jira/browse/HBASE-23552 > Project: HBase > Issue Type: Test > Components: integration tests >Reporter: Nick Dimiduk >Priority: Trivial > > We have this nice description in the java doc on ITBLL but it's unformatted > and thus illegible. Clean it up to it can be read by humans. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] ndimiduk opened a new pull request #920: HBASE-23552 Format Javadocs on ITBLL
ndimiduk opened a new pull request #920: HBASE-23552 Format Javadocs on ITBLL URL: https://github.com/apache/hbase/pull/920 We have this nice description in the java doc on ITBLL but it's unformatted and thus illegible. Add some formatting so that it can be read by humans. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (HBASE-23552) Format Javadocs on ITBLL
[ https://issues.apache.org/jira/browse/HBASE-23552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk reassigned HBASE-23552: Assignee: Nick Dimiduk > Format Javadocs on ITBLL > > > Key: HBASE-23552 > URL: https://issues.apache.org/jira/browse/HBASE-23552 > Project: HBase > Issue Type: Test > Components: integration tests >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Trivial > > We have this nice description in the java doc on ITBLL but it's unformatted > and thus illegible. Add some formatting so that it can be read by humans. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23552) Format Javadocs on ITBLL
[ https://issues.apache.org/jira/browse/HBASE-23552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-23552: - Description: We have this nice description in the java doc on ITBLL but it's unformatted and thus illegible. Add some formatting so that it can be read by humans. (was: We have this nice description in the java doc on ITBLL but it's unformatted and thus illegible. Clean it up to it can be read by humans.) > Format Javadocs on ITBLL > > > Key: HBASE-23552 > URL: https://issues.apache.org/jira/browse/HBASE-23552 > Project: HBase > Issue Type: Test > Components: integration tests >Reporter: Nick Dimiduk >Priority: Trivial > > We have this nice description in the java doc on ITBLL but it's unformatted > and thus illegible. Add some formatting so that it can be read by humans. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HBASE-23552) Format Javadocs on ITBLL
[ https://issues.apache.org/jira/browse/HBASE-23552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-23552 started by Nick Dimiduk. > Format Javadocs on ITBLL > > > Key: HBASE-23552 > URL: https://issues.apache.org/jira/browse/HBASE-23552 > Project: HBase > Issue Type: Test > Components: integration tests >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk >Priority: Trivial > > We have this nice description in the java doc on ITBLL but it's unformatted > and thus illegible. Add some formatting so that it can be read by humans. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] VladRodionov closed pull request #623: HBASE-22749: Distributed MOB compactions
VladRodionov closed pull request #623: HBASE-22749: Distributed MOB compactions URL: https://github.com/apache/hbase/pull/623 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] belugabehr commented on a change in pull request #911: HBASE-23379: Clean Up FSUtil getRegionLocalityMappingFromFS
belugabehr commented on a change in pull request #911: HBASE-23379: Clean Up
FSUtil getRegionLocalityMappingFromFS
URL: https://github.com/apache/hbase/pull/911#discussion_r355643452
##
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
##
@@ -1708,44 +1702,36 @@ public boolean accept(Path path) {
FileStatus[] statusList = fs.globStatus(queryPath, pathFilter);
+if (LOG.isDebugEnabled()) {
+ LOG.debug("Query Path: {} ; # list of files: {}", queryPath,
Arrays.toString(statusList));
+}
+
if (null == statusList) {
return;
-} else {
- LOG.debug("Query Path: " + queryPath + " ; # list of files: " +
- statusList.length);
}
// lower the number of threads in case we have very few expected regions
-threadPoolSize = Math.min(threadPoolSize, statusList.length);
+final int finalThreadPoolSize = Math.min(threadPoolSize,
statusList.length);
Review comment:
OK. Will make requested change. Please note that it's easier to debug when
the arguments can be assigned 'final' and not subject to change during the
lifespan of the method call.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] belugabehr commented on issue #911: HBASE-23379: Clean Up FSUtil getRegionLocalityMappingFromFS
belugabehr commented on issue #911: HBASE-23379: Clean Up FSUtil getRegionLocalityMappingFromFS URL: https://github.com/apache/hbase/pull/911#issuecomment-563399102 @HorizonNet Thanks for the review!! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #920: HBASE-23552 Format Javadocs on ITBLL
Apache-HBase commented on issue #920: HBASE-23552 Format Javadocs on ITBLL URL: https://github.com/apache/hbase/pull/920#issuecomment-563407955 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 26s | master passed | | +1 :green_heart: | compile | 0m 30s | master passed | | +1 :green_heart: | checkstyle | 0m 19s | master passed | | +1 :green_heart: | shadedjars | 4m 37s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 17s | master passed | | +0 :ok: | spotbugs | 4m 56s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 0s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 0s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed | | +1 :green_heart: | javac | 0m 30s | the patch passed | | -1 :x: | checkstyle | 0m 19s | hbase-it: The patch generated 3 new + 35 unchanged - 0 fixed = 38 total (was 35) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 46s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 56s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 15s | the patch passed | | +1 :green_heart: | findbugs | 0m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 51s | hbase-it in the patch passed. | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 45m 52s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-920/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/920 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 38be6894c7a4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-920/out/precommit/personality/provided.sh | | git revision | master / 82e155eb26 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-920/1/artifact/out/diff-checkstyle-hbase-it.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-920/1/testReport/ | | Max. process+thread count | 402 (vs. ulimit of 1) | | modules | C: hbase-it U: hbase-it | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-920/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] belugabehr commented on issue #910: HBASE-23378: Clean Up FSUtil setClusterId
belugabehr commented on issue #910: HBASE-23378: Clean Up FSUtil setClusterId URL: https://github.com/apache/hbase/pull/910#issuecomment-563408977 @HorizonNet When you are too helpful, you get asked to do more. :) Can you please also take a look at this PR as well? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] VladRodionov opened a new pull request #921: HBASE-22749: Distributed MOB compactions
VladRodionov opened a new pull request #921: HBASE-22749: Distributed MOB compactions URL: https://github.com/apache/hbase/pull/921 Finalizing HBASE-22749. Rebased patch against current HBase master, created new pull PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22749) Distributed MOB compactions
[ https://issues.apache.org/jira/browse/HBASE-22749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991906#comment-16991906 ] Vladimir Rodionov commented on HBASE-22749: --- Created new PR, old one was closed as obsolete. > Distributed MOB compactions > > > Key: HBASE-22749 > URL: https://issues.apache.org/jira/browse/HBASE-22749 > Project: HBase > Issue Type: New Feature > Components: mob >Reporter: Vladimir Rodionov >Assignee: Vladimir Rodionov >Priority: Major > Attachments: HBASE-22749-branch-2.2-v4.patch, > HBASE-22749-master-v1.patch, HBASE-22749-master-v2.patch, > HBASE-22749-master-v3.patch, HBASE-22749-master-v4.patch, > HBase-MOB-2.0-v3.0.pdf > > > There are several drawbacks in the original MOB 1.0 (Moderate Object > Storage) implementation, which can limit the adoption of the MOB feature: > # MOB compactions are executed in a Master as a chore, which limits > scalability because all I/O goes through a single HBase Master server. > # Yarn/Mapreduce framework is required to run MOB compactions in a scalable > way, but this won’t work in a stand-alone HBase cluster. > # Two separate compactors for MOB and for regular store files and their > interactions can result in a data loss (see HBASE-22075) > The design goals for MOB 2.0 were to provide 100% MOB 1.0 - compatible > implementation, which is free of the above drawbacks and can be used as a > drop in replacement in existing MOB deployments. So, these are design goals > of a MOB 2.0: > # Make MOB compactions scalable without relying on Yarn/Mapreduce framework > # Provide unified compactor for both MOB and regular store files > # Make it more robust especially w.r.t. to data losses. > # Simplify and reduce the overall MOB code. > # Provide 100% compatible implementation with MOB 1.0. > # No migration of data should be required between MOB 1.0 and MOB 2.0 - just > software upgrade. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] joshelser commented on a change in pull request #884: HBASE-23347 Allowable custom authentication methods for RPCs
joshelser commented on a change in pull request #884: HBASE-23347 Allowable custom authentication methods for RPCs URL: https://github.com/apache/hbase/pull/884#discussion_r355665727 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcServer.java ## @@ -55,43 +48,10 @@ private UserGroupInformation attemptingUser; // user name before auth Review comment: Inadvertently dropped it from delegation token auth'n. Reintroduced. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] busbey commented on issue #921: HBASE-22749: Distributed MOB compactions
busbey commented on issue #921: HBASE-22749: Distributed MOB compactions URL: https://github.com/apache/hbase/pull/921#issuecomment-563425479 just to be clear, this includes all the follow-on commits from #623 as well right? i.e. HBASE-23209, HBASE-23188, HBASE-23266, HBASE-23267, and HBASE-23189? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] joshelser commented on a change in pull request #884: HBASE-23347 Allowable custom authentication methods for RPCs
joshelser commented on a change in pull request #884: HBASE-23347 Allowable
custom authentication methods for RPCs
URL: https://github.com/apache/hbase/pull/884#discussion_r355671771
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/security/provider/SimpleSaslServerAuthenticationProvider.java
##
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.security.provider;
+
+import java.io.IOException;
+import java.util.Map;
+
+import javax.security.sasl.SaslServer;
+
+import org.apache.hadoop.hbase.HBaseInterfaceAudience;
+import org.apache.hadoop.security.UserGroupInformation;
+import org.apache.hadoop.security.token.SecretManager;
+import org.apache.hadoop.security.token.TokenIdentifier;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.yetus.audience.InterfaceStability;
+
+@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.AUTHENTICATION)
+@InterfaceStability.Evolving
+public class SimpleSaslServerAuthenticationProvider extends
SimpleSaslClientAuthenticationProvider
+implements SaslServerAuthenticationProvider {
+
+ @Override
+ public SaslServer createServer(SecretManager secretManager,
+ Map saslProps) throws IOException {
+return null;
Review comment:
Sure, that looks fine.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] VladRodionov commented on issue #921: HBASE-22749: Distributed MOB compactions
VladRodionov commented on issue #921: HBASE-22749: Distributed MOB compactions URL: https://github.com/apache/hbase/pull/921#issuecomment-563431337 Yes, it includes all from PR #623 + some fixes to make this patch mergeable with current HBase master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] HorizonNet commented on a change in pull request #911: HBASE-23379: Clean Up FSUtil getRegionLocalityMappingFromFS
HorizonNet commented on a change in pull request #911: HBASE-23379: Clean Up
FSUtil getRegionLocalityMappingFromFS
URL: https://github.com/apache/hbase/pull/911#discussion_r355677818
##
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
##
@@ -1708,44 +1702,36 @@ public boolean accept(Path path) {
FileStatus[] statusList = fs.globStatus(queryPath, pathFilter);
+if (LOG.isDebugEnabled()) {
+ LOG.debug("Query Path: {} ; # list of files: {}", queryPath,
Arrays.toString(statusList));
+}
+
if (null == statusList) {
return;
-} else {
- LOG.debug("Query Path: " + queryPath + " ; # list of files: " +
- statusList.length);
}
// lower the number of threads in case we have very few expected regions
-threadPoolSize = Math.min(threadPoolSize, statusList.length);
+final int finalThreadPoolSize = Math.min(threadPoolSize,
statusList.length);
Review comment:
Ah, haven't seen this one. Agree on the `final` topic. Can we revert this
one to `final int`? A better name would be good.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] HorizonNet commented on a change in pull request #911: HBASE-23379: Clean Up FSUtil getRegionLocalityMappingFromFS
HorizonNet commented on a change in pull request #911: HBASE-23379: Clean Up
FSUtil getRegionLocalityMappingFromFS
URL: https://github.com/apache/hbase/pull/911#discussion_r355678593
##
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
##
@@ -1708,44 +1701,36 @@ public boolean accept(Path path) {
FileStatus[] statusList = fs.globStatus(queryPath, pathFilter);
+if (LOG.isDebugEnabled()) {
+ LOG.debug("Query Path: {} ; # list of files: {}", queryPath,
Arrays.toString(statusList));
+}
+
if (null == statusList) {
return;
-} else {
- LOG.debug("Query Path: " + queryPath + " ; # list of files: " +
- statusList.length);
}
// lower the number of threads in case we have very few expected regions
threadPoolSize = Math.min(threadPoolSize, statusList.length);
// run in multiple threads
-ThreadPoolExecutor tpe = new ThreadPoolExecutor(threadPoolSize,
-threadPoolSize, 60, TimeUnit.SECONDS,
-new ArrayBlockingQueue<>(statusList.length),
-Threads.newDaemonThreadFactory("FSRegionQuery"));
+final ExecutorService tpe = Executors.newFixedThreadPool(threadPoolSize,
+ Threads.newDaemonThreadFactory("FSRegionQuery"));
try {
// ignore all file status items that are not of interest
for (FileStatus regionStatus : statusList) {
-if (null == regionStatus) {
+if (null == regionStatus || !regionStatus.isDirectory()) {
continue;
}
-if (!regionStatus.isDirectory()) {
- continue;
+final Path regionPath = regionStatus.getPath();
+if (null != regionPath) {
+ tpe.execute(new FSRegionScanner(fs, regionPath, null,
regionDegreeLocalityMapping));
Review comment:
Why passing `null` instead of `regionToBestLocalityRSMapping`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r355682676
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
##
@@ -86,74 +87,176 @@ void fix() throws IOException {
* If hole, it papers it over by adding a region in the filesystem and to
hbase:meta.
* Does not assign.
*/
- void fixHoles(CatalogJanitor.Report report) throws IOException {
-List> holes = report.getHoles();
+ void fixHoles(CatalogJanitor.Report report) {
+final List> holes = report.getHoles();
if (holes.isEmpty()) {
- LOG.debug("No holes.");
+ LOG.info("CatalogJanitor Report contains no holes to fix. Skipping.");
return;
}
-for (Pair p: holes) {
- RegionInfo ri = getHoleCover(p);
- if (ri == null) {
-continue;
- }
- Configuration configuration = this.masterServices.getConfiguration();
- HRegion.createRegionDir(configuration, ri,
FSUtils.getRootDir(configuration));
- // If an error here, then we'll have a region in the filesystem but not
- // in hbase:meta (if the below fails). Should be able to rerun the fix.
- // Add to hbase:meta and then update in-memory state so it knows of new
- // Region; addRegionToMeta adds region and adds a state column set to
CLOSED.
- MetaTableAccessor.addRegionToMeta(this.masterServices.getConnection(),
ri);
- this.masterServices.getAssignmentManager().getRegionStates().
- updateRegionState(ri, RegionState.State.CLOSED);
- LOG.info("Fixed hole by adding {} in CLOSED state; region NOT assigned
(assign to ONLINE).",
- ri);
-}
+
+LOG.info("Identified {} region holes to fix.", holes.size());
+
+final List newRegionInfos = mapHolesToRegionInfos(holes);
+final List newRegionDirectories =
+ createRegionDirectories(masterServices, newRegionInfos);
+final List newMetaEntries = createMetaEntries(masterServices,
newRegionDirectories);
+final List scheduledProcedures =
+
masterServices.getAssignmentManager().createAssignProceduresBestEffort(newMetaEntries);
+LOG.info(
+ "Scheduled {}/{} new regions for assignment.",
scheduledProcedures.size(), holes.size());
}
/**
- * @return Calculated RegionInfo that covers the hole hole
+ * Create a new {@link RegionInfo} corresponding to each provided "hole"
pair.
*/
- private RegionInfo getHoleCover(Pair hole) {
-RegionInfo holeCover = null;
-RegionInfo left = hole.getFirst();
-RegionInfo right = hole.getSecond();
+ private static List mapHolesToRegionInfos(
Review comment:
Good making it static.
We are creating RIs in here, no? Not 'map'ping holes to RIs? Gives
impression the RIs were there already rather than being created.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r355132830
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestAssignmentManager.java
##
@@ -171,17 +171,11 @@ public void testAssignAnAssignedRegion() throws
Exception {
// wait first assign
waitOnFuture(futureA);
am.getRegionStates().isRegionInState(hri, State.OPEN);
-// Second should be a noop. We should recognize region is already OPEN
internally
-// and skip out doing nothing.
-// wait second assign
-Future futureB = submitProcedure(createAssignProcedure(hri));
-waitOnFuture(futureB);
-am.getRegionStates().isRegionInState(hri, State.OPEN);
-// TODO: What else can we do to ensure just a noop.
+assertNull(createAssignProcedure(hri));
Review comment:
Good
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r355132615
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
##
@@ -86,74 +87,176 @@ void fix() throws IOException {
* If hole, it papers it over by adding a region in the filesystem and to
hbase:meta.
* Does not assign.
*/
- void fixHoles(CatalogJanitor.Report report) throws IOException {
-List> holes = report.getHoles();
+ void fixHoles(CatalogJanitor.Report report) {
+final List> holes = report.getHoles();
if (holes.isEmpty()) {
- LOG.debug("No holes.");
+ LOG.info("CatalogJanitor Report contains no holes to fix. Skipping.");
Review comment:
I like the improved log message but not change in log level. Ideally CJ
would output one summary line rather than one per feature.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r355680821
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
##
@@ -86,74 +87,176 @@ void fix() throws IOException {
* If hole, it papers it over by adding a region in the filesystem and to
hbase:meta.
* Does not assign.
*/
- void fixHoles(CatalogJanitor.Report report) throws IOException {
-List> holes = report.getHoles();
+ void fixHoles(CatalogJanitor.Report report) {
+final List> holes = report.getHoles();
if (holes.isEmpty()) {
- LOG.debug("No holes.");
+ LOG.info("CatalogJanitor Report contains no holes to fix. Skipping.");
return;
}
-for (Pair p: holes) {
- RegionInfo ri = getHoleCover(p);
- if (ri == null) {
-continue;
- }
- Configuration configuration = this.masterServices.getConfiguration();
- HRegion.createRegionDir(configuration, ri,
FSUtils.getRootDir(configuration));
- // If an error here, then we'll have a region in the filesystem but not
- // in hbase:meta (if the below fails). Should be able to rerun the fix.
- // Add to hbase:meta and then update in-memory state so it knows of new
- // Region; addRegionToMeta adds region and adds a state column set to
CLOSED.
- MetaTableAccessor.addRegionToMeta(this.masterServices.getConnection(),
ri);
- this.masterServices.getAssignmentManager().getRegionStates().
- updateRegionState(ri, RegionState.State.CLOSED);
- LOG.info("Fixed hole by adding {} in CLOSED state; region NOT assigned
(assign to ONLINE).",
- ri);
-}
+
+LOG.info("Identified {} region holes to fix.", holes.size());
Review comment:
Need this at INFO when we have another log line below at INFO on what we are
going to do?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r355683606
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
##
@@ -86,74 +87,176 @@ void fix() throws IOException {
* If hole, it papers it over by adding a region in the filesystem and to
hbase:meta.
* Does not assign.
*/
- void fixHoles(CatalogJanitor.Report report) throws IOException {
-List> holes = report.getHoles();
+ void fixHoles(CatalogJanitor.Report report) {
+final List> holes = report.getHoles();
if (holes.isEmpty()) {
- LOG.debug("No holes.");
+ LOG.info("CatalogJanitor Report contains no holes to fix. Skipping.");
return;
}
-for (Pair p: holes) {
- RegionInfo ri = getHoleCover(p);
- if (ri == null) {
-continue;
- }
- Configuration configuration = this.masterServices.getConfiguration();
- HRegion.createRegionDir(configuration, ri,
FSUtils.getRootDir(configuration));
- // If an error here, then we'll have a region in the filesystem but not
- // in hbase:meta (if the below fails). Should be able to rerun the fix.
- // Add to hbase:meta and then update in-memory state so it knows of new
- // Region; addRegionToMeta adds region and adds a state column set to
CLOSED.
- MetaTableAccessor.addRegionToMeta(this.masterServices.getConnection(),
ri);
- this.masterServices.getAssignmentManager().getRegionStates().
- updateRegionState(ri, RegionState.State.CLOSED);
- LOG.info("Fixed hole by adding {} in CLOSED state; region NOT assigned
(assign to ONLINE).",
- ri);
-}
+
+LOG.info("Identified {} region holes to fix.", holes.size());
+
+final List newRegionInfos = mapHolesToRegionInfos(holes);
+final List newRegionDirectories =
+ createRegionDirectories(masterServices, newRegionInfos);
+final List newMetaEntries = createMetaEntries(masterServices,
newRegionDirectories);
+final List scheduledProcedures =
+
masterServices.getAssignmentManager().createAssignProceduresBestEffort(newMetaEntries);
+LOG.info(
+ "Scheduled {}/{} new regions for assignment.",
scheduledProcedures.size(), holes.size());
}
/**
- * @return Calculated RegionInfo that covers the hole hole
+ * Create a new {@link RegionInfo} corresponding to each provided "hole"
pair.
*/
- private RegionInfo getHoleCover(Pair hole) {
-RegionInfo holeCover = null;
-RegionInfo left = hole.getFirst();
-RegionInfo right = hole.getSecond();
+ private static List mapHolesToRegionInfos(
+final List> holes) {
+final List newRegionInfos = holes.stream()
+ .map(MetaFixer::getHoleCover)
+ .filter(Optional::isPresent)
+ .map(Optional::get)
+ .collect(Collectors.toList());
+LOG.debug("Constructed {}/{} RegionInfo descriptors corresponding to
identified holes.",
+ newRegionInfos.size(), holes.size());
+return newRegionInfos;
+ }
+
+ /**
+ * @return Attempts to calculate a new {@link RegionInfo} that covers the
region range described
+ * in {@code hole}.
+ */
+ private static Optional getHoleCover(Pair hole) {
+final RegionInfo left = hole.getFirst();
+final RegionInfo right = hole.getSecond();
+
if (left.getTable().equals(right.getTable())) {
// Simple case.
if (Bytes.compareTo(left.getEndKey(), right.getStartKey()) >= 0) {
-LOG.warn("Skipping hole fix; left-side endKey is not less than
right-side startKey; " +
-"left=<{}>, right=<{}>", left, right);
-return holeCover;
- }
- holeCover = buildRegionInfo(left.getTable(), left.getEndKey(),
right.getStartKey());
-} else {
- boolean leftUndefined = left.equals(RegionInfo.UNDEFINED);
- boolean rightUnefined = right.equals(RegionInfo.UNDEFINED);
- boolean last = left.isLast();
- boolean first = right.isFirst();
- if (leftUndefined && rightUnefined) {
-LOG.warn("Skipping hole fix; both the hole left-side and right-side
RegionInfos are " +
-"UNDEFINED; left=<{}>, right=<{}>", left, right);
-return holeCover;
- }
- if (leftUndefined || last) {
-holeCover = buildRegionInfo(right.getTable(),
HConstants.EMPTY_START_ROW,
-right.getStartKey());
- } else if (rightUnefined || first) {
-holeCover = buildRegionInfo(left.getTable(), left.getEndKey(),
HConstants.EMPTY_END_ROW);
- } else {
-LOG.warn("Skipping hole fix; don't know what to do with left=<{}>,
right=<{}>",
-left, right);
-return holeCover;
+LOG.warn("Skipping hole fix; left-side endKey is not less than
right-side startKey;"
+ + " left=<{}>, right=<{}>", left, right);
+return Optional.empty();
}
+ return Optio
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r355132861
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestAssignmentManager.java
##
@@ -171,17 +171,11 @@ public void testAssignAnAssignedRegion() throws
Exception {
// wait first assign
waitOnFuture(futureA);
am.getRegionStates().isRegionInState(hri, State.OPEN);
-// Second should be a noop. We should recognize region is already OPEN
internally
-// and skip out doing nothing.
-// wait second assign
-Future futureB = submitProcedure(createAssignProcedure(hri));
-waitOnFuture(futureB);
-am.getRegionStates().isRegionInState(hri, State.OPEN);
-// TODO: What else can we do to ensure just a noop.
+assertNull(createAssignProcedure(hri));
// TODO: Though second assign is noop, it's considered success, can noop
be handled in a
// better way?
-assertEquals(assignSubmittedCount + 2,
assignProcMetrics.getSubmittedCounter().getCount());
+assertEquals(assignSubmittedCount + 1,
assignProcMetrics.getSubmittedCounter().getCount());
Review comment:
Why does the count change?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r355686750
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/AssignmentManager.java
##
@@ -724,24 +731,57 @@ static int compare(TransitRegionStateProcedure left,
TransitRegionStateProcedure
return RegionInfo.COMPARATOR.compare(left.getRegion(), right.getRegion());
}
- private TransitRegionStateProcedure createAssignProcedure(RegionStateNode
regionNode,
- ServerName targetServer, boolean override) {
-TransitRegionStateProcedure proc;
+ private TransitRegionStateProcedure createAssignProcedure(final RegionInfo
regionInfo)
+throws HBaseIOException {
+return createAssignProcedure(regionInfo, null);
+ }
+
+ private TransitRegionStateProcedure createAssignProcedure(final RegionInfo
regionInfo,
+final ServerName targetServer) throws HBaseIOException {
+// TODO: should we use getRegionStateNode?
+final RegionStateNode regionNode =
regionStates.getOrCreateRegionStateNode(regionInfo);
+return createAssignProcedure(regionNode, targetServer, false);
+ }
+
+ /**
+ * Queue a new {@link TransitRegionStateProcedure} representing this
assignment request.
+ * @param regionNode the region to assign
+ * @param targetServer the target server for this assignment (optional)
+ * @param override when {@code true}, remove any existing procedure found on
{@code regionNode}
+ */
+ private TransitRegionStateProcedure createAssignProcedure(final
RegionStateNode regionNode,
+final ServerName targetServer, final boolean override) throws
HBaseIOException {
+final TransitRegionStateProcedure proc;
regionNode.lock();
try {
- if(override && regionNode.getProcedure() != null) {
+ if (override && regionNode.getProcedure() != null) {
regionNode.unsetProcedure(regionNode.getProcedure());
}
assert regionNode.getProcedure() == null;
- proc = TransitRegionStateProcedure.assign(getProcedureEnvironment(),
-regionNode.getRegionInfo(), targetServer);
+ preTransitCheck(regionNode, STATES_EXPECTED_ON_ASSIGN);
Review comment:
So this is new. There are states taht need to be added to
STATES_EXPECTED_ON_ASSIGN given the errors you see?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r355684692
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
##
@@ -244,4 +347,47 @@ static boolean isOverlap(RegionInfo ri, Pair pair) {
}
return ri.isOverlap(pair.getFirst()) || ri.isOverlap(pair.getSecond());
}
+
+ /**
+ * A union over {@link L} and {@link R}.
+ */
+ private static class Either {
Review comment:
Nice.
Belongs in RegionInfo? Or you think only for use by fixer?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing URL: https://github.com/apache/hbase/pull/917#discussion_r355132809 ## File path: hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestAssignmentManagerBase.java ## @@ -26,7 +27,10 @@ import java.io.UncheckedIOException; import java.net.SocketTimeoutException; import java.util.Arrays; +import java.util.Collections; +import java.util.List; import java.util.NavigableMap; +import java.util.Optional; Review comment: Unused imports? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r355680221
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MetaFixer.java
##
@@ -86,74 +87,176 @@ void fix() throws IOException {
* If hole, it papers it over by adding a region in the filesystem and to
hbase:meta.
* Does not assign.
*/
- void fixHoles(CatalogJanitor.Report report) throws IOException {
-List> holes = report.getHoles();
+ void fixHoles(CatalogJanitor.Report report) {
+final List> holes = report.getHoles();
if (holes.isEmpty()) {
- LOG.debug("No holes.");
+ LOG.info("CatalogJanitor Report contains no holes to fix. Skipping.");
Review comment:
Why name the class in the log message? Isn't it present before the above log
txt in the logs already?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2]
`fixHoles` should queue assignment procedures for any regions its fixing
URL: https://github.com/apache/hbase/pull/917#discussion_r355133022
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/master/assignment/TestAMAssignWithRandExec.java
##
@@ -44,10 +46,15 @@ public void testAssignWithRandExec() throws Exception {
rsDispatcher.setMockRsExecutor(new RandRsExecutor());
// Loop a bunch of times so we hit various combos of exceptions.
+int submitCount = 0;
for (int i = 0; i < 10; i++) {
LOG.info("ROUND=" + i);
TransitRegionStateProcedure proc = createAssignProcedure(hri);
- waitOnFuture(submitProcedure(proc));
+ if (proc != null) {
+waitOnFuture(submitProcedure(proc));
+submitCount++;
+ }
}
+assertNotEquals("failed to create any procedures", 0, submitCount);
Review comment:
Good
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing
saintstack commented on a change in pull request #917: HBASE-23383 [hbck2] `fixHoles` should queue assignment procedures for any regions its fixing URL: https://github.com/apache/hbase/pull/917#discussion_r355133148 ## File path: hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/TruncateTableProcedure.java ## @@ -139,7 +140,10 @@ protected Flow executeFromState(final MasterProcedureEnv env, TruncateTableState break; case TRUNCATE_TABLE_ASSIGN_REGIONS: CreateTableProcedure.setEnablingState(env, getTableName()); - addChildProcedure(env.getAssignmentManager().createRoundRobinAssignProcedures(regions)); + final TransitRegionStateProcedure[] assignments = env.getAssignmentManager() +.createRoundRobinAssignProcedures(regions) +.toArray(new TransitRegionStateProcedure[0]); + addChildProcedure(assignments); Review comment: Old way was better? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] HorizonNet commented on a change in pull request #920: HBASE-23552 Format Javadocs on ITBLL
HorizonNet commented on a change in pull request #920: HBASE-23552 Format Javadocs on ITBLL URL: https://github.com/apache/hbase/pull/920#discussion_r355693998 ## File path: hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java ## @@ -122,73 +122,100 @@ import org.apache.hbase.thirdparty.org.apache.commons.cli.ParseException; /** + * * This is an integration test borrowed from goraci, written by Keith Turner, * which is in turn inspired by the Accumulo test called continous ingest (ci). * The original source code can be found here: - * https://github.com/keith-turner/goraci - * https://github.com/enis/goraci/ - * + * + * https://github.com/keith-turner/goraci + * https://github.com/enis/goraci/ + * + * + * * Apache Accumulo [0] has a simple test suite that verifies that data is not * lost at scale. This test suite is called continuous ingest. This test runs * many ingest clients that continually create linked lists containing 25 * million nodes. At some point the clients are stopped and a map reduce job is - * run to ensure no linked list has a hole. A hole indicates data was lost.·· - * + * run to ensure no linked list has a hole. A hole indicates data was lost. + * + * * The nodes in the linked list are random. This causes each linked list to * spread across the table. Therefore if one part of a table loses data, then it * will be detected by references in another part of the table. - * - * THE ANATOMY OF THE TEST + * + * + * THE ANATOMY OF THE TEST * * Below is rough sketch of how data is written. For specific details look at * the Generator code. - * - * 1 Write out 1 million nodes· 2 Flush the client· 3 Write out 1 million that - * reference previous million· 4 If this is the 25th set of 1 million nodes, - * then update 1st set of million to point to last· 5 goto 1 - * + * + * + * + * Write out 1 million nodes + * Flush the client + * Write out 1 million that reference previous million + * If this is the 25th set of 1 million nodes, then update 1st set of + * million to point to last + * goto 1 + * + * + * * The key is that nodes only reference flushed nodes. Therefore a node should * never reference a missing node, even if the ingest client is killed at any * point in time. - * + * + * * When running this test suite w/ Accumulo there is a script running in * parallel called the Aggitator that randomly and continuously kills server - * processes.·· The outcome was that many data loss bugs were found in Accumulo - * by doing this.· This test suite can also help find bugs that impact uptime - * and stability when· run for days or weeks.·· - * - * This test suite consists the following· - a few Java programs· - a little - * helper script to run the java programs - a maven script to build it.·· - * + * processes. The outcome was that many data loss bugs were found in Accumulo + * by doing this. This test suite can also help find bugs that impact uptime + * and stability when run for days or weeks. + * + * + * This test suite consists the following + * + * a few Java programs + * a little helper script to run the java programs + * a maven script to build it + * Review comment: A closing `ul` seems to be missing, leading to a wrong indentation afterwards in the output. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23380) General Cleanup of FSUtil
[
https://issues.apache.org/jira/browse/HBASE-23380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991994#comment-16991994
]
HBase QA commented on HBASE-23380:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
6s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
44s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
16s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} hbase-server: The patch generated 0 new + 66
unchanged - 8 fixed = 66 total (was 74) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
39s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
15m 43s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}156m
50s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}216m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-912/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/912 |
| JIRA Issue | HBASE-23380 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux 7a71b0a72cae 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-912/out/precommit/personality/provided.sh
|
| git revision | master / 82e155eb26 |
| D
[GitHub] [hbase] Apache-HBase commented on issue #912: HBASE-23380: General Cleanup of FSUtil
Apache-HBase commented on issue #912: HBASE-23380: General Cleanup of FSUtil URL: https://github.com/apache/hbase/pull/912#issuecomment-563468920 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 2m 6s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 52s | master passed | | +1 :green_heart: | compile | 0m 56s | master passed | | +1 :green_heart: | checkstyle | 1m 23s | master passed | | +1 :green_heart: | shadedjars | 4m 44s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | | +0 :ok: | spotbugs | 4m 16s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 14s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 58s | the patch passed | | +1 :green_heart: | compile | 0m 57s | the patch passed | | +1 :green_heart: | javac | 0m 57s | the patch passed | | +1 :green_heart: | checkstyle | 1m 19s | hbase-server: The patch generated 0 new + 66 unchanged - 8 fixed = 66 total (was 74) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 43s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 34s | the patch passed | | +1 :green_heart: | findbugs | 4m 9s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 156m 50s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 216m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-912/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/912 | | JIRA Issue | HBASE-23380 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 7a71b0a72cae 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-912/out/precommit/personality/provided.sh | | git revision | master / 82e155eb26 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-912/3/testReport/ | | Max. process+thread count | 4466 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-912/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] belugabehr commented on a change in pull request #911: HBASE-23379: Clean Up FSUtil getRegionLocalityMappingFromFS
belugabehr commented on a change in pull request #911: HBASE-23379: Clean Up
FSUtil getRegionLocalityMappingFromFS
URL: https://github.com/apache/hbase/pull/911#discussion_r355728658
##
File path: hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
##
@@ -1708,44 +1701,36 @@ public boolean accept(Path path) {
FileStatus[] statusList = fs.globStatus(queryPath, pathFilter);
+if (LOG.isDebugEnabled()) {
+ LOG.debug("Query Path: {} ; # list of files: {}", queryPath,
Arrays.toString(statusList));
+}
+
if (null == statusList) {
return;
-} else {
- LOG.debug("Query Path: " + queryPath + " ; # list of files: " +
- statusList.length);
}
// lower the number of threads in case we have very few expected regions
threadPoolSize = Math.min(threadPoolSize, statusList.length);
// run in multiple threads
-ThreadPoolExecutor tpe = new ThreadPoolExecutor(threadPoolSize,
-threadPoolSize, 60, TimeUnit.SECONDS,
-new ArrayBlockingQueue<>(statusList.length),
-Threads.newDaemonThreadFactory("FSRegionQuery"));
+final ExecutorService tpe = Executors.newFixedThreadPool(threadPoolSize,
+ Threads.newDaemonThreadFactory("FSRegionQuery"));
try {
// ignore all file status items that are not of interest
for (FileStatus regionStatus : statusList) {
-if (null == regionStatus) {
+if (null == regionStatus || !regionStatus.isDirectory()) {
continue;
}
-if (!regionStatus.isDirectory()) {
- continue;
+final Path regionPath = regionStatus.getPath();
+if (null != regionPath) {
+ tpe.execute(new FSRegionScanner(fs, regionPath, null,
regionDegreeLocalityMapping));
Review comment:
I removed `regionToBestLocalityRSMapping` from the method signature. This
is a private method and the one method that calls this method passes in a
hard-coded 'null' value. So, to simplify, I just cut out the middleman and
pass in 'null' directly.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (HBASE-23550) FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length (8) exceed the capacity of the array: 41
[
https://issues.apache.org/jira/browse/HBASE-23550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser resolved HBASE-23550.
Resolution: Invalid
You're running into PHOENIX-5455. You need a newer version of Phoenix that can
handle the KeyValue changes referenced by HBASE-22034
> FSHLog crash with java.lang.IllegalArgumentException: offset (40) + length
> (8) exceed the capacity of the array: 41
> ---
>
> Key: HBASE-23550
> URL: https://issues.apache.org/jira/browse/HBASE-23550
> Project: HBase
> Issue Type: Bug
>Affects Versions: 1.4.10
>Reporter: Alex Batyrshin
>Priority: Major
>
> We are using HBase-1.4.10 with Phoenix-4.14.2, so config file looks like:
> {code:java}
>
> hbase.regionserver.wal.codec
>
> org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec
> {code}
> Trying to dump wal
> {code:java}
> $ ./bin/hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump
> file:///tmp/hbase06-wal
> ...
> Sequence=146194465 , region=51e4797be6c6602054d4aaeb94e01526 at write
> timestamp=Mon Dec 09 18:46:30 MSK 2019
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:_0
> cell total size sum: 120
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:apd
> cell total size sum: 128
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:st
> cell total size sum: 120
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:pt
> cell total size sum: 120
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:gt
> cell total size sum: 128
> row=\x157809008119\x00\xC7\x02:;d\x02\x15\x46218513?jTGJ=u,
> column=d:d:o
> cell total size sum: 128
> edit heap size: 784
> position: 49623
> Exception in thread "main" java.lang.IllegalArgumentException: offset (40) +
> length (8) exceed the capacity of the array: 41
> at
> org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:779)
> at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:753)
> at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:738)
> at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1569)
> at org.apache.hadoop.hbase.KeyValue.getTimestamp(KeyValue.java:1560)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.toStringMap(WALPrettyPrinter.java:352)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.processFile(WALPrettyPrinter.java:297)
> at
> org.apache.hadoop.hbase.wal.WALPrettyPrinter.run(WALPrettyPrinter.java:438)
> at
> org.apache.hadoop.hbase.regionserver.wal.FSHLog.main(FSHLog.java:2028){code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] ndimiduk commented on a change in pull request #920: HBASE-23552 Format Javadocs on ITBLL
ndimiduk commented on a change in pull request #920: HBASE-23552 Format Javadocs on ITBLL URL: https://github.com/apache/hbase/pull/920#discussion_r355736075 ## File path: hbase-it/src/test/java/org/apache/hadoop/hbase/test/IntegrationTestBigLinkedList.java ## @@ -122,73 +122,100 @@ import org.apache.hbase.thirdparty.org.apache.commons.cli.ParseException; /** + * * This is an integration test borrowed from goraci, written by Keith Turner, * which is in turn inspired by the Accumulo test called continous ingest (ci). * The original source code can be found here: - * https://github.com/keith-turner/goraci - * https://github.com/enis/goraci/ - * + * + * https://github.com/keith-turner/goraci + * https://github.com/enis/goraci/ + * + * + * * Apache Accumulo [0] has a simple test suite that verifies that data is not * lost at scale. This test suite is called continuous ingest. This test runs * many ingest clients that continually create linked lists containing 25 * million nodes. At some point the clients are stopped and a map reduce job is - * run to ensure no linked list has a hole. A hole indicates data was lost.·· - * + * run to ensure no linked list has a hole. A hole indicates data was lost. + * + * * The nodes in the linked list are random. This causes each linked list to * spread across the table. Therefore if one part of a table loses data, then it * will be detected by references in another part of the table. - * - * THE ANATOMY OF THE TEST + * + * + * THE ANATOMY OF THE TEST * * Below is rough sketch of how data is written. For specific details look at * the Generator code. - * - * 1 Write out 1 million nodes· 2 Flush the client· 3 Write out 1 million that - * reference previous million· 4 If this is the 25th set of 1 million nodes, - * then update 1st set of million to point to last· 5 goto 1 - * + * + * + * + * Write out 1 million nodes + * Flush the client + * Write out 1 million that reference previous million + * If this is the 25th set of 1 million nodes, then update 1st set of + * million to point to last + * goto 1 + * + * + * * The key is that nodes only reference flushed nodes. Therefore a node should * never reference a missing node, even if the ingest client is killed at any * point in time. - * + * + * * When running this test suite w/ Accumulo there is a script running in * parallel called the Aggitator that randomly and continuously kills server - * processes.·· The outcome was that many data loss bugs were found in Accumulo - * by doing this.· This test suite can also help find bugs that impact uptime - * and stability when· run for days or weeks.·· - * - * This test suite consists the following· - a few Java programs· - a little - * helper script to run the java programs - a maven script to build it.·· - * + * processes. The outcome was that many data loss bugs were found in Accumulo + * by doing this. This test suite can also help find bugs that impact uptime + * and stability when run for days or weeks. + * + * + * This test suite consists the following + * + * a few Java programs + * a little helper script to run the java programs + * a maven script to build it + * Review comment: Yes you're right. Looks like checkstyle actually catches these issues, nice! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #911: HBASE-23379: Clean Up FSUtil getRegionLocalityMappingFromFS
Apache-HBase commented on issue #911: HBASE-23379: Clean Up FSUtil getRegionLocalityMappingFromFS URL: https://github.com/apache/hbase/pull/911#issuecomment-563483173 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | -0 :warning: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 26s | master passed | | +1 :green_heart: | compile | 0m 55s | master passed | | +1 :green_heart: | checkstyle | 1m 18s | master passed | | +1 :green_heart: | shadedjars | 4m 32s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 39s | master passed | | +0 :ok: | spotbugs | 4m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 58s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 58s | the patch passed | | +1 :green_heart: | compile | 0m 55s | the patch passed | | +1 :green_heart: | javac | 0m 55s | the patch passed | | +1 :green_heart: | checkstyle | 1m 21s | the patch passed | | +1 :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 39s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 15s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | | +1 :green_heart: | findbugs | 4m 20s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 154m 7s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 212m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-911/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/911 | | JIRA Issue | HBASE-23379 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c7642b41e5cf 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-911/out/precommit/personality/provided.sh | | git revision | master / 82e155eb26 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-911/2/testReport/ | | Max. process+thread count | 4061 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-911/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23379) Clean Up FSUtil getRegionLocalityMappingFromFS
[
https://issues.apache.org/jira/browse/HBASE-23379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16992019#comment-16992019
]
HBase QA commented on HBASE-23379:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
32s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
1s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
39s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 15s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}154m
7s{color} | {color:green} hbase-server in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}212m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.5 Server=19.03.5 base:
https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-911/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/911 |
| JIRA Issue | HBASE-23379 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs
shadedjars hadoopcheck hbaseanti checkstyle compile |
| uname | Linux c7642b41e5cf 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-911/out/precommit/personality/provided.sh
|
| git revision | master / 82e155eb26 |
| Default Java | 1.8.0_181 |
| Test Results |
https://builds.apache.or
[jira] [Commented] (HBASE-18095) Provide an option for clients to find the server hosting META that does not involve the ZooKeeper client
[
https://issues.apache.org/jira/browse/HBASE-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16992031#comment-16992031
]
Hudson commented on HBASE-18095:
Results for branch HBASE-18095/client-locate-meta-no-zookeeper
[build #5 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/5/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/5//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/5//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-18095%252Fclient-locate-meta-no-zookeeper/5//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Provide an option for clients to find the server hosting META that does not
> involve the ZooKeeper client
>
>
> Key: HBASE-18095
> URL: https://issues.apache.org/jira/browse/HBASE-18095
> Project: HBase
> Issue Type: New Feature
> Components: Client
>Reporter: Andrew Kyle Purtell
>Assignee: Bharath Vissapragada
>Priority: Major
> Attachments: HBASE-18095.master-v1.patch, HBASE-18095.master-v2.patch
>
>
> Clients are required to connect to ZooKeeper to find the location of the
> regionserver hosting the meta table region. Site configuration provides the
> client a list of ZK quorum peers and the client uses an embedded ZK client to
> query meta location. Timeouts and retry behavior of this embedded ZK client
> are managed orthogonally to HBase layer settings and in some cases the ZK
> cannot manage what in theory the HBase client can, i.e. fail fast upon outage
> or network partition.
> We should consider new configuration settings that provide a list of
> well-known master and backup master locations, and with this information the
> client can contact any of the master processes directly. Any master in either
> active or passive state will track meta location and respond to requests for
> it with its cached last known location. If this location is stale, the client
> can ask again with a flag set that requests the master refresh its location
> cache and return the up-to-date location. Every client interaction with the
> cluster thus uses only HBase RPC as transport, with appropriate settings
> applied to the connection. The configuration toggle that enables this
> alternative meta location lookup should be false by default.
> This removes the requirement that HBase clients embed the ZK client and
> contact the ZK service directly at the beginning of the connection lifecycle.
> This has several benefits. ZK service need not be exposed to clients, and
> their potential abuse, yet no benefit ZK provides the HBase server cluster is
> compromised. Normalizing HBase client and ZK client timeout settings and
> retry behavior - in some cases, impossible, i.e. for fail-fast - is no longer
> necessary.
> And, from [~ghelmling]: There is an additional complication here for
> token-based authentication. When a delegation token is used for SASL
> authentication, the client uses the cluster ID obtained from Zookeeper to
> select the token identifier to use. So there would also need to be some
> Zookeeper-less, unauthenticated way to obtain the cluster ID as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #884: HBASE-23347 Allowable custom authentication methods for RPCs
Apache-HBase commented on issue #884: HBASE-23347 Allowable custom authentication methods for RPCs URL: https://github.com/apache/hbase/pull/884#issuecomment-563492993 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 8s | master passed | | +1 :green_heart: | compile | 1m 45s | master passed | | +1 :green_heart: | checkstyle | 2m 18s | master passed | | +1 :green_heart: | shadedjars | 4m 35s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 22s | master passed | | +0 :ok: | spotbugs | 4m 5s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 5m 56s | master passed | | -0 :warning: | patch | 4m 28s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 58s | the patch passed | | +1 :green_heart: | compile | 1m 44s | the patch passed | | +1 :green_heart: | javac | 1m 44s | the patch passed | | +1 :green_heart: | checkstyle | 0m 26s | The patch passed checkstyle in hbase-common | | +1 :green_heart: | checkstyle | 0m 34s | hbase-client: The patch generated 0 new + 22 unchanged - 10 fixed = 22 total (was 32) | | +1 :green_heart: | checkstyle | 1m 19s | hbase-server: The patch generated 0 new + 26 unchanged - 3 fixed = 26 total (was 29) | | -1 :x: | whitespace | 0m 0s | The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | xml | 0m 1s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedjars | 4m 33s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 41s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 1m 19s | the patch passed | | +1 :green_heart: | findbugs | 6m 19s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 8s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 55s | hbase-client in the patch passed. | | -1 :x: | unit | 154m 32s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 1m 39s | The patch does not generate ASF License warnings. | | | | 227m 47s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-884/10/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/884 | | Optional Tests | dupname asflicense markdownlint javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile xml | | uname | Linux 46e92cfb39f0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-884/out/precommit/personality/provided.sh | | git revision | master / 82e155eb26 | | Default Java | 1.8.0_181 | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-884/10/artifact/out/whitespace-eol.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-884/10/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-884/10/testReport/ | | Max. process+thread count | 4830 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-884/10/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatic
[jira] [Commented] (HBASE-22749) Distributed MOB compactions
[
https://issues.apache.org/jira/browse/HBASE-22749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16992038#comment-16992038
]
HBase QA commented on HBASE-22749:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 18 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
37s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 4m
21s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
15s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
29s{color} | {color:red} hbase-server: The patch generated 78 new + 326
unchanged - 47 fixed = 404 total (was 373) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
17s{color} | {color:red} hbase-it: The patch generated 2 new + 0 unchanged - 0
fixed = 2 total (was 0) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 43 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
40s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
16m 6s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
43s{color} | {color:red} hbase-server generated 2 new + 0 unchanged - 0 fixed =
2 total (was 0) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}154m 16s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
12s{color} | {color:green} hbase-it in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
6s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}218m 24s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hbase-server |
| | Possible null pointer dereference of mobRefData in
org.apache.hadoop.hbase.master.MobFileCleanerChore.cleanupObsoleteMobFiles(C
[GitHub] [hbase] Apache-HBase commented on issue #921: HBASE-22749: Distributed MOB compactions
Apache-HBase commented on issue #921: HBASE-22749: Distributed MOB compactions URL: https://github.com/apache/hbase/pull/921#issuecomment-563493055 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 18 new or modified test files. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 20s | master passed | | +1 :green_heart: | compile | 1m 25s | master passed | | +1 :green_heart: | checkstyle | 1m 51s | master passed | | +1 :green_heart: | shadedjars | 4m 37s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | master passed | | +0 :ok: | spotbugs | 4m 21s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 4m 19s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 5m 22s | the patch passed | | +1 :green_heart: | compile | 1m 27s | the patch passed | | +1 :green_heart: | javac | 1m 27s | the patch passed | | -1 :x: | checkstyle | 1m 29s | hbase-server: The patch generated 78 new + 326 unchanged - 47 fixed = 404 total (was 373) | | -1 :x: | checkstyle | 0m 17s | hbase-it: The patch generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | | -1 :x: | whitespace | 0m 0s | The patch 43 line(s) with tabs. | | +1 :green_heart: | shadedjars | 4m 40s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 16m 6s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 49s | the patch passed | | -1 :x: | findbugs | 4m 43s | hbase-server generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | ||| _ Other Tests _ | | -1 :x: | unit | 154m 16s | hbase-server in the patch failed. | | +1 :green_heart: | unit | 1m 12s | hbase-it in the patch passed. | | +1 :green_heart: | asflicense | 1m 6s | The patch does not generate ASF License warnings. | | | | 218m 24s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-server | | | Possible null pointer dereference of mobRefData in org.apache.hadoop.hbase.master.MobFileCleanerChore.cleanupObsoleteMobFiles(Configuration, TableName) Dereferenced at MobFileCleanerChore.java:mobRefData in org.apache.hadoop.hbase.master.MobFileCleanerChore.cleanupObsoleteMobFiles(Configuration, TableName) Dereferenced at MobFileCleanerChore.java:[line 200] | | | Nullcheck of mobFileWriter at line 563 of value previously dereferenced in org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.commitOrAbortMobWriter(StoreFileWriter, long, long, boolean) At DefaultMobStoreCompactor.java:563 of value previously dereferenced in org.apache.hadoop.hbase.mob.DefaultMobStoreCompactor.commitOrAbortMobWriter(StoreFileWriter, long, long, boolean) At DefaultMobStoreCompactor.java:[line 561] | | Failed junit tests | hadoop.hbase.regionserver.TestMobStoreCompaction | | | hadoop.hbase.regionserver.TestRegionServerMetrics | | | hadoop.hbase.client.TestAsyncRegionAdminApi | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/921 | | JIRA Issue | HBASE-22749 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 27f2548fbfed 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-921/out/precommit/personality/provided.sh | | git revision | master / 82e155eb26 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/diff-checkstyle-hbase-server.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-921/1/artifact/out/diff-checkstyle-hbase-it.txt | | w
[GitHub] [hbase] Apache-HBase commented on issue #920: HBASE-23552 Format Javadocs on ITBLL
Apache-HBase commented on issue #920: HBASE-23552 Format Javadocs on ITBLL URL: https://github.com/apache/hbase/pull/920#issuecomment-563496564 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 58s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 31s | master passed | | +1 :green_heart: | compile | 0m 29s | master passed | | +1 :green_heart: | checkstyle | 0m 20s | master passed | | +1 :green_heart: | shadedjars | 4m 38s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 16s | master passed | | +0 :ok: | spotbugs | 4m 57s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 0m 0s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 3s | the patch passed | | +1 :green_heart: | compile | 0m 30s | the patch passed | | +1 :green_heart: | javac | 0m 30s | the patch passed | | +1 :green_heart: | checkstyle | 0m 19s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 4m 37s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 15m 40s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | | +1 :green_heart: | findbugs | 0m 0s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 52s | hbase-it in the patch passed. | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 47m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-920/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/920 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 785afe5eb990 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-920/out/precommit/personality/provided.sh | | git revision | master / 82e155eb26 | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-920/2/testReport/ | | Max. process+thread count | 405 (vs. ulimit of 1) | | modules | C: hbase-it U: hbase-it | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-920/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) | | Powered by | Apache Yetus 0.11.1 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Reopened] (HBASE-23085) Network and Data related Actions
[ https://issues.apache.org/jira/browse/HBASE-23085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Kyle Purtell reopened HBASE-23085: - This commit has a terminology problem. The universal technical term for network packet is packet, not "package". > Network and Data related Actions > > > Key: HBASE-23085 > URL: https://issues.apache.org/jira/browse/HBASE-23085 > Project: HBase > Issue Type: Sub-task > Components: integration tests >Reporter: Szabolcs Bukros >Assignee: Szabolcs Bukros >Priority: Minor > Fix For: 3.0.0, 2.3.0, 2.2.3 > > > Add additional actions to: > * manipulate network packages with tc (reorder, loose,...) > * add CPU load > * fill the disk > * corrupt or delete regionserver data files > Create new monkey factories for the new actions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
