[jira] [Commented] (HBASE-22699) refactor isMetaClearingException
[ https://issues.apache.org/jira/browse/HBASE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939849#comment-16939849 ] Junhong Xu commented on HBASE-22699: ok, And one more thing， should the RegionOpeningException clear the meta cache? For now it is. > refactor isMetaClearingException > > > Key: HBASE-22699 > URL: https://issues.apache.org/jira/browse/HBASE-22699 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Junhong Xu >Assignee: Junhong Xu >Priority: Minor > Fix For: 3.0.0, 2.3.0, 2.2.2 > > > It is not so readable -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #667: HBASE-23055 Alter hbase:meta
Apache-HBase commented on issue #667: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/667#issuecomment-536152992 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 11 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 34s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 54s | master passed | | :green_heart: | compile | 2m 2s | master passed | | :green_heart: | checkstyle | 2m 47s | master passed | | :green_heart: | shadedjars | 4m 57s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 32s | master passed | | :blue_heart: | spotbugs | 4m 17s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 6m 46s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 14s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 24s | the patch passed | | :green_heart: | compile | 2m 6s | the patch passed | | :green_heart: | javac | 2m 6s | the patch passed | | :green_heart: | checkstyle | 0m 27s | The patch passed checkstyle in hbase-common | | :green_heart: | checkstyle | 0m 35s | hbase-client: The patch generated 0 new + 97 unchanged - 4 fixed = 97 total (was 101) | | :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-zookeeper | | :green_heart: | checkstyle | 1m 35s | hbase-server: The patch generated 0 new + 444 unchanged - 21 fixed = 444 total (was 465) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 5m 1s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 17m 14s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 30s | the patch passed | | :green_heart: | findbugs | 7m 25s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 2m 55s | hbase-common in the patch passed. | | :green_heart: | unit | 1m 44s | hbase-client in the patch passed. | | :green_heart: | unit | 0m 48s | hbase-zookeeper in the patch passed. | | :broken_heart: | unit | 236m 29s | hbase-server in the patch failed. | | :green_heart: | asflicense | 1m 36s | The patch does not generate ASF License warnings. | | | | 318m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-667/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/667 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d091e4973aad 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-667/out/precommit/personality/provided.sh | | git revision | master / b45c0d0efa | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-667/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-667/2/testReport/ | | Max. process+thread count | 4824 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-zookeeper hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-667/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22991) Release 1.4.11
[ https://issues.apache.org/jira/browse/HBASE-22991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939843#comment-16939843 ] Sean Busbey commented on HBASE-22991: - created HBASE-23086 to track the TestShell failure > Release 1.4.11 > -- > > Key: HBASE-22991 > URL: https://issues.apache.org/jira/browse/HBASE-22991 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.4.11 > > Attachments: Flaky_20Test_20Report.zip > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-23086) TestShell failing on branch-1 and branch-1.4
Sean Busbey created HBASE-23086: --- Summary: TestShell failing on branch-1 and branch-1.4 Key: HBASE-23086 URL: https://issues.apache.org/jira/browse/HBASE-23086 Project: HBase Issue Type: Bug Components: shell Affects Versions: 1.5.0, 1.4.11 Reporter: Sean Busbey Fix For: 1.5.0, 1.4.11 here's a failed run from branch-1.4 [https://builds.apache.org/job/HBase-Flaky-Tests/job/branch-1.4/3958/testReport/junit/org.apache.hadoop.hbase.client/TestShell/testRunShellTests/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22991) Release 1.4.11
[ https://issues.apache.org/jira/browse/HBASE-22991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939840#comment-16939840 ] Sean Busbey commented on HBASE-22991: - [the flaky test list shows TestShell is failing|https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/job/branch-1.4/489/artifact/dashboard.html] > Release 1.4.11 > -- > > Key: HBASE-22991 > URL: https://issues.apache.org/jira/browse/HBASE-22991 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.4.11 > > Attachments: Flaky_20Test_20Report.zip > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23012) Release 1.3.6
[ https://issues.apache.org/jira/browse/HBASE-23012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939839#comment-16939839 ] Sean Busbey commented on HBASE-23012: - Next I checked the untrusted test list for the branch to make sure nothing was consistently failing (and found out something is :( ) [https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests/] > Release 1.3.6 > - > > Key: HBASE-23012 > URL: https://issues.apache.org/jira/browse/HBASE-23012 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sakthi >Assignee: Sakthi >Priority: Major > Fix For: 1.3.6 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-23012) Release 1.3.6
[
https://issues.apache.org/jira/browse/HBASE-23012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939838#comment-16939838
]
Sean Busbey commented on HBASE-23012:
-
Here's how I work through comparing changes logged in git and those accounted
for in jira as of the 1.4.11 RCs. The goal is for jira to be the source of
truth so that we can use it to generate release notes. HBASE-14025 has steps
for doing the comparison for a new minor release. This is specific to a
maintenance release.
1. Make a text file with the list of jiras in the fix version
- Browse to project home -> versions -> 1.4.11 -> Release Notes -> Configure
Release Notes -> format: text -> Create (leads to these [text release notes for
1.4.11|https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345617=Text=12310753=Create]
- the text box at the bottom will have the contents to copy
- filter to just have jira keys (this example is OS X specific)
{code}
pbpaste | grep -o -E "\[HBASE-[0-9]*\]" | grep -o -E "HBASE-[0-9]*" | sort -u >
~/tmp/hbase-release-process/CHANGES_1.4.11_jira.txt
{code}
2. Make a test file with the list of jiras in git commit messages
{code}
$ git checkout branch-1.4
$ git log --oneline rel/1.4.10...HEAD | grep -o -E '^[^ ]* HBASE-[0-9]* ' |
grep -o -E 'HBASE-[0-9]*' | sort -u >
~/tmp/hbase-release-process/CHANGES_1.4.11_git.txt
{code}
3. use the two files to list those things in jira but not git
{code}
$ comm -1 -3 ~/tmp/hbase-release-process/CHANGES_1.4.11_git.txt
~/tmp/hbase-release-process/CHANGES_1.4.11_jira.txt
{code}
4. use the two files to list those things in git but not in jira
{code}
$ comm -2 -3 ~/tmp/hbase-release-process/CHANGES_1.4.11_git.txt
~/tmp/hbase-release-process/CHANGES_1.4.11_jira.txt
{code}
> Release 1.3.6
> -
>
> Key: HBASE-23012
> URL: https://issues.apache.org/jira/browse/HBASE-23012
> Project: HBase
> Issue Type: Task
> Components: community
>Reporter: Sakthi
>Assignee: Sakthi
>Priority: Major
> Fix For: 1.3.6
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22991) Release 1.4.11
[ https://issues.apache.org/jira/browse/HBASE-22991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939834#comment-16939834 ] Sean Busbey commented on HBASE-22991: - compared git and jira, made some changes. h5. Things in jira, but not in git actually 1.4.10 * HBASE-21920 * HBASE-22540 fixed in branch-1 and not 1.4.z * HBASE-22495 duplicate with a fix version * HBASE-22797 * HBASE-22901 h5. in git but not jira mostly back ports updated jira to set 1.4.11 fix version * HBASE-22605 * HBASE-22629 * HBASE-22804 * HBASE-23023 > Release 1.4.11 > -- > > Key: HBASE-22991 > URL: https://issues.apache.org/jira/browse/HBASE-22991 > Project: HBase > Issue Type: Task > Components: community >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 1.4.11 > > Attachments: Flaky_20Test_20Report.zip > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23023) upgrade shellcheck used to test in nightly and precommit
[ https://issues.apache.org/jira/browse/HBASE-23023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey resolved HBASE-23023. - Fix Version/s: 2.2.2 2.1.7 1.4.11 1.3.6 2.3.0 1.5.0 3.0.0 Resolution: Fixed > upgrade shellcheck used to test in nightly and precommit > > > Key: HBASE-23023 > URL: https://issues.apache.org/jira/browse/HBASE-23023 > Project: HBase > Issue Type: Task > Components: build >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Major > Fix For: 3.0.0, 1.5.0, 2.3.0, 1.3.6, 1.4.11, 2.1.7, 2.2.2 > > > all our docker images are using an out of date version of shellcheck. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22804) Provide an API to get list of successful regions and total expected regions in Canary
[ https://issues.apache.org/jira/browse/HBASE-22804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-22804: Fix Version/s: (was: 1.4.12) 1.4.11 > Provide an API to get list of successful regions and total expected regions > in Canary > - > > Key: HBASE-22804 > URL: https://issues.apache.org/jira/browse/HBASE-22804 > Project: HBase > Issue Type: Improvement > Components: canary >Affects Versions: 3.0.0, 1.3.0, 1.4.0, 1.5.0, 2.0.0, 2.1.5, 2.2.1 >Reporter: Caroline >Assignee: Caroline >Priority: Minor > Labels: Canary > Fix For: 3.0.0, 1.5.0, 2.3.1, 1.3.6, 1.4.11, 2.1.7, 2.2.2 > > Attachments: HBASE-22804.branch-1.001.patch, > HBASE-22804.branch-1.002.patch, HBASE-22804.branch-1.003.patch, > HBASE-22804.branch-1.004.patch, HBASE-22804.branch-1.005.patch, > HBASE-22804.branch-1.006.patch, HBASE-22804.branch-1.007.patch, > HBASE-22804.branch-1.008.patch, HBASE-22804.branch-1.009.patch, > HBASE-22804.branch-1.009.patch, HBASE-22804.branch-1.010.patch, > HBASE-22804.branch-2.001.patch, HBASE-22804.branch-2.002.patch, > HBASE-22804.branch-2.003.patch, HBASE-22804.branch-2.004.patch, > HBASE-22804.branch-2.005.patch, HBASE-22804.branch-2.006.patch, > HBASE-22804.master.001.patch, HBASE-22804.master.002.patch, > HBASE-22804.master.003.patch, HBASE-22804.master.004.patch, > HBASE-22804.master.005.patch, HBASE-22804.master.006.patch > > > At present HBase Canary tool only prints the successes as part of logs. > Providing an API to get the list of successes, as well as total number of > expected regions, will make it easier to get a more accurate availability > estimate. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22629) Remove TestReplicationDroppedTables from branch-1
[ https://issues.apache.org/jira/browse/HBASE-22629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-22629: Fix Version/s: 1.4.11 > Remove TestReplicationDroppedTables from branch-1 > - > > Key: HBASE-22629 > URL: https://issues.apache.org/jira/browse/HBASE-22629 > Project: HBase > Issue Type: Bug >Affects Versions: 1.5.0 >Reporter: Andrew Kyle Purtell >Assignee: Andrew Kyle Purtell >Priority: Minor > Fix For: 1.5.0, 1.4.11 > > Attachments: HBASE-22629-branch-1.patch > > > TestReplicationDroppedTables has been flaky from initial commit and is > outright broken on recent branch-1 now. This test was contributed by us but > we dropped this test from our internal fork a while back. Do the same in open > source. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22605) Ref guide includes dev guidance only applicable to EOM versions
[
https://issues.apache.org/jira/browse/HBASE-22605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-22605:
Fix Version/s: 1.4.11
> Ref guide includes dev guidance only applicable to EOM versions
> ---
>
> Key: HBASE-22605
> URL: https://issues.apache.org/jira/browse/HBASE-22605
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Sean Busbey
>Assignee: Mingliang Liu
>Priority: Trivial
> Labels: beginner
> Fix For: 3.0.0, 2.3.0, 2.0.6, 2.2.1, 2.1.6, 1.4.11
>
> Attachments: HBASE-22605.000.patch
>
>
> The ref guide section on developer guidance has this blurb:
> {quote}
> h2. Implementing Writable
> h3. Applies pre-0.96 only
> bq. In 0.96, HBase moved to protocol buffers (protobufs). The below section
> on Writables applies to 0.94.x and previous, not to 0.96 and beyond.
> Every class returned by RegionServers must implement the Writable interface.
> If you are creating a new class that needs to implement this interface, do
> not forget the default constructor.
> {quote}
> ([ref|http://hbase.apache.org/book.html#common.patch.feedback.writable])
> this should be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22132) Backport HBASE-22115 "HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe" intent to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-22132: Summary: Backport HBASE-22115 "HBase RPC aspires to grow an infinite tree of trace scopes; some other places are also unsafe" intent to branch-1 (was: Backport HBASE-22115 intent to branch-1) > Backport HBASE-22115 "HBase RPC aspires to grow an infinite tree of trace > scopes; some other places are also unsafe" intent to branch-1 > --- > > Key: HBASE-22132 > URL: https://issues.apache.org/jira/browse/HBASE-22132 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.5.0 >Reporter: Andrew Kyle Purtell >Assignee: Viraj Jasani >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > Attachments: HBASE-22132-branch-1.patch > > > Check the exposure of branch-1 code to the problems described on HBASE-22115 > and apply the fix approach there. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22706) Backport HBASE-21292 "IdLock.getLockEntry() may hang if interrupted" to branch-1
[ https://issues.apache.org/jira/browse/HBASE-22706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-22706: Summary: Backport HBASE-21292 "IdLock.getLockEntry() may hang if interrupted" to branch-1 (was: Backport HBASE-21292 to branch-1) > Backport HBASE-21292 "IdLock.getLockEntry() may hang if interrupted" to > branch-1 > > > Key: HBASE-22706 > URL: https://issues.apache.org/jira/browse/HBASE-22706 > Project: HBase > Issue Type: Sub-task >Reporter: Pankaj Kumar >Assignee: Pankaj Kumar >Priority: Major > Fix For: 1.5.0, 1.3.6, 1.4.11 > > Attachments: HBASE-22706.branch-1.patch, HBASE-22706.branch-1.patch > > > Recently we met the same problem in one of our production env (HBase-1.3.1). > > I think we missed this, > https://issues.apache.org/jira/browse/HBASE-21292?focusedCommentId=16656135=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16656135 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22901) TestFlushSnapshotFromClient is failing
[
https://issues.apache.org/jira/browse/HBASE-22901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-22901:
Fix Version/s: (was: 1.4.11)
(was: 1.5.0)
> TestFlushSnapshotFromClient is failing
> --
>
> Key: HBASE-22901
> URL: https://issues.apache.org/jira/browse/HBASE-22901
> Project: HBase
> Issue Type: Bug
> Components: snapshots, test
>Affects Versions: 1.5.0, 1.4.11
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
>
> TestFlushSnapshotFromClient#testConcurrentSnapshottingAttempts always fails
> on branch-1 and branch-1.4.
> {noformat}
> java.lang.AssertionError: We expect at least 1 snapshot of table2
> at
> org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient.testConcurrentSnapshottingAttempts(TestFlushSnapshotFromClient.java:503)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-22797) [backport] HBASE-22778 upgrade jackson-databind to 2.9.9.2
[ https://issues.apache.org/jira/browse/HBASE-22797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-22797: Fix Version/s: (was: 1.4.11) (was: 1.3.6) (was: 1.5.0) > [backport] HBASE-22778 upgrade jackson-databind to 2.9.9.2 > -- > > Key: HBASE-22797 > URL: https://issues.apache.org/jira/browse/HBASE-22797 > Project: HBase > Issue Type: Bug > Components: security >Reporter: Reid Chan >Priority: Blocker > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22540) [Memstore] Correct counters in MemStoreChunkPool
[ https://issues.apache.org/jira/browse/HBASE-22540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-22540: Fix Version/s: (was: 1.4.11) 1.4.10 > [Memstore] Correct counters in MemStoreChunkPool > > > Key: HBASE-22540 > URL: https://issues.apache.org/jira/browse/HBASE-22540 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.6 >Reporter: Reid Chan >Assignee: Reid Chan >Priority: Minor > Fix For: 1.5.0, 1.4.10 > > > As title reads. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-22495) Update SyncTable section, explaining from which specific minor versions "doDeletes/doPuts" is available
[ https://issues.apache.org/jira/browse/HBASE-22495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-22495: Fix Version/s: (was: 1.4.11) 1.5.0 > Update SyncTable section, explaining from which specific minor versions > "doDeletes/doPuts" is available > --- > > Key: HBASE-22495 > URL: https://issues.apache.org/jira/browse/HBASE-22495 > Project: HBase > Issue Type: Bug > Components: documentation >Affects Versions: 1.2.0 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 1.5.0, 2.3.0 > > Attachments: HBASE-22495.master.001 > > > On SyncTable section from ref guide for 1.2, we are mentioning > "doDeletes/doPuts", but these were only added by HBASE-20305. Original patch > proposed there was for master branch only, and related branch-1 and branch-2 > patches were only committed recently, so minimal supported version is 1.4.10, > for branch-1 releases, and 2.1.5 for branch-2. Am proposing a patch that > mentions which are the minimum versions that support such commands. > Also, we should add this whole section to 2.0 and 2.1 Documentations (these > are currently missing). > Thanks [~npopa] for pointing this out. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-21920) Ignoring 'empty' end_key while calculating end_key for new region in HBCK -fixHdfsOverlaps command can cause data loss
[
https://issues.apache.org/jira/browse/HBASE-21920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-21920:
Fix Version/s: (was: 1.4.11)
1.4.10
> Ignoring 'empty' end_key while calculating end_key for new region in HBCK
> -fixHdfsOverlaps command can cause data loss
> --
>
> Key: HBASE-21920
> URL: https://issues.apache.org/jira/browse/HBASE-21920
> Project: HBase
> Issue Type: Bug
> Components: hbck
>Reporter: Syeda Arshiya Tabreen
>Assignee: Syeda Arshiya Tabreen
>Priority: Major
> Fix For: 1.5.0, 1.4.10, 1.3.6
>
> Attachments: HBASE-21920.branch-1.001.patch,
> HBASE-21920.branch-1.002.patch, HBASE-21920.branch-1.002.patch,
> HBASE-21920.branch-1.002.patch, HBASE-21920.branch-1.002.patch,
> HBASE-21920.branch-1.patch
>
>
> When running *-fixHdfsOverlaps* command due to overlap in the regions of the
> table ,it moves all the hfiles of overlapping regions into new region with
> start_key and end_key calculating based on minimum and maximum start_key and
> end_key of all overlapping regions.
> When calculating start_key and end_key for new region,end_key with 'empty' is
> not considered which leads to data loss when scanned using '*startrow'.*
> *For example:*
> 1.create table 't'
> 2.Insert records \{00,111,200} into the table 't'and flush the data
> 3.split the table 't' with split-key '100'
> 4.Now we have three regions( 1 parent and two daughter regions )
> 1.*Region-1*('Empty','Empty') => \{00,111,200}
> 2.*Region-2*('Empty','100')=>\{00}
> 3.*Region-3*('100','Empty')=>\{111,200}
> 5.Make sure parent region is not deleted in file system and run
> -*fixHdfsOverlaps* command
> This -*fixHdfsOverlaps* command will move all the hfiles of the three regions
> {*Region-1,Region- 2,Region-3*} into a new region(*Region-4*) created with
> start_key='*Empty'* and end_key='*100'*
> This is because it does not consider end_key=*'Empty'* and considers
> end_key=*'100'* as maximum which in turn makes all the hfiles of three
> regions to move into new region even if records in hfile is more than the
> end_key='*100'* and one empty region *Region -5 (100,Empty)* will be
> created because table region end key was not empty.
> Now we have 2 regions:
> 1.*Region-4*(Empty,100)=>\{00,111,200}
> 2.*Region-5*(100,Empty)=>{}
> when the entire table scan is done, all the records will be displayed, there
> wont be any data loss but scan with start_key is done below are the results:
> 1.scan 't', \{ STARTROW => '00'} => \{00,111,200}
> 2.scan 't', \{ STARTROW => '100'}=>{}
> The second scan will give empty result because it searches the rows in
> *Region -5*(100,Empty) which contains no records but records \{111,200} is
> present in *Region-4*(Empty,100).
> The problem exists only when end_key=*'Empty'* is present in any of the
> overlapping regions.I think if end_key is present in any of the overlapping
> regions,we have to consider it as maximum end_key.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23038) Provide consistent and clear logging about disabling chores
[
https://issues.apache.org/jira/browse/HBASE-23038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-23038:
Fix Version/s: 2.3.0
1.5.0
3.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Provide consistent and clear logging about disabling chores
> ---
>
> Key: HBASE-23038
> URL: https://issues.apache.org/jira/browse/HBASE-23038
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
>Reporter: Sean Busbey
>Assignee: Sanjeet Nishad
>Priority: Minor
> Labels: beginner
> Fix For: 3.0.0, 1.5.0, 2.3.0
>
> Attachments: HBASE-23038.001.patch, HBASE-23038.002.patch,
> HBASE-23038.master.003.patch
>
>
> Right now if you want to disable any of our chores you can set the period to
> be <= 0. Sometimes, if you do this you get a nice message:
> {code}
> 2019-09-16 22:10:16,756 INFO [master-1:16000.activeMasterManager]
> master.HMaster: The period is 0 seconds, MobCompactionChore is disabled
> {code}
> And sometimes you get an opaque message:
> {code}
> 2019-09-16 22:09:45,333 INFO [master-1:16000.activeMasterManager]
> hbase.ChoreService: Could not successfully schedule chore: LogsCleaner
> 2019-09-16 22:09:45,340 INFO [master-1:16000.activeMasterManager]
> hbase.ChoreService: Could not successfully schedule chore: HFileCleaner
> {code}
> This is because sometimes we just blindly submit to ChoreService which
> submits to a java ScheduledExecutorService and then catches the
> IllegalArgumentException.
> We should remove the one-offs and make it so ChoreService checks the period
> before accepting a submittal and produces a consistent "Foo is disabled"
> message.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22902) At regionserver start there's a request to roll the WAL
[
https://issues.apache.org/jira/browse/HBASE-22902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939805#comment-16939805
]
HBase QA commented on HBASE-22902:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:orange}-0{color} | {color:orange} test4tests {color} | {color:orange}
0m 0s{color} | {color:orange} The patch doesn't appear to include any new or
modified tests. Please justify why no new tests are needed for this patch. Also
please list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
25s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
5s{color} | {color:green} branch-1 passed with JDK v1.8.0_222 {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
21s{color} | {color:red} hbase-server in branch-1 failed with JDK v1.7.0_232.
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
44s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
9s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} branch-1 passed with JDK v1.8.0_222 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} branch-1 passed with JDK v1.7.0_232 {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
2s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
58s{color} | {color:green} branch-1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
50s{color} | {color:green} the patch passed with JDK v1.8.0_222 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
50s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
22s{color} | {color:red} hbase-server in the patch failed with JDK v1.7.0_232.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 22s{color}
| {color:red} hbase-server in the patch failed with JDK v1.7.0_232. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 3m
6s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
5m 9s{color} | {color:green} Patch does not cause any errors with Hadoop 2.8.5
2.9.2. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed with JDK v1.8.0_222 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed with JDK v1.7.0_232 {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}150m 10s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} |
[GitHub] [hbase] Apache-HBase commented on issue #668: HBASE-22902: At regionserver start there's a request to roll the WAL
Apache-HBase commented on issue #668: HBASE-22902: At regionserver start there's a request to roll the WAL URL: https://github.com/apache/hbase/pull/668#issuecomment-536136344 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 14s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :yellow_heart: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ branch-1 Compile Tests _ | | :green_heart: | mvninstall | 8m 25s | branch-1 passed | | :green_heart: | compile | 2m 5s | branch-1 passed with JDK v1.8.0_222 | | :broken_heart: | compile | 0m 21s | hbase-server in branch-1 failed with JDK v1.7.0_232. | | :green_heart: | checkstyle | 1m 44s | branch-1 passed | | :green_heart: | shadedjars | 3m 9s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 36s | branch-1 passed with JDK v1.8.0_222 | | :green_heart: | javadoc | 0m 41s | branch-1 passed with JDK v1.7.0_232 | | :blue_heart: | spotbugs | 3m 2s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 2m 58s | branch-1 passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 2m 3s | the patch passed | | :green_heart: | compile | 1m 50s | the patch passed with JDK v1.8.0_222 | | :green_heart: | javac | 1m 50s | the patch passed | | :broken_heart: | compile | 0m 22s | hbase-server in the patch failed with JDK v1.7.0_232. | | :broken_heart: | javac | 0m 22s | hbase-server in the patch failed with JDK v1.7.0_232. | | :green_heart: | checkstyle | 1m 40s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 3m 6s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 5m 9s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | :green_heart: | javadoc | 0m 31s | the patch passed with JDK v1.8.0_222 | | :green_heart: | javadoc | 0m 41s | the patch passed with JDK v1.7.0_232 | | :green_heart: | findbugs | 3m 4s | the patch passed | ||| _ Other Tests _ | | :broken_heart: | unit | 150m 10s | hbase-server in the patch failed. | | :green_heart: | asflicense | 0m 30s | The patch does not generate ASF License warnings. | | | | 193m 50s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestAdmin2 | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-668/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/668 | | JIRA Issue | HBASE-22902 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 19c25e5d30e1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-668/out/precommit/personality/provided.sh | | git revision | branch-1 / f77c14d | | Default Java | 1.7.0_232 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:1.8.0_222 /usr/lib/jvm/zulu-7-amd64:1.7.0_232 | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-668/1/artifact/out/branch-compile-hbase-server-jdk1.7.0_232.txt | | compile | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-668/1/artifact/out/patch-compile-hbase-server-jdk1.7.0_232.txt | | javac | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-668/1/artifact/out/patch-compile-hbase-server-jdk1.7.0_232.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-668/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-668/1/testReport/ | | Max. process+thread count | 4266 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-668/1/console | | versions | git=1.9.1
[jira] [Updated] (HBASE-22902) At regionserver start there's a request to roll the WAL
[
https://issues.apache.org/jira/browse/HBASE-22902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal updated HBASE-22902:
Attachment: HBASE-22902_branch-1_v1.patch
Status: Patch Available (was: In Progress)
> At regionserver start there's a request to roll the WAL
> ---
>
> Key: HBASE-22902
> URL: https://issues.apache.org/jira/browse/HBASE-22902
> Project: HBase
> Issue Type: Bug
> Components: wal
>Affects Versions: 3.0.0, 1.5.0, 2.3.0
>Reporter: David Manning
>Assignee: Sandeep Pal
>Priority: Minor
> Attachments: HBASE-22902_branch-1_v1.patch
>
>
> See HBASE-22301 for logic that requests to roll the WAL if regionserver
> encounters a slow write pipeline. In the logs, during regionserver start, I
> see that the WAL is requested to roll once. It's strange that we roll the WAL
> because it wasn't a slow sync. It appears when this code executes, we haven't
> initialized the {{rollOnSyncNs}} variable to use for determining whether it's
> a slow sync. Current pipeline also shows empty in the logs.
> Disclaimer: I'm experiencing this after backporting this to 1.3.x and
> building it there - I haven't attempted in 1.5.x, though I'd expect similar
> results.
> Regionserver logs follow (notice *threshold=0 ms, current pipeline: []*):
> {noformat}
> Tue Aug 20 23:29:50 GMT 2019 Starting regionserver
> ...
> 2019-08-20 23:29:57,824 INFO wal.FSHLog - WAL configuration: blocksize=256
> MB, rollsize=243.20 MB, prefix=[truncated]%2C1566343792434, suffix=,
> logDir=hdfs://[truncated]/hbase/WALs/[truncated],1566343792434,
> archiveDir=hdfs://[truncated]/hbase/oldWALs
> 2019-08-20 23:29:58,104 INFO wal.FSHLog - Slow sync cost: 186 ms, current
> pipeline: []
> 2019-08-20 23:29:58,104 WARN wal.FSHLog - Requesting log roll because we
> exceeded slow sync threshold; time=186 ms, threshold=0 ms, current pipeline:
> []
> 2019-08-20 23:29:58,107 DEBUG regionserver.ReplicationSourceManager - Start
> tracking logs for wal group [truncated]%2C1566343792434 for peer 1
> 2019-08-20 23:29:58,107 INFO wal.FSHLog - New WAL
> /hbase/WALs/[truncated],1566343792434/[truncated]%2C1566343792434.1566343797824
> 2019-08-20 23:29:58,109 DEBUG regionserver.ReplicationSource - Starting up
> worker for wal group [truncated]%2C1566343792434{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22012) SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table
[
https://issues.apache.org/jira/browse/HBASE-22012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939796#comment-16939796
]
Hudson commented on HBASE-22012:
Results for branch branch-2
[build #2304 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2304/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2304//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2304//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2/2304//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable
> table
>
>
> Key: HBASE-22012
> URL: https://issues.apache.org/jira/browse/HBASE-22012
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.2.1
>Reporter: Ajeet Rai
>Assignee: Shardul Singh
>Priority: Major
> Labels: Quota, Space
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> Space Quota: Policy state is getting changed from disable to Observance after
> sometime automatically.
> Steps:
> 1: Create a table with space quota policy as Disable
> 2: Put some data so that table state is in space quota violation
> 3: So observe that table state is in violation
> 4: Now wait for some time
> 5: Observe that after some time table state is changing to to Observance
> however table is still disabled
> edit (elserj): The table is automatically moved back from the violation state
> because of the code added that tried to ride over RITs. When a Region is not
> online (whether normally or abnormally), the RegionSizeReports are not sent
> from RS to Master. Eventually, enough Regions are not reported which dips
> below the acceptable threshold and we automatically move the table back to
> the "acceptable" space quota state (not in violation). We could skip this
> failsafe when we're checking for a quota that has the DisableTable violation
> policy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23054) Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm
[ https://issues.apache.org/jira/browse/HBASE-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939792#comment-16939792 ] Michael Stack commented on HBASE-23054: --- +1 Excellent. > Remove synchronization block from MetaTableMetrics and fix LossyCounting > algorithm > -- > > Key: HBASE-23054 > URL: https://issues.apache.org/jira/browse/HBASE-23054 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.5 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Attachments: HBASE-23054.master.001.patch, > HBASE-23054.master.002.patch > > > While trying to use LossyCounting for HBASE-15519 , found following bugs in > current implementation > – Remove synchronization block from MetaTableMetrics to avoid congestion at > the code > – Fix license format > – Fix LossyCounting algorithm as per > [http://www.vldb.org/conf/2002/S10P03.pdf > |http://www.vldb.org/conf/2002/S10P03.pdf] > -- Avoid doing sweep on every insert in LossyCounting > – Remove extra redundant data structures from MetaTableMetrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #664: HBASE-23083 Collect Executor status info periodically and report to metric system
saintstack commented on a change in pull request #664: HBASE-23083 Collect
Executor status info periodically and report to metric system
URL: https://github.com/apache/hbase/pull/664#discussion_r329281286
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/ExecutorStatusChore.java
##
@@ -0,0 +1,84 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase;
+
+import java.util.Map;
+import org.apache.hadoop.hbase.executor.ExecutorService;
+import org.apache.hadoop.hbase.executor.ExecutorService.ExecutorStatus;
+import org.apache.hadoop.hbase.regionserver.MetricsRegionServerSource;
+import org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceImpl;
+import org.apache.hadoop.hbase.util.Pair;
+import org.apache.hadoop.metrics2.lib.DynamicMetricsRegistry;
+import org.apache.hadoop.metrics2.lib.MutableGaugeLong;
+import org.apache.hadoop.util.StringUtils;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+
+/**
+ * The Class ExecutorStatusChore for collect Executor status info periodically
+ * and report to metrics system
+ */
+@InterfaceAudience.Private
+public class ExecutorStatusChore extends ScheduledChore {
+ private static final Logger LOG =
LoggerFactory.getLogger(HealthCheckChore.class);
+ public static final String WAKE_FREQ =
"hbase.executors.status.collect.period";
+ public static final int DEFAULT_WAKE_FREQ = 6;
+ private ExecutorService service;
+ private DynamicMetricsRegistry metricsRegistry;
+
+ public ExecutorStatusChore(int sleepTime, Stoppable stopper, ExecutorService
service,
+ MetricsRegionServerSource metrics) {
+super("ExecutorStatusChore", stopper, sleepTime);
+LOG.info("ExecutorStatusChore runs every " +
StringUtils.formatTime(sleepTime));
Review comment:
s/runs every/runs every {}/ and s/+/,/... i.e. use parameterized logging.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #664: HBASE-23083 Collect Executor status info periodically and report to metric system
saintstack commented on a change in pull request #664: HBASE-23083 Collect Executor status info periodically and report to metric system URL: https://github.com/apache/hbase/pull/664#discussion_r329280786 ## File path: hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java ## @@ -1191,6 +1191,9 @@ "hbase.node.health.failure.threshold"; public static final int DEFAULT_HEALTH_FAILURE_THRESHOLD = 3; + public static final String EXECUTOR_STATUS_COLLECTE_ENABLED = + "hbase.executors.status.collect.enabled"; + public static final boolean DEFAULT_EXECUTOR_STATUS_COLLECTE_ENABLED = true; Review comment: COLLECTE<= extra 'E'. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #664: HBASE-23083 Collect Executor status info periodically and report to metric system
saintstack commented on a change in pull request #664: HBASE-23083 Collect
Executor status info periodically and report to metric system
URL: https://github.com/apache/hbase/pull/664#discussion_r329281476
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/ExecutorStatusChore.java
##
@@ -0,0 +1,84 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase;
+
+import java.util.Map;
+import org.apache.hadoop.hbase.executor.ExecutorService;
+import org.apache.hadoop.hbase.executor.ExecutorService.ExecutorStatus;
+import org.apache.hadoop.hbase.regionserver.MetricsRegionServerSource;
+import org.apache.hadoop.hbase.regionserver.MetricsRegionServerSourceImpl;
+import org.apache.hadoop.hbase.util.Pair;
+import org.apache.hadoop.metrics2.lib.DynamicMetricsRegistry;
+import org.apache.hadoop.metrics2.lib.MutableGaugeLong;
+import org.apache.hadoop.util.StringUtils;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import
org.apache.hbase.thirdparty.com.google.common.annotations.VisibleForTesting;
+
+/**
+ * The Class ExecutorStatusChore for collect Executor status info periodically
+ * and report to metrics system
+ */
+@InterfaceAudience.Private
+public class ExecutorStatusChore extends ScheduledChore {
+ private static final Logger LOG =
LoggerFactory.getLogger(HealthCheckChore.class);
+ public static final String WAKE_FREQ =
"hbase.executors.status.collect.period";
+ public static final int DEFAULT_WAKE_FREQ = 6;
+ private ExecutorService service;
+ private DynamicMetricsRegistry metricsRegistry;
+
+ public ExecutorStatusChore(int sleepTime, Stoppable stopper, ExecutorService
service,
+ MetricsRegionServerSource metrics) {
+super("ExecutorStatusChore", stopper, sleepTime);
+LOG.info("ExecutorStatusChore runs every " +
StringUtils.formatTime(sleepTime));
+this.service = service;
+this.metricsRegistry = ((MetricsRegionServerSourceImpl)
metrics).getMetricsRegistry();
+ }
+
+ @Override
+ protected void chore() {
+try{
+ // thread pool monitor
+ Map statuses = service.getAllExecutorStatuses();
+ for (Map.Entry statusEntry :
statuses.entrySet()) {
+String name = statusEntry.getKey();
+String poolName = name.split("-")[0];
Review comment:
Does this presume a format for the name? If so, add a comment on what is
expected?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] saintstack commented on a change in pull request #664: HBASE-23083 Collect Executor status info periodically and report to metric system
saintstack commented on a change in pull request #664: HBASE-23083 Collect
Executor status info periodically and report to metric system
URL: https://github.com/apache/hbase/pull/664#discussion_r329281885
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
##
@@ -1922,6 +1926,14 @@ private void startServices() throws IOException {
HConstants.DEFAULT_THREAD_WAKE_FREQUENCY);
healthCheckChore = new HealthCheckChore(sleepTime, this,
getConfiguration());
}
+// Executor status collect thread.
+if (this.conf.getBoolean(HConstants.EXECUTOR_STATUS_COLLECTE_ENABLED,
+HConstants.DEFAULT_EXECUTOR_STATUS_COLLECTE_ENABLED)) {
+ int sleepTime = this.conf.getInt(ExecutorStatusChore.WAKE_FREQ,
+ ExecutorStatusChore.DEFAULT_WAKE_FREQ);
+ executorStatusChore = new ExecutorStatusChore(sleepTime, this,
this.getExecutorService(),
Review comment:
Are there other executor services running in RS? This the only one?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23054) Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm
[ https://issues.apache.org/jira/browse/HBASE-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939791#comment-16939791 ] Ankit Singhal commented on HBASE-23054: --- Thank you soo much [~stack] for the review. Have uploaded the patch(002) after incorporating all the review comments. > Remove synchronization block from MetaTableMetrics and fix LossyCounting > algorithm > -- > > Key: HBASE-23054 > URL: https://issues.apache.org/jira/browse/HBASE-23054 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.5 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Attachments: HBASE-23054.master.001.patch, > HBASE-23054.master.002.patch > > > While trying to use LossyCounting for HBASE-15519 , found following bugs in > current implementation > – Remove synchronization block from MetaTableMetrics to avoid congestion at > the code > – Fix license format > – Fix LossyCounting algorithm as per > [http://www.vldb.org/conf/2002/S10P03.pdf > |http://www.vldb.org/conf/2002/S10P03.pdf] > -- Avoid doing sweep on every insert in LossyCounting > – Remove extra redundant data structures from MetaTableMetrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HBASE-23054) Remove synchronization block from MetaTableMetrics and fix LossyCounting algorithm
[ https://issues.apache.org/jira/browse/HBASE-23054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal updated HBASE-23054: -- Attachment: HBASE-23054.master.002.patch > Remove synchronization block from MetaTableMetrics and fix LossyCounting > algorithm > -- > > Key: HBASE-23054 > URL: https://issues.apache.org/jira/browse/HBASE-23054 > Project: HBase > Issue Type: Bug >Affects Versions: 2.1.5 >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Major > Attachments: HBASE-23054.master.001.patch, > HBASE-23054.master.002.patch > > > While trying to use LossyCounting for HBASE-15519 , found following bugs in > current implementation > – Remove synchronization block from MetaTableMetrics to avoid congestion at > the code > – Fix license format > – Fix LossyCounting algorithm as per > [http://www.vldb.org/conf/2002/S10P03.pdf > |http://www.vldb.org/conf/2002/S10P03.pdf] > -- Avoid doing sweep on every insert in LossyCounting > – Remove extra redundant data structures from MetaTableMetrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-22012) SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table
[
https://issues.apache.org/jira/browse/HBASE-22012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939789#comment-16939789
]
Hudson commented on HBASE-22012:
Results for branch branch-2.2
[build #643 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/643/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/643//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/643//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/branch-2.2/643//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable
> table
>
>
> Key: HBASE-22012
> URL: https://issues.apache.org/jira/browse/HBASE-22012
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.2.1
>Reporter: Ajeet Rai
>Assignee: Shardul Singh
>Priority: Major
> Labels: Quota, Space
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> Space Quota: Policy state is getting changed from disable to Observance after
> sometime automatically.
> Steps:
> 1: Create a table with space quota policy as Disable
> 2: Put some data so that table state is in space quota violation
> 3: So observe that table state is in violation
> 4: Now wait for some time
> 5: Observe that after some time table state is changing to to Observance
> however table is still disabled
> edit (elserj): The table is automatically moved back from the violation state
> because of the code added that tried to ride over RITs. When a Region is not
> online (whether normally or abnormally), the RegionSizeReports are not sent
> from RS to Master. Eventually, enough Regions are not reported which dips
> below the acceptable threshold and we automatically move the table back to
> the "acceptable" space quota state (not in violation). We could skip this
> failsafe when we're checking for a quota that has the DisableTable violation
> policy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #611: HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command
Apache-HBase commented on issue #611: HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command URL: https://github.com/apache/hbase/pull/611#issuecomment-536114414 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 36s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :blue_heart: | prototool | 0m 0s | prototool was not available. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :yellow_heart: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 34s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 28s | master passed | | :green_heart: | compile | 2m 28s | master passed | | :green_heart: | checkstyle | 2m 25s | master passed | | :green_heart: | shadedjars | 4m 47s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 29s | master passed | | :blue_heart: | spotbugs | 4m 20s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 8m 11s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 7s | the patch passed | | :green_heart: | compile | 2m 30s | the patch passed | | :green_heart: | cc | 2m 30s | the patch passed | | :green_heart: | javac | 2m 30s | the patch passed | | :broken_heart: | checkstyle | 0m 41s | hbase-client: The patch generated 1 new + 180 unchanged - 0 fixed = 181 total (was 180) | | :broken_heart: | rubocop | 0m 10s | The patch generated 3 new + 256 unchanged - 1 fixed = 259 total (was 257) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 6m 1s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 20m 17s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | hbaseprotoc | 2m 53s | the patch passed | | :green_heart: | javadoc | 1m 40s | the patch passed | | :green_heart: | findbugs | 10m 7s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 0m 51s | hbase-protocol-shaded in the patch passed. | | :green_heart: | unit | 2m 10s | hbase-client in the patch passed. | | :green_heart: | unit | 162m 28s | hbase-server in the patch passed. | | :green_heart: | unit | 7m 14s | hbase-shell in the patch passed. | | :green_heart: | asflicense | 2m 8s | The patch does not generate ASF License warnings. | | | | 260m 54s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/611 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool rubocop | | uname | Linux 78f96f1a122f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-611/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/2/artifact/out/diff-checkstyle-hbase-client.txt | | rubocop | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/2/artifact/out/diff-patch-rubocop.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/2/testReport/ | | Max. process+thread count | 4806 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-client hbase-server hbase-shell U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-611/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 rubocop=0.74.0 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated.
[GitHub] [hbase] Apache-HBase commented on issue #667: HBASE-23055 Alter hbase:meta
Apache-HBase commented on issue #667: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/667#issuecomment-536107334 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 55s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 1s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 11 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 37s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 20s | master passed | | :green_heart: | compile | 2m 2s | master passed | | :green_heart: | checkstyle | 2m 45s | master passed | | :green_heart: | shadedjars | 4m 32s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 41s | master passed | | :blue_heart: | spotbugs | 4m 0s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 6m 24s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 4m 59s | the patch passed | | :green_heart: | compile | 2m 4s | the patch passed | | :green_heart: | javac | 2m 4s | the patch passed | | :broken_heart: | checkstyle | 0m 33s | hbase-client: The patch generated 1 new + 97 unchanged - 4 fixed = 98 total (was 101) | | :broken_heart: | checkstyle | 1m 27s | hbase-server: The patch generated 8 new + 446 unchanged - 19 fixed = 454 total (was 465) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 33s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 53s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 46s | the patch passed | | :green_heart: | findbugs | 7m 38s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 3m 6s | hbase-common in the patch passed. | | :green_heart: | unit | 1m 54s | hbase-client in the patch passed. | | :green_heart: | unit | 0m 49s | hbase-zookeeper in the patch passed. | | :broken_heart: | unit | 158m 12s | hbase-server in the patch failed. | | :green_heart: | asflicense | 2m 10s | The patch does not generate ASF License warnings. | | | | 238m 59s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.wal.TestLogRollingNoCluster | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-667/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/667 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 56a2a3b9bfc7 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-667/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-667/1/artifact/out/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-667/1/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-667/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-667/1/testReport/ | | Max. process+thread count | 4694 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-zookeeper hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-667/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
[GitHub] [hbase] sandeepvinayak opened a new pull request #668: calling rollwriter after initialization of the config variables since…
sandeepvinayak opened a new pull request #668: calling rollwriter after initialization of the config variables since… URL: https://github.com/apache/hbase/pull/668 Rollwriter eventually calls `postSync` which checks for if there is a need for rolling log based on `rollOnSyncNs` and `slowSyncNs`. So, everytime the FSLog is initialized it requests a log roll because the values of these two variables are zero. Moving this call after the initialization of variables. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve…
Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve… URL: https://github.com/apache/hbase/pull/435#issuecomment-536093374 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 2s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :blue_heart: | shelldocs | 0m 0s | Shelldocs was not available. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 1m 8s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 7m 14s | master passed | | :green_heart: | compile | 2m 5s | master passed | | :green_heart: | checkstyle | 2m 10s | master passed | | :green_heart: | shadedjars | 5m 10s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 32s | master passed | | :blue_heart: | spotbugs | 4m 21s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 6m 1s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 16s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 30s | the patch passed | | :green_heart: | compile | 2m 1s | the patch passed | | :green_heart: | javac | 2m 1s | the patch passed | | :broken_heart: | checkstyle | 1m 27s | hbase-server: The patch generated 1 new + 9 unchanged - 0 fixed = 10 total (was 9) | | :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 46s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 40s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 23s | the patch passed | | :broken_heart: | findbugs | 0m 56s | hbase-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | :green_heart: | unit | 3m 3s | hbase-common in the patch passed. | | :green_heart: | unit | 164m 8s | hbase-server in the patch passed. | | :green_heart: | unit | 4m 8s | hbase-rsgroup in the patch passed. | | :green_heart: | asflicense | 1m 24s | The patch does not generate ASF License warnings. | | | | 244m 6s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-common | | | org.apache.hadoop.hbase.Version.() invokes inefficient new String(String) constructor At Version.java:inefficient new String(String) constructor At Version.java:[line 10] | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/101/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/435 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile shellcheck shelldocs | | uname | Linux 3aebef302a39 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-435/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/101/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/101/artifact/out/new-findbugs-hbase-common.html | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/101/testReport/ | | Max. process+thread count | 4660 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server hbase-rsgroup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/101/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) shellcheck=0.7.0 findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service.
[GitHub] [hbase] Apache-HBase commented on issue #655: HBASE-23055 Alter hbase:meta
Apache-HBase commented on issue #655: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/655#issuecomment-536078349 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 39s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 9 new or modified test files. | ||| _ branch-2 Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 12s | branch-2 passed | | :green_heart: | compile | 2m 4s | branch-2 passed | | :green_heart: | checkstyle | 2m 54s | branch-2 passed | | :green_heart: | shadedjars | 4m 11s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 40s | branch-2 passed | | :blue_heart: | spotbugs | 0m 34s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 5m 47s | branch-2 passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 4m 51s | the patch passed | | :green_heart: | compile | 2m 5s | the patch passed | | :green_heart: | javac | 2m 5s | the patch passed | | :green_heart: | checkstyle | 0m 28s | The patch passed checkstyle in hbase-common | | :green_heart: | checkstyle | 0m 42s | hbase-client: The patch generated 0 new + 168 unchanged - 5 fixed = 168 total (was 173) | | :green_heart: | checkstyle | 0m 14s | The patch passed checkstyle in hbase-zookeeper | | :green_heart: | checkstyle | 1m 31s | hbase-server: The patch generated 0 new + 473 unchanged - 19 fixed = 473 total (was 492) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 10s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 15s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 39s | the patch passed | | :green_heart: | findbugs | 6m 17s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 2m 56s | hbase-common in the patch passed. | | :green_heart: | unit | 3m 26s | hbase-client in the patch passed. | | :green_heart: | unit | 0m 51s | hbase-zookeeper in the patch passed. | | :green_heart: | unit | 155m 8s | hbase-server in the patch passed. | | :green_heart: | asflicense | 2m 6s | The patch does not generate ASF License warnings. | | | | 231m 37s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-655/10/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/655 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 57383cbeb3cb 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-655/out/precommit/personality/provided.sh | | git revision | branch-2 / 43b4ad9e8a | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-655/10/testReport/ | | Max. process+thread count | 4170 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-zookeeper hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-655/10/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #664: HBASE-23083 Collect Executor status info periodically and report to metric system
Apache-HBase commented on issue #664: HBASE-23083 Collect Executor status info periodically and report to metric system URL: https://github.com/apache/hbase/pull/664#issuecomment-536073745 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 50s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 39s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 50s | master passed | | :green_heart: | compile | 1m 34s | master passed | | :green_heart: | checkstyle | 2m 12s | master passed | | :green_heart: | shadedjars | 5m 54s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 4s | master passed | | :blue_heart: | spotbugs | 4m 55s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 5m 50s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 32s | the patch passed | | :green_heart: | compile | 1m 27s | the patch passed | | :green_heart: | javac | 1m 27s | the patch passed | | :green_heart: | checkstyle | 0m 28s | The patch passed checkstyle in hbase-common | | :green_heart: | checkstyle | 1m 23s | hbase-server: The patch generated 0 new + 64 unchanged - 9 fixed = 64 total (was 73) | | :green_heart: | whitespace | 0m 1s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 5m 11s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 17m 45s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 58s | the patch passed | | :green_heart: | findbugs | 5m 8s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 3m 5s | hbase-common in the patch passed. | | :green_heart: | unit | 154m 28s | hbase-server in the patch passed. | | :green_heart: | asflicense | 1m 7s | The patch does not generate ASF License warnings. | | | | 227m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/664 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 71be3e705a6c 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-664/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/3/testReport/ | | Max. process+thread count | 4946 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #626: HBASE-23017 Verify the file integrity in persistent IOEngine
Apache-HBase commented on issue #626: HBASE-23017 Verify the file integrity in persistent IOEngine URL: https://github.com/apache/hbase/pull/626#issuecomment-536070490 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :blue_heart: | prototool | 0m 0s | prototool was not available. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 34s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 43s | master passed | | :green_heart: | compile | 1m 37s | master passed | | :green_heart: | checkstyle | 1m 34s | master passed | | :green_heart: | shadedjars | 4m 55s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 48s | master passed | | :blue_heart: | spotbugs | 4m 33s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 7m 11s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 14s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 25s | the patch passed | | :green_heart: | compile | 1m 37s | the patch passed | | :green_heart: | cc | 1m 37s | the patch passed | | :green_heart: | javac | 1m 37s | the patch passed | | :broken_heart: | checkstyle | 1m 28s | hbase-server: The patch generated 1 new + 30 unchanged - 0 fixed = 31 total (was 30) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 55s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 17m 5s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | hbaseprotoc | 1m 32s | the patch passed | | :green_heart: | javadoc | 0m 47s | the patch passed | | :green_heart: | findbugs | 7m 21s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 0m 41s | hbase-protocol-shaded in the patch passed. | | :green_heart: | unit | 153m 36s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 49s | The patch does not generate ASF License warnings. | | | | 225m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-626/9/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/626 | | Optional Tests | dupname asflicense cc unit hbaseprotoc prototool javac javadoc spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 960d5ad52845 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-626/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-626/9/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-626/9/testReport/ | | Max. process+thread count | 4365 (vs. ulimit of 1) | | modules | C: hbase-protocol-shaded hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-626/9/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #603: HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff
Apache-HBase commented on issue #603: HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff URL: https://github.com/apache/hbase/pull/603#issuecomment-536069606 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 29s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 1m 0s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 42s | master passed | | :green_heart: | compile | 1m 20s | master passed | | :green_heart: | checkstyle | 1m 57s | master passed | | :green_heart: | shadedjars | 5m 3s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 57s | master passed | | :blue_heart: | spotbugs | 4m 25s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 5m 11s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 13s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 23s | the patch passed | | :green_heart: | compile | 1m 20s | the patch passed | | :green_heart: | javac | 1m 20s | the patch passed | | :green_heart: | checkstyle | 1m 55s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 5m 1s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 17m 7s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 3s | the patch passed | | :green_heart: | findbugs | 5m 55s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 3m 2s | hbase-common in the patch passed. | | :green_heart: | unit | 151m 38s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 48s | The patch does not generate ASF License warnings. | | | | 222m 2s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-603/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/603 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux a835b65d9152 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-603/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-603/5/testReport/ | | Max. process+thread count | 4412 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-603/5/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on issue #611: HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command
virajjasani commented on issue #611: HBASE-22903 : Table to RegionStatesCount metrics - Use for broken alter_status command URL: https://github.com/apache/hbase/pull/611#issuecomment-536049986 Requesting your review @saintstack This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] virajjasani commented on issue #665: Update start-hbase.sh
virajjasani commented on issue #665: Update start-hbase.sh URL: https://github.com/apache/hbase/pull/665#issuecomment-536040344 Thanks for the PR @Tikiri16 Could you please specify the relevant Jira here? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23055) Alter hbase:meta
[ https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939637#comment-16939637 ] Michael Stack commented on HBASE-23055: --- #667 is PR for master branch. #655 is PR for branch-2. Its ready to go. Needs +1. > Alter hbase:meta > > > Key: HBASE-23055 > URL: https://issues.apache.org/jira/browse/HBASE-23055 > Project: HBase > Issue Type: Task >Reporter: Michael Stack >Assignee: Michael Stack >Priority: Major > > hbase:meta is currently hardcoded. Its schema cannot be change. > This issue is about allowing edits to hbase:meta schema. It will allow our > being able to set encodings such as the block-with-indexes which will help > quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first > step on road to being able to split meta. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] saintstack opened a new pull request #667: HBASE-23055 Alter hbase:meta
saintstack opened a new pull request #667: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/667 Make it so hbase:meta can be altered. TableState for hbase:meta is kept in Master. State is in-memory transient so if Master fails, hbase:meta is ENABLED again. hbase:meta schema will be bootstrapped from the filesystem. Changes to filesystem schema are atomic so we should be ok if Master fails mid-edit (TBD) Undoes a bunch of guards that prevented our being able to edit hbase:meta. At minimmum, need to add in a bunch of WARNING. TODO: Tests, more clarity around hbase:meta table state, and undoing references to hard-coded hbase:meta regioninfo. M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java Throw illegal access exception if you try to use MetaTableAccessor getting state of the hbase:meta table. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java For table state, go to master rather than go to meta direct. Going to meta won't work for hbase;meta state. Puts load on Master. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java Change isTableDisabled/Enabled implementation to ask the Master instead. This will give the Master's TableStateManager's opinion rather than client figuring it for themselves reading meta table direct. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java TODO: Cleanup in here. Go to master for state, not to meta. M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java Logging cleanup. M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java Shutdown access. M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java Just cleanup. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java Add state holder for hbase:meta. Removed unused methods. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java Shut down access. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java Allow hbase:meta to be disabled. M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java Allow hbase:meta to be enabled. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve…
Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve… URL: https://github.com/apache/hbase/pull/435#issuecomment-536016215 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 15s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :blue_heart: | shelldocs | 0m 0s | Shelldocs was not available. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 35s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 58s | master passed | | :green_heart: | compile | 1m 51s | master passed | | :green_heart: | checkstyle | 2m 10s | master passed | | :green_heart: | shadedjars | 5m 5s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 24s | master passed | | :blue_heart: | spotbugs | 4m 47s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 6m 29s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 14s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 37s | the patch passed | | :green_heart: | compile | 1m 50s | the patch passed | | :green_heart: | javac | 1m 50s | the patch passed | | :broken_heart: | checkstyle | 1m 29s | hbase-server: The patch generated 1 new + 9 unchanged - 0 fixed = 10 total (was 9) | | :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 5m 15s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 18m 1s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 21s | the patch passed | | :broken_heart: | findbugs | 1m 3s | hbase-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | :green_heart: | unit | 3m 0s | hbase-common in the patch passed. | | :broken_heart: | unit | 294m 2s | hbase-server in the patch failed. | | :green_heart: | unit | 6m 57s | hbase-rsgroup in the patch passed. | | :green_heart: | asflicense | 1m 16s | The patch does not generate ASF License warnings. | | | | 378m 58s | | | Reason | Tests | |---:|:--| | FindBugs | module:hbase-common | | | org.apache.hadoop.hbase.Version.() invokes inefficient new String(String) constructor At Version.java:inefficient new String(String) constructor At Version.java:[line 10] | | Failed junit tests | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.util.TestFromClientSide3WoUnsafe | | | hadoop.hbase.tool.TestSecureBulkLoadHFiles | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/100/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/435 | | Optional Tests | dupname asflicense shellcheck shelldocs javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 1ca575527a14 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-435/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/100/artifact/out/diff-checkstyle-hbase-server.txt | | findbugs | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/100/artifact/out/new-findbugs-hbase-common.html | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/100/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/100/testReport/ | | Max. process+thread count | 4806 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server hbase-rsgroup U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/100/console | | versions |
[jira] [Commented] (HBASE-23082) Backport low-latency snapshot tracking for space quotas to 2.x
[
https://issues.apache.org/jira/browse/HBASE-23082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939585#comment-16939585
]
HBase QA commented on HBASE-23082:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 3m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:blue}0{color} | {color:blue} prototool {color} | {color:blue} 0m
1s{color} | {color:blue} prototool was not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 15 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
38s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
22s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
49s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
33s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 3m
39s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
6s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch passed checkstyle in hbase-protocol-shaded
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch passed checkstyle in hbase-hadoop-compat
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch passed checkstyle in hbase-hadoop2-compat
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} hbase-client: The patch generated 0 new + 2
unchanged - 1 fixed = 2 total (was 3) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
36s{color} | {color:red} hbase-server: The patch generated 7 new + 579
unchanged - 6 fixed = 586 total (was 585) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 2 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 4m
43s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
17m 32s{color} | {color:green} Patch does not cause any errors with Hadoop
2.8.5 2.9.2 or 3.1.2. {color} |
| {color:green}+1{color} | {color:green} hbaseprotoc {color} | {color:green}
2m 22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color}
[jira] [Updated] (HBASE-23055) Alter hbase:meta
[
https://issues.apache.org/jira/browse/HBASE-23055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Stack updated HBASE-23055:
--
Release Note:
Adds being able to edit hbase:meta table schema. For example,
hbase(main):005:0> disable 'hbase:meta'
Took 0.7362 seconds
hbase(main):006:0> alter 'hbase:meta', {NAME => 'info', DATA_BLOCK_ENCODING =>
'ROW_INDEX_V1'}
Updating all regions with the new schema...
All regions updated.
Done.
Took 1.2138 seconds
hbase(main):007:0> enable 'hbase:meta'
Took 0.7370 seconds
Known issue is if you try to describe tables while hbase:meta is offline,
you'll get scanner timeout as we read all table descriptors even if we only
want one (TODO: refactor how descriptor is done). When hbase:meta is offline,
other requests to meta will fail. If Master crashes while hbase:meta is
disabled, on new Master, hbase:meta will be enabled again.
Also has queries of table state go via the Master now since it knows status for
user-space and system-space tables. Previous it would ask the RS carrying
hbase:meta directly; i.e. client presumed implementation.
was:
Adds being able to edit hbase:meta table schema. For example,
hbase(main):005:0> disable 'hbase:meta'
Took 0.7362 seconds
hbase(main):006:0> alter 'hbase:meta', {NAME => 'info', DATA_BLOCK_ENCODING =>
'ROW_INDEX_V1'}
Updating all regions with the new schema...
All regions updated.
Done.
Took 1.2138 seconds
hbase(main):007:0> enable 'hbase:meta'
Took 0.7370 seconds
Known issue is if you try to describe tables while hbase:meta is offline,
you'll get scanner timeout as we read all table descriptors even if we only
want one (TODO: refactor how descriptor is done). When hbase:meta is offline,
other requests to meta will fail. If Master crashes while hbase:meta is
disabled, on new Master, hbase:meta will be enabled again.
> Alter hbase:meta
>
>
> Key: HBASE-23055
> URL: https://issues.apache.org/jira/browse/HBASE-23055
> Project: HBase
> Issue Type: Task
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
>
> hbase:meta is currently hardcoded. Its schema cannot be change.
> This issue is about allowing edits to hbase:meta schema. It will allow our
> being able to set encodings such as the block-with-indexes which will help
> quell CPU usage on host carrying hbase:meta. A dynamic hbase:meta is first
> step on road to being able to split meta.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] chenxu14 commented on issue #581: HBASE-22888 Share some stuffs with the initial reader when new stream reader created
chenxu14 commented on issue #581: HBASE-22888 Share some stuffs with the initial reader when new stream reader created URL: https://github.com/apache/hbase/pull/581#issuecomment-53634 Sorry, there are too many classes to review, Maybe I should split it into sub tasks? @Apache9 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] ZhaoBQ commented on a change in pull request #626: HBASE-23017 Verify the file integrity in persistent IOEngine
ZhaoBQ commented on a change in pull request #626: HBASE-23017 Verify the file
integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/626#discussion_r329143724
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/PersistentIOEngine.java
##
@@ -0,0 +1,119 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.hfile.bucket;
+
+import java.io.File;
+import java.io.IOException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.util.Shell;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A class implementing PersistentIOEngine interface supports file integrity
verification
+ * for {@link BucketCache} which use persistent IOEngine
+ */
+@InterfaceAudience.Private
+public abstract class PersistentIOEngine implements IOEngine {
+ private static final Logger LOG =
LoggerFactory.getLogger(PersistentIOEngine.class);
+ private static final DuFileCommand DU = new DuFileCommand(new String[]
{"du", ""});
+ protected final String[] filePaths;
+
+ public PersistentIOEngine(String[] filePaths) {
+this.filePaths = filePaths;
+ }
+
+ /**
+ * Verify cache files's integrity
+ * @param algorithm the backingMap persistence path
+ */
+ protected void verifyFileIntegrity(byte[] persistentChecksum, String
algorithm)
Review comment:
In branch-1 the method verifyFileIntegrity() return boolean type to indicate
whether the verification is successful. Here we throw IOE if verification
failed. Because the method parsePB() use this way.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] ZhaoBQ commented on a change in pull request #626: HBASE-23017 Verify the file integrity in persistent IOEngine
ZhaoBQ commented on a change in pull request #626: HBASE-23017 Verify the file
integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/626#discussion_r329140730
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/PersistentIOEngine.java
##
@@ -0,0 +1,119 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.hfile.bucket;
+
+import java.io.File;
+import java.io.IOException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.util.Shell;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A class implementing PersistentIOEngine interface supports file integrity
verification
+ * for {@link BucketCache} which use persistent IOEngine
+ */
+@InterfaceAudience.Private
+public abstract class PersistentIOEngine implements IOEngine {
+ private static final Logger LOG =
LoggerFactory.getLogger(PersistentIOEngine.class);
+ private static final DuFileCommand DU = new DuFileCommand(new String[]
{"du", ""});
+ protected final String[] filePaths;
+
+ public PersistentIOEngine(String[] filePaths) {
Review comment:
Because FileIOEngine can use multiple files as cache file.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] chenxu14 commented on a change in pull request #603: HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff
chenxu14 commented on a change in pull request #603: HBASE-22965 RS Crash due
to DBE reference to an reused ByteBuff
URL: https://github.com/apache/hbase/pull/603#discussion_r329138404
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java
##
@@ -760,5 +763,30 @@ public void testGetShortMidpoint() {
0, expectedArray.length);
}
+ @Test
+ public void testDBEShipped() throws IOException {
+for (DataBlockEncoding encoding : DataBlockEncoding.values()) {
+ DataBlockEncoder encoder = encoding.getEncoder();
+ if (encoder == null) {
+continue;
+ }
+ Path f = new Path(ROOT_DIR, testName.getMethodName() + "_" + encoding);
+ HFileContext context = new HFileContextBuilder()
+ .withIncludesTags(false)
+ .withDataBlockEncoding(encoding).build();
+ HFileWriterImpl writer = (HFileWriterImpl) HFile.getWriterFactory(conf,
cacheConf)
+ .withPath(fs, f).withFileContext(context).create();
+
+ KeyValue kv = new KeyValue(Bytes.toBytes("testkey"),
Bytes.toBytes("family"),
+Bytes.toBytes("qual"), Bytes.toBytes("testvalue"));
+ ByteBuffer buffer = ByteBuffer.wrap(kv.getBuffer());
+ ByteBufferKeyValue bbCell = new ByteBufferKeyValue(buffer, 0,
buffer.remaining());
+ writer.append(bbCell);
+ writer.beforeShipped();
+
+ Cell cell = writer.blockWriter.getEncodingState().getLastCell();
+ assertTrue(cell instanceof KeyValue);
Review comment:
thanks for the feedback
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #664: HBASE-23083 Collect Executor status info periodically and report to metric system
Apache-HBase commented on issue #664: HBASE-23083 Collect Executor status info periodically and report to metric system URL: https://github.com/apache/hbase/pull/664#issuecomment-535992986 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 36s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 9s | master passed | | :green_heart: | compile | 1m 20s | master passed | | :green_heart: | checkstyle | 1m 45s | master passed | | :green_heart: | shadedjars | 4m 37s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 58s | master passed | | :blue_heart: | spotbugs | 4m 5s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 52s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 6m 0s | the patch passed | | :green_heart: | compile | 1m 30s | the patch passed | | :green_heart: | javac | 1m 30s | the patch passed | | :green_heart: | checkstyle | 0m 28s | The patch passed checkstyle in hbase-common | | :green_heart: | checkstyle | 1m 33s | hbase-server: The patch generated 0 new + 64 unchanged - 9 fixed = 64 total (was 73) | | :broken_heart: | whitespace | 0m 0s | The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | :green_heart: | shadedjars | 4m 51s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 56s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 0s | the patch passed | | :green_heart: | findbugs | 5m 11s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 3m 10s | hbase-common in the patch passed. | | :green_heart: | unit | 164m 12s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 55s | The patch does not generate ASF License warnings. | | | | 231m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/664 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux e4e32d0ea9e7 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-664/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/2/artifact/out/whitespace-eol.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/2/testReport/ | | Max. process+thread count | 4984 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #652: HBASE-23035 Retain region to the last RegionServer make the failover …
Apache-HBase commented on issue #652: HBASE-23035 Retain region to the last RegionServer make the failover … URL: https://github.com/apache/hbase/pull/652#issuecomment-535975119 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 3m 37s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. | ||| _ master Compile Tests _ | | :green_heart: | mvninstall | 6m 2s | master passed | | :green_heart: | compile | 1m 0s | master passed | | :green_heart: | checkstyle | 1m 29s | master passed | | :green_heart: | shadedjars | 5m 3s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 40s | master passed | | :blue_heart: | spotbugs | 4m 42s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 40s | master passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 5m 44s | the patch passed | | :green_heart: | compile | 1m 0s | the patch passed | | :green_heart: | javac | 1m 0s | the patch passed | | :green_heart: | checkstyle | 1m 28s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 5m 1s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 18m 1s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 36s | the patch passed | | :green_heart: | findbugs | 4m 35s | the patch passed | ||| _ Other Tests _ | | :broken_heart: | unit | 230m 27s | hbase-server in the patch failed. | | :green_heart: | asflicense | 0m 27s | The patch does not generate ASF License warnings. | | | | 296m 43s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-652/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/652 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 9b9c2baf32a3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-652/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-652/4/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-652/4/testReport/ | | Max. process+thread count | 4718 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-652/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #657: Hbase 22932 Add rs group management methods in Admin and AsyncAdmin
Apache-HBase commented on issue #657: Hbase 22932 Add rs group management methods in Admin and AsyncAdmin URL: https://github.com/apache/hbase/pull/657#issuecomment-535974476 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 1s | No case conflicting files found. | | :blue_heart: | prototool | 0m 0s | prototool was not available. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 12 new or modified test files. | ||| _ HBASE-22514 Compile Tests _ | | :blue_heart: | mvndep | 0m 36s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 21s | HBASE-22514 passed | | :green_heart: | compile | 3m 23s | HBASE-22514 passed | | :green_heart: | checkstyle | 3m 7s | HBASE-22514 passed | | :green_heart: | shadedjars | 4m 46s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 2m 3s | HBASE-22514 passed | | :blue_heart: | spotbugs | 1m 33s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 9m 43s | HBASE-22514 passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 4s | the patch passed | | :green_heart: | compile | 3m 15s | the patch passed | | :green_heart: | cc | 3m 15s | the patch passed | | :green_heart: | javac | 3m 15s | the patch passed | | :broken_heart: | checkstyle | 0m 37s | hbase-client: The patch generated 9 new + 162 unchanged - 0 fixed = 171 total (was 162) | | :broken_heart: | checkstyle | 1m 23s | hbase-server: The patch generated 20 new + 89 unchanged - 2 fixed = 109 total (was 91) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 5m 1s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 16m 20s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | hbaseprotoc | 2m 58s | the patch passed | | :broken_heart: | javadoc | 0m 38s | hbase-server generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | :green_heart: | findbugs | 10m 27s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 0m 46s | hbase-protocol-shaded in the patch passed. | | :green_heart: | unit | 1m 56s | hbase-client in the patch passed. | | :broken_heart: | unit | 158m 4s | hbase-server in the patch failed. | | :green_heart: | unit | 3m 22s | hbase-thrift in the patch passed. | | :green_heart: | unit | 1m 10s | hbase-it in the patch passed. | | :green_heart: | asflicense | 2m 43s | The patch does not generate ASF License warnings. | | | | 253m 23s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.TestRegionServerReportForDuty | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/6/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/657 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux 3aa136cddf77 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-657/out/precommit/personality/provided.sh | | git revision | HBASE-22514 / 9bcb64bbe7 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/6/artifact/out/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/6/artifact/out/diff-checkstyle-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/6/artifact/out/diff-javadoc-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/6/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/6/testReport/ | | Max. process+thread count | 4546 (vs. ulimit of 1) |
[GitHub] [hbase] Apache-HBase commented on issue #663: HBASE-23078 BaseLoadBalancer should consider region replicas when ran…
Apache-HBase commented on issue #663: HBASE-23078 BaseLoadBalancer should consider region replicas when ran… URL: https://github.com/apache/hbase/pull/663#issuecomment-535969878 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :green_heart: | mvninstall | 5m 26s | master passed | | :green_heart: | compile | 0m 56s | master passed | | :green_heart: | checkstyle | 1m 21s | master passed | | :green_heart: | shadedjars | 4m 31s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 38s | master passed | | :blue_heart: | spotbugs | 4m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 3m 59s | master passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 4m 55s | the patch passed | | :green_heart: | compile | 0m 56s | the patch passed | | :green_heart: | javac | 0m 56s | the patch passed | | :green_heart: | checkstyle | 1m 19s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 32s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 30s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 35s | the patch passed | | :green_heart: | findbugs | 4m 10s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 157m 12s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 35s | The patch does not generate ASF License warnings. | | | | 213m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/663 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 44d0e592d884 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-663/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/3/testReport/ | | Max. process+thread count | 4855 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23038) Provide consistent and clear logging about disabling chores
[
https://issues.apache.org/jira/browse/HBASE-23038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939505#comment-16939505
]
Sean Busbey commented on HBASE-23038:
-
I agree that failure is unrelated and doesn't need to be handled here.
+1 on v3 as well.
> Provide consistent and clear logging about disabling chores
> ---
>
> Key: HBASE-23038
> URL: https://issues.apache.org/jira/browse/HBASE-23038
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
>Reporter: Sean Busbey
>Assignee: Sanjeet Nishad
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-23038.001.patch, HBASE-23038.002.patch,
> HBASE-23038.master.003.patch
>
>
> Right now if you want to disable any of our chores you can set the period to
> be <= 0. Sometimes, if you do this you get a nice message:
> {code}
> 2019-09-16 22:10:16,756 INFO [master-1:16000.activeMasterManager]
> master.HMaster: The period is 0 seconds, MobCompactionChore is disabled
> {code}
> And sometimes you get an opaque message:
> {code}
> 2019-09-16 22:09:45,333 INFO [master-1:16000.activeMasterManager]
> hbase.ChoreService: Could not successfully schedule chore: LogsCleaner
> 2019-09-16 22:09:45,340 INFO [master-1:16000.activeMasterManager]
> hbase.ChoreService: Could not successfully schedule chore: HFileCleaner
> {code}
> This is because sometimes we just blindly submit to ChoreService which
> submits to a java ScheduledExecutorService and then catches the
> IllegalArgumentException.
> We should remove the one-offs and make it so ChoreService checks the period
> before accepting a submittal and produces a consistent "Foo is disabled"
> message.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-23038) Provide consistent and clear logging about disabling chores
[
https://issues.apache.org/jira/browse/HBASE-23038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939491#comment-16939491
]
Sanjeet Nishad commented on HBASE-23038:
Hi [~busbey],[~vjasani]
The failing unit test seems to be unrelated to the current issue. Does that
need fixing up as a part of this Jira?
thanks.
> Provide consistent and clear logging about disabling chores
> ---
>
> Key: HBASE-23038
> URL: https://issues.apache.org/jira/browse/HBASE-23038
> Project: HBase
> Issue Type: Improvement
> Components: master, regionserver
>Reporter: Sean Busbey
>Assignee: Sanjeet Nishad
>Priority: Minor
> Labels: beginner
> Attachments: HBASE-23038.001.patch, HBASE-23038.002.patch,
> HBASE-23038.master.003.patch
>
>
> Right now if you want to disable any of our chores you can set the period to
> be <= 0. Sometimes, if you do this you get a nice message:
> {code}
> 2019-09-16 22:10:16,756 INFO [master-1:16000.activeMasterManager]
> master.HMaster: The period is 0 seconds, MobCompactionChore is disabled
> {code}
> And sometimes you get an opaque message:
> {code}
> 2019-09-16 22:09:45,333 INFO [master-1:16000.activeMasterManager]
> hbase.ChoreService: Could not successfully schedule chore: LogsCleaner
> 2019-09-16 22:09:45,340 INFO [master-1:16000.activeMasterManager]
> hbase.ChoreService: Could not successfully schedule chore: HFileCleaner
> {code}
> This is because sometimes we just blindly submit to ChoreService which
> submits to a java ScheduledExecutorService and then catches the
> IllegalArgumentException.
> We should remove the one-offs and make it so ChoreService checks the period
> before accepting a submittal and produces a consistent "Foo is disabled"
> message.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22012) SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable table
[
https://issues.apache.org/jira/browse/HBASE-22012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939486#comment-16939486
]
Hudson commented on HBASE-22012:
Results for branch master
[build #1485 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1485/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1485//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1485//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1485//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> SpaceQuota DisableTableViolationPolicy will cause cycles of enable/disable
> table
>
>
> Key: HBASE-22012
> URL: https://issues.apache.org/jira/browse/HBASE-22012
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.2.1
>Reporter: Ajeet Rai
>Assignee: Shardul Singh
>Priority: Major
> Labels: Quota, Space
> Fix For: 3.0.0, 2.3.0, 2.1.7, 2.2.2
>
>
> Space Quota: Policy state is getting changed from disable to Observance after
> sometime automatically.
> Steps:
> 1: Create a table with space quota policy as Disable
> 2: Put some data so that table state is in space quota violation
> 3: So observe that table state is in violation
> 4: Now wait for some time
> 5: Observe that after some time table state is changing to to Observance
> however table is still disabled
> edit (elserj): The table is automatically moved back from the violation state
> because of the code added that tried to ride over RITs. When a Region is not
> online (whether normally or abnormally), the RegionSizeReports are not sent
> from RS to Master. Eventually, enough Regions are not reported which dips
> below the acceptable threshold and we automatically move the table back to
> the "acceptable" space quota state (not in violation). We could skip this
> failsafe when we're checking for a quota that has the DisableTable violation
> policy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-22380) break circle replication when doing bulkload
[
https://issues.apache.org/jira/browse/HBASE-22380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939485#comment-16939485
]
Hudson commented on HBASE-22380:
Results for branch master
[build #1485 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/master/1485/]: (x)
*{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1485//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1485//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/master/1485//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> break circle replication when doing bulkload
>
>
> Key: HBASE-22380
> URL: https://issues.apache.org/jira/browse/HBASE-22380
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 3.0.0, 1.5.0, 2.2.0, 1.4.10, 2.0.5, 2.3.0, 2.1.5, 1.3.5
>Reporter: chenxu
>Assignee: Wellington Chevreuil
>Priority: Critical
> Labels: bulkload
> Fix For: 3.0.0, 1.5.0, 2.3.0, 2.1.7, 2.2.2, 1.4.12
>
> Attachments: HBASE-22380.branch-2.1.0001.patch,
> HBASE-22380.branch-2.1.0003.patch, HBASE-22380.branch-2.1.0004.patch
>
>
> when enabled master-master bulkload replication, HFiles will be replicated
> circularly between two clusters
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #650: HBASE-23035 Retain region to the last RegionServer make the failover …
Apache-HBase commented on issue #650: HBASE-23035 Retain region to the last RegionServer make the failover … URL: https://github.com/apache/hbase/pull/650#issuecomment-535933593 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 39s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | ||| _ branch-2.1 Compile Tests _ | | :green_heart: | mvninstall | 5m 10s | branch-2.1 passed | | :green_heart: | compile | 0m 55s | branch-2.1 passed | | :green_heart: | checkstyle | 1m 17s | branch-2.1 passed | | :green_heart: | shadedjars | 3m 57s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 36s | branch-2.1 passed | | :blue_heart: | spotbugs | 2m 44s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 2m 43s | branch-2.1 passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 4m 34s | the patch passed | | :green_heart: | compile | 0m 54s | the patch passed | | :green_heart: | javac | 0m 54s | the patch passed | | :green_heart: | checkstyle | 1m 18s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 3m 55s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 19m 32s | Patch does not cause any errors with Hadoop 2.7.7 2.8.5 or 3.0.3 3.1.2. | | :green_heart: | javadoc | 0m 34s | the patch passed | | :green_heart: | findbugs | 2m 38s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 138m 9s | hbase-server in the patch passed. | | :green_heart: | asflicense | 0m 28s | The patch does not generate ASF License warnings. | | | | 193m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-650/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/650 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux c42139cd7122 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-650/out/precommit/personality/provided.sh | | git revision | branch-2.1 / 23adf23a4f | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-650/4/testReport/ | | Max. process+thread count | 4124 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-650/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-connectors] meszibalu commented on issue #42: HBASE-22817 Use hbase-shaded dependencies in hbase-spark
meszibalu commented on issue #42: HBASE-22817 Use hbase-shaded dependencies in hbase-spark URL: https://github.com/apache/hbase-connectors/pull/42#issuecomment-535908719 Which leads to: `[Fri Sep 27 13:30:06 CEST 2019 DEBUG]: Maven: finish re-ordering modules [Fri Sep 27 13:30:06 CEST 2019 DEBUG]: Finished list: kafka kafka/hbase-kafka-model kafka/hbase-kafka-proxy spark spark/hbase-spark-protocol hbase-connectors-assembly . spark/hbase-spark spark/hbase-spark-it spark/hbase-spark-protocol-shaded` So hbase-spark-protocol* stuff is at the and of CHANGED_MODULES. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #664: HBASE-23083 Collect Executor status info periodically and report to metric system
Apache-HBase commented on issue #664: HBASE-23083 Collect Executor status info periodically and report to metric system URL: https://github.com/apache/hbase/pull/664#issuecomment-535907178 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 35s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 20s | master passed | | :green_heart: | compile | 1m 21s | master passed | | :green_heart: | checkstyle | 1m 45s | master passed | | :green_heart: | shadedjars | 4m 32s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 59s | master passed | | :blue_heart: | spotbugs | 4m 8s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 58s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 4m 51s | the patch passed | | :green_heart: | compile | 1m 20s | the patch passed | | :green_heart: | javac | 1m 20s | the patch passed | | :green_heart: | checkstyle | 0m 26s | The patch passed checkstyle in hbase-common | | :green_heart: | checkstyle | 1m 17s | hbase-server: The patch generated 0 new + 64 unchanged - 9 fixed = 64 total (was 73) | | :broken_heart: | whitespace | 0m 0s | The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | :green_heart: | shadedjars | 4m 35s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 35s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 56s | the patch passed | | :green_heart: | findbugs | 5m 15s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 3m 6s | hbase-common in the patch passed. | | :green_heart: | unit | 155m 3s | hbase-server in the patch passed. | | :green_heart: | asflicense | 1m 7s | The patch does not generate ASF License warnings. | | | | 220m 27s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/664 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d6d03048db29 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-664/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/1/artifact/out/whitespace-eol.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/1/testReport/ | | Max. process+thread count | 4879 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-664/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #666: Update stop-hbase.sh
Apache-HBase commented on issue #666: Update stop-hbase.sh URL: https://github.com/apache/hbase/pull/666#issuecomment-535905105 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 34s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :blue_heart: | shelldocs | 0m 0s | Shelldocs was not available. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | :broken_heart: | shellcheck | 0m 0s | The patch generated 1 new + 14 unchanged - 0 fixed = 15 total (was 14) | | :broken_heart: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | ||| _ Other Tests _ | | :green_heart: | asflicense | 0m 16s | The patch does not generate ASF License warnings. | | | | 2m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-666/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/666 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux ffe0b19d4d14 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-666/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | shellcheck | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-666/1/artifact/out/diff-patch-shellcheck.txt | | whitespace | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-666/1/artifact/out/whitespace-eol.txt | | Max. process+thread count | 44 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-666/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) shellcheck=0.7.0 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23085) Network and Data related Actions
Szabolcs Bukros created HBASE-23085: --- Summary: Network and Data related Actions Key: HBASE-23085 URL: https://issues.apache.org/jira/browse/HBASE-23085 Project: HBase Issue Type: Sub-task Components: integration tests Reporter: Szabolcs Bukros Assignee: Szabolcs Bukros Add additional actions to: * manipulate network packages with tc (reorder, loose,...) * add CPU load * fill the disk * corrupt or delete regionserver data files Create new monkey factories for the new actions. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Tikiri16 opened a new pull request #666: Update stop-hbase.sh
Tikiri16 opened a new pull request #666: Update stop-hbase.sh URL: https://github.com/apache/hbase/pull/666 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #655: HBASE-23055 Alter hbase:meta
Apache-HBase commented on issue #655: HBASE-23055 Alter hbase:meta URL: https://github.com/apache/hbase/pull/655#issuecomment-535900038 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 24s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 1s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 9 new or modified test files. | ||| _ branch-2 Compile Tests _ | | :blue_heart: | mvndep | 0m 13s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 48s | branch-2 passed | | :green_heart: | compile | 2m 3s | branch-2 passed | | :green_heart: | checkstyle | 2m 54s | branch-2 passed | | :green_heart: | shadedjars | 4m 34s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 1m 32s | branch-2 passed | | :blue_heart: | spotbugs | 0m 34s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 6m 3s | branch-2 passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 13s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 5m 22s | the patch passed | | :green_heart: | compile | 2m 2s | the patch passed | | :green_heart: | javac | 2m 2s | the patch passed | | :green_heart: | checkstyle | 0m 26s | The patch passed checkstyle in hbase-common | | :green_heart: | checkstyle | 0m 40s | hbase-client: The patch generated 0 new + 168 unchanged - 5 fixed = 168 total (was 173) | | :green_heart: | checkstyle | 0m 12s | The patch passed checkstyle in hbase-zookeeper | | :green_heart: | checkstyle | 1m 38s | hbase-server: The patch generated 0 new + 473 unchanged - 19 fixed = 473 total (was 492) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 31s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 16m 56s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 1m 31s | the patch passed | | :green_heart: | findbugs | 6m 13s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 2m 48s | hbase-common in the patch passed. | | :green_heart: | unit | 3m 31s | hbase-client in the patch passed. | | :green_heart: | unit | 0m 50s | hbase-zookeeper in the patch passed. | | :broken_heart: | unit | 233m 7s | hbase-server in the patch failed. | | :green_heart: | asflicense | 1m 36s | The patch does not generate ASF License warnings. | | | | 313m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-655/9/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/655 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux aa6bd83af383 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-655/out/precommit/personality/provided.sh | | git revision | branch-2 / 43b4ad9e8a | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-655/9/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-655/9/testReport/ | | Max. process+thread count | 4786 (vs. ulimit of 1) | | modules | C: hbase-common hbase-client hbase-zookeeper hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-655/9/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-22514) Move rsgroup feature into core of HBase
[
https://issues.apache.org/jira/browse/HBASE-22514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939329#comment-16939329
]
Hudson commented on HBASE-22514:
Results for branch HBASE-22514
[build #129 on
builds.a.o|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/129/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/129//General_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/129//JDK8_Nightly_Build_Report_(Hadoop2)/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/129//JDK8_Nightly_Build_Report_(Hadoop3)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for
details|https://builds.apache.org/job/HBase%20Nightly/job/HBASE-22514/129//artifact/output-integration/hadoop-2.log].
(note that this means we didn't run on Hadoop 3)
> Move rsgroup feature into core of HBase
> ---
>
> Key: HBASE-22514
> URL: https://issues.apache.org/jira/browse/HBASE-22514
> Project: HBase
> Issue Type: Umbrella
> Components: Admin, Client, rsgroup
>Reporter: Yechao Chen
>Assignee: Duo Zhang
>Priority: Major
> Attachments: HBASE-22514.master.001.patch,
> image-2019-05-31-18-25-38-217.png
>
>
> The class RSGroupAdminClient is not public
> we need to use java api RSGroupAdminClient to manager RSG
> so RSGroupAdminClient should be public
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #657: Hbase 22932 Add rs group management methods in Admin and AsyncAdmin
Apache-HBase commented on issue #657: Hbase 22932 Add rs group management methods in Admin and AsyncAdmin URL: https://github.com/apache/hbase/pull/657#issuecomment-535888317 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 1s | No case conflicting files found. | | :blue_heart: | prototool | 0m 0s | prototool was not available. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 12 new or modified test files. | ||| _ HBASE-22514 Compile Tests _ | | :blue_heart: | mvndep | 0m 36s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 13s | HBASE-22514 passed | | :green_heart: | compile | 3m 3s | HBASE-22514 passed | | :green_heart: | checkstyle | 3m 10s | HBASE-22514 passed | | :green_heart: | shadedjars | 4m 36s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 2m 1s | HBASE-22514 passed | | :blue_heart: | spotbugs | 1m 29s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 9m 14s | HBASE-22514 passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 4m 59s | the patch passed | | :green_heart: | compile | 3m 18s | the patch passed | | :green_heart: | cc | 3m 18s | the patch passed | | :green_heart: | javac | 3m 18s | the patch passed | | :broken_heart: | checkstyle | 0m 40s | hbase-client: The patch generated 11 new + 342 unchanged - 0 fixed = 353 total (was 342) | | :broken_heart: | checkstyle | 1m 19s | hbase-server: The patch generated 22 new + 89 unchanged - 2 fixed = 111 total (was 91) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 45s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 28s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | hbaseprotoc | 2m 55s | the patch passed | | :broken_heart: | javadoc | 0m 36s | hbase-server generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1) | | :green_heart: | findbugs | 10m 5s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 0m 42s | hbase-protocol-shaded in the patch passed. | | :green_heart: | unit | 1m 54s | hbase-client in the patch passed. | | :broken_heart: | unit | 158m 45s | hbase-server in the patch failed. | | :green_heart: | unit | 3m 42s | hbase-thrift in the patch passed. | | :green_heart: | unit | 1m 6s | hbase-it in the patch passed. | | :green_heart: | asflicense | 2m 27s | The patch does not generate ASF License warnings. | | | | 251m 11s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.regionserver.TestRegionServerReportForDuty | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/5/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/657 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc prototool | | uname | Linux 73abd25506c6 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-657/out/precommit/personality/provided.sh | | git revision | HBASE-22514 / 9bcb64bbe7 | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/5/artifact/out/diff-checkstyle-hbase-client.txt | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/5/artifact/out/diff-checkstyle-hbase-server.txt | | javadoc | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/5/artifact/out/diff-javadoc-javadoc-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/5/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-657/5/testReport/ | | Max. process+thread count | 4498 (vs. ulimit of 1) |
[jira] [Commented] (HBASE-22767) System table RIT STUCK if their RSGroup has no highest version RSes
[ https://issues.apache.org/jira/browse/HBASE-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939314#comment-16939314 ] Xiaolin Ha commented on HBASE-22767: Separating purging BOGUS_SERVER_NAME to HBASE-23080. > System table RIT STUCK if their RSGroup has no highest version RSes > --- > > Key: HBASE-22767 > URL: https://issues.apache.org/jira/browse/HBASE-22767 > Project: HBase > Issue Type: Bug > Components: rsgroup >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > AM chooses highest version region servers as participants for system tables, > including META table. If system table group has no highest version region > servers, then the reassignment of their regions will be always the BOGUS > server defined in RSGroup. > In our test environment using branch-2.2, we isolate system tables in a > rsgroup containing only one server. And when upgrading RSs, we have met the > problem that META is always assigned to the BOGUS server while the group > server has already been online for a while. META RIT is stuck and can not be > recovered by hbck2. > I made a UT reproduce this problem, steps are: > 1. add a group, move 1 server to it; > 2. move meta table to the group; > 3. restart the group server and downgrade its version; > 4. meta rit stuck. > > ROOT cause is AM filters highest version RSs for system tables. So if we do > not change the versions of system table group servers, but upgrade the > versions of other group servers, then if there is reassignment for any system > tables, such as balancer moving their regions, RIT STUCK!! > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #656: HBASE-23063 Add an option to enable multiget in parallel
Apache-HBase commented on issue #656: HBASE-23063 Add an option to enable multiget in parallel URL: https://github.com/apache/hbase/pull/656#issuecomment-535887545 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 35s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 16s | master passed | | :green_heart: | compile | 1m 20s | master passed | | :green_heart: | checkstyle | 1m 51s | master passed | | :green_heart: | shadedjars | 4m 33s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 58s | master passed | | :blue_heart: | spotbugs | 3m 56s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 41s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 4m 56s | the patch passed | | :green_heart: | compile | 1m 20s | the patch passed | | :green_heart: | javac | 1m 20s | the patch passed | | :green_heart: | checkstyle | 0m 27s | The patch passed checkstyle in hbase-common | | :green_heart: | checkstyle | 1m 23s | hbase-server: The patch generated 0 new + 192 unchanged - 6 fixed = 192 total (was 198) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 31s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 40s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 55s | the patch passed | | :green_heart: | findbugs | 5m 25s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 3m 1s | hbase-common in the patch passed. | | :green_heart: | unit | 156m 33s | hbase-server in the patch passed. | | :green_heart: | asflicense | 1m 6s | The patch does not generate ASF License warnings. | | | | 222m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-656/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/656 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 777a8b44ad28 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-656/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-656/4/testReport/ | | Max. process+thread count | 4579 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-656/4/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the file integrity in persistent IOEngine
anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the
file integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/626#discussion_r329010477
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
##
@@ -1053,6 +1066,7 @@ private void persistToFile() throws IOException {
}
try (FileOutputStream fos = new FileOutputStream(persistencePath, false)) {
fos.write(ProtobufMagic.PB_MAGIC);
+ checksum = ((PersistentIOEngine) ioEngine).calculateChecksum(algorithm);
BucketProtoUtils.toPB(this).writeDelimitedTo(fos);
Review comment:
The checksum can be passed as an argument. Keeping checksum as a state var
in BC does not look correct.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the file integrity in persistent IOEngine
anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the
file integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/626#discussion_r329012178
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/PersistentIOEngine.java
##
@@ -0,0 +1,119 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.hfile.bucket;
+
+import java.io.File;
+import java.io.IOException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.util.Shell;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A class implementing PersistentIOEngine interface supports file integrity
verification
+ * for {@link BucketCache} which use persistent IOEngine
+ */
+@InterfaceAudience.Private
+public abstract class PersistentIOEngine implements IOEngine {
+ private static final Logger LOG =
LoggerFactory.getLogger(PersistentIOEngine.class);
+ private static final DuFileCommand DU = new DuFileCommand(new String[]
{"du", ""});
+ protected final String[] filePaths;
+
+ public PersistentIOEngine(String[] filePaths) {
Review comment:
Why the arg not using String... ? U can just call super(filePath); from
FileMmapIOEngine ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the file integrity in persistent IOEngine
anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the
file integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/626#discussion_r329010959
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
##
@@ -1525,6 +1551,10 @@ float getMemoryFactor() {
return memoryFactor;
}
+ public byte[] getChecksum() {
Review comment:
Is this public getter needed then?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the file integrity in persistent IOEngine
anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the
file integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/626#discussion_r329010835
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
##
@@ -1131,6 +1155,8 @@ private void verifyCapacityAndClasses(long capacitySize,
String ioclass, String
}
private void parsePB(BucketCacheProtos.BucketCacheEntry proto) throws
IOException {
+((PersistentIOEngine)
ioEngine).verifyFileIntegrity(proto.getChecksum().toByteArray(),
Review comment:
For compatibility, u need to check whether the checksum is there or not. Use
proto.hasChecksum() before calling the getter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the file integrity in persistent IOEngine
anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the
file integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/626#discussion_r329010132
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
##
@@ -238,6 +239,17 @@
/** In-memory bucket size */
private float memoryFactor;
+ private static final String FILE_VERIFY_ALGORITHM =
+"hbase.bucketcache.persistent.file.integrity.check.algorithm";
+ private static final String DEFAULT_FILE_VERIFY_ALGORITHM = "MD5";
+
+ /**
+ * Use {@link java.security.MessageDigest} class's encryption algorithms to
check
+ * persistent file integrity, default algorithm is MD5
+ * */
+ private String algorithm;
+ private byte[] checksum;
Review comment:
Why to keep this as state in BC?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] sunhelly commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve…
sunhelly commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve… URL: https://github.com/apache/hbase/pull/435#issuecomment-535886907 Ping @saintstack for reviewing. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the file integrity in persistent IOEngine
anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the
file integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/626#discussion_r329010044
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
##
@@ -1067,7 +1081,9 @@ private void retrieveFromFile(int[] bucketSizes) throws
IOException {
}
assert !cacheEnabled;
-try (FileInputStream in = deleteFileOnClose(persistenceFile)) {
Review comment:
Why this change?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the file integrity in persistent IOEngine
anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the
file integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/626#discussion_r329012449
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/PersistentIOEngine.java
##
@@ -0,0 +1,119 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io.hfile.bucket;
+
+import java.io.File;
+import java.io.IOException;
+import java.security.MessageDigest;
+import java.security.NoSuchAlgorithmException;
+
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.hadoop.util.Shell;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A class implementing PersistentIOEngine interface supports file integrity
verification
+ * for {@link BucketCache} which use persistent IOEngine
+ */
+@InterfaceAudience.Private
+public abstract class PersistentIOEngine implements IOEngine {
+ private static final Logger LOG =
LoggerFactory.getLogger(PersistentIOEngine.class);
+ private static final DuFileCommand DU = new DuFileCommand(new String[]
{"du", ""});
+ protected final String[] filePaths;
+
+ public PersistentIOEngine(String[] filePaths) {
+this.filePaths = filePaths;
+ }
+
+ /**
+ * Verify cache files's integrity
+ * @param algorithm the backingMap persistence path
+ */
+ protected void verifyFileIntegrity(byte[] persistentChecksum, String
algorithm)
Review comment:
Whether this in line with the branch-1 patch now?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the file integrity in persistent IOEngine
anoopsjohn commented on a change in pull request #626: HBASE-23017 Verify the
file integrity in persistent IOEngine
URL: https://github.com/apache/hbase/pull/626#discussion_r329009236
##
File path: hbase-protocol-shaded/src/main/protobuf/BucketCacheEntry.proto
##
@@ -26,11 +26,12 @@ option java_generate_equals_and_hash = true;
option optimize_for = SPEED;
message BucketCacheEntry {
- required int64 cache_capacity = 1;
- required string io_class = 2;
- required string map_class = 3;
- map deserializers = 4;
- required BackingMap backing_map = 5;
+ optional bytes checksum = 1;
Review comment:
Put checksum bytes as the last entry. Do not change the existing order.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (HBASE-23084) Assertion "Added a key not lexically larger than previous" or "prevKey followed by a smaller key" caused abortion of RS
[
https://issues.apache.org/jira/browse/HBASE-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939305#comment-16939305
]
Duo Zhang commented on HBASE-23084:
---
What is the version of HBase?
> Assertion "Added a key not lexically larger than previous" or "prevKey
> followed by a smaller key" caused abortion of RS
> --
>
> Key: HBASE-23084
> URL: https://issues.apache.org/jira/browse/HBASE-23084
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Environment: We store many phoenix tables in hbase, only one table
> trigger this issues.
> When two jacent keys in wrong order, the observation as follows.
> 1. the prevKey is written into hbase via Put rpc
> 2. the current key via phoenix upsert statment( Multi rpc in underlying hbase)
> 3. the current Cell extends 1MB.
> 4. the error is easy to re-ocurr.
>Reporter: ranpanfeng
>Priority: Critical
>
> when auto flush and manual flush is invoked, cell in immutable memstore
> segment is scanned one by one to build data block and bloom filter block,
> during this time, two adjacent keys are compared to assert that they are in
> lexicographical order, but regretfully, the assertion fails in some
> scenarios, when RS encounter these assertion failure, always call abort
> function to terminate abnormally.
>
> the version is 0ba7cc01a13dbb7fec8e6c5d599fc8b4936bae61 tag: rel/2.1.5
>
> At least three cases are found.
> *error case#1: getQualifierArray throws
> java.lang.ArrayIndexOutOfBoundsException*
>
> {code:java}
> 17:57:50.328 [MemStoreFlusher.0] ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer - * ABORTING region
> server ${regionserver}: Replay of WAL required. Forcing server shutdown *
> org.apache.hadoop.hbase.DroppedSnapshotException: region: ${region}.
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2788)
> [hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2467)
> [hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2439)
> [hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2329)
> [hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612)
> [hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581)
> [hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68)
> [hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361)
> [hbase-server-2.1.5.jar:2.1.5]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
> Caused by: java.lang.ArrayIndexOutOfBoundsException
> at
> org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1155)
> ~[hbase-common-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.CellUtil.copyQualifierTo(CellUtil.java:309)
> ~[hbase-common-2.1.5.jar:2.1.5]
> at org.apache.hadoop.hbase.CellUtil.cloneQualifier(CellUtil.java:121)
> ~[hbase-common-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.ByteBufferKeyValue.getQualifierArray(ByteBufferKeyValue.java:112)
> ~[hbase-common-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.CellUtil.getCellKeyAsString(CellUtil.java:1306)
> ~[hbase-common-2.1.5.jar:2.1.5]
> at org.apache.hadoop.hbase.CellUtil.toString(CellUtil.java:1483)
> ~[hbase-common-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.ByteBufferKeyValue.toString(ByteBufferKeyValue.java:296)
> ~[hbase-common-2.1.5.jar:2.1.5]
> at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_144]
> at java.lang.StringBuilder.append(StringBuilder.java:131)
> ~[?:1.8.0_144]
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.checkScanOrder(StoreScanner.java:1007)
> ~[hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:582)
> ~[hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:127)
> ~[hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:69)
> ~[hbase-server-2.1.5.jar:2.1.5]
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1021)
>
[GitHub] [hbase-connectors] meszibalu edited a comment on issue #42: HBASE-22817 Use hbase-shaded dependencies in hbase-spark
meszibalu edited a comment on issue #42: HBASE-22817 Use hbase-shaded dependencies in hbase-spark URL: https://github.com/apache/hbase-connectors/pull/42#issuecomment-535883384 According to `maven-branch-dirlist-root.txt`, yetus fails at the beginning: `[ERROR] Failed to execute goal on project hbase-kafka-proxy: Could not resolve dependencies for project org.apache.hbase.connectors.kafka:hbase-kafka-proxy:jar:1.0.1-SNAPSHOT: Could not find artifact org.apache.hbase.connectors.kafka:hbase-kafka-model:jar:1.0.1-SNAPSHOT in Nexus (http://repository.apache.org/snapshots) -> [Help 1] [ERROR] Failed to execute goal on project hbase-spark-it: Could not resolve dependencies for project org.apache.hbase.connectors.spark:hbase-spark-it:jar:1.0.1-SNAPSHOT: Could not find artifact org.apache.hbase.connectors.spark:hbase-spark:jar:1.0.1-SNAPSHOT in Nexus (http://repository.apache.org/snapshots) -> [Help 1]` This is the first step which is executed by yetus. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase-connectors] meszibalu commented on issue #42: HBASE-22817 Use hbase-shaded dependencies in hbase-spark
meszibalu commented on issue #42: HBASE-22817 Use hbase-shaded dependencies in hbase-spark URL: https://github.com/apache/hbase-connectors/pull/42#issuecomment-535883384 According to `maven-branch-dirlist-root.txt`, yetus fails at the beginning: `[ERROR] Failed to execute goal on project hbase-kafka-proxy: Could not resolve dependencies for project org.apache.hbase.connectors.kafka:hbase-kafka-proxy:jar:1.0.1-SNAPSHOT: Could not find artifact org.apache.hbase.connectors.kafka:hbase-kafka-model:jar:1.0.1-SNAPSHOT in Nexus (http://repository.apache.org/snapshots) -> [Help 1] [ERROR] Failed to execute goal on project hbase-spark-it: Could not resolve dependencies for project org.apache.hbase.connectors.spark:hbase-spark-it:jar:1.0.1-SNAPSHOT: Could not find artifact org.apache.hbase.connectors.spark:hbase-spark:jar:1.0.1-SNAPSHOT in Nexus (http://repository.apache.org/snapshots) -> [Help 1]` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve…
Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve… URL: https://github.com/apache/hbase/pull/435#issuecomment-535882928 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 0s | Docker mode activated. | | :broken_heart: | patch | 0m 11s | https://github.com/apache/hbase/pull/435 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/435 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/99/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] anoopsjohn commented on a change in pull request #603: HBASE-22965 RS Crash due to DBE reference to an reused ByteBuff
anoopsjohn commented on a change in pull request #603: HBASE-22965 RS Crash due
to DBE reference to an reused ByteBuff
URL: https://github.com/apache/hbase/pull/603#discussion_r329007832
##
File path:
hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java
##
@@ -760,5 +763,30 @@ public void testGetShortMidpoint() {
0, expectedArray.length);
}
+ @Test
+ public void testDBEShipped() throws IOException {
+for (DataBlockEncoding encoding : DataBlockEncoding.values()) {
+ DataBlockEncoder encoder = encoding.getEncoder();
+ if (encoder == null) {
+continue;
+ }
+ Path f = new Path(ROOT_DIR, testName.getMethodName() + "_" + encoding);
+ HFileContext context = new HFileContextBuilder()
+ .withIncludesTags(false)
+ .withDataBlockEncoding(encoding).build();
+ HFileWriterImpl writer = (HFileWriterImpl) HFile.getWriterFactory(conf,
cacheConf)
+ .withPath(fs, f).withFileContext(context).create();
+
+ KeyValue kv = new KeyValue(Bytes.toBytes("testkey"),
Bytes.toBytes("family"),
+Bytes.toBytes("qual"), Bytes.toBytes("testvalue"));
+ ByteBuffer buffer = ByteBuffer.wrap(kv.getBuffer());
+ ByteBufferKeyValue bbCell = new ByteBufferKeyValue(buffer, 0,
buffer.remaining());
+ writer.append(bbCell);
+ writer.beforeShipped();
+
+ Cell cell = writer.blockWriter.getEncodingState().getLastCell();
+ assertTrue(cell instanceof KeyValue);
Review comment:
The UT is not really doing a meaningful test. Write a cell first and then
pollute its backing ByteBuffer by directly manipulating few bytes which comes
at some KeyLen part or so. Then write another cell. Call the beforeShipped()
before manipulating this 1st BB. With out the patch, we will end up in some
issues. The patch should fix it.
Even we can remove all these new getters exposed in different classes for
testing.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve…
Apache-HBase commented on issue #435: HBASE-22767 System table RIT STUCK if their RSGroup has no highest ve… URL: https://github.com/apache/hbase/pull/435#issuecomment-535879974 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 0s | Docker mode activated. | | :broken_heart: | patch | 0m 12s | https://github.com/apache/hbase/pull/435 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/435 | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-435/98/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Issue Comment Deleted] (HBASE-22767) System table RIT STUCK if their RSGroup has no highest version RSes
[ https://issues.apache.org/jira/browse/HBASE-22767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaolin Ha updated HBASE-22767: --- Comment: was deleted (was: Here version stands for package version. System tables will always be assigned to servers of highest version, but this design will not work in rsgroup, because if system tables are isolated in a rsgroup without highest version servers, there will be RIT stucks of their regions. I approve of removing BOGUS server, but we still need to find a server for regions whose group has no online servers. [~zghaobac] suggested to randomly choose servers from DEFAULT group. But still some works of HBASE-22514 need to be done before implement this idea ? Or we need to fix this issue on other branches? ) > System table RIT STUCK if their RSGroup has no highest version RSes > --- > > Key: HBASE-22767 > URL: https://issues.apache.org/jira/browse/HBASE-22767 > Project: HBase > Issue Type: Bug > Components: rsgroup >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > > AM chooses highest version region servers as participants for system tables, > including META table. If system table group has no highest version region > servers, then the reassignment of their regions will be always the BOGUS > server defined in RSGroup. > In our test environment using branch-2.2, we isolate system tables in a > rsgroup containing only one server. And when upgrading RSs, we have met the > problem that META is always assigned to the BOGUS server while the group > server has already been online for a while. META RIT is stuck and can not be > recovered by hbck2. > I made a UT reproduce this problem, steps are: > 1. add a group, move 1 server to it; > 2. move meta table to the group; > 3. restart the group server and downgrade its version; > 4. meta rit stuck. > > ROOT cause is AM filters highest version RSs for system tables. So if we do > not change the versions of system table group servers, but upgrade the > versions of other group servers, then if there is reassignment for any system > tables, such as balancer moving their regions, RIT STUCK!! > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] Apache-HBase commented on issue #665: Update start-hbase.sh
Apache-HBase commented on issue #665: Update start-hbase.sh URL: https://github.com/apache/hbase/pull/665#issuecomment-535877325 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :blue_heart: | shelldocs | 0m 0s | Shelldocs was not available. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | ||| _ Patch Compile Tests _ | | :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | ||| _ Other Tests _ | | :green_heart: | asflicense | 0m 17s | The patch does not generate ASF License warnings. | | | | 2m 41s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-665/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/665 | | Optional Tests | dupname asflicense shellcheck shelldocs | | uname | Linux 925ce572ad76 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-665/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Max. process+thread count | 47 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-665/1/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) shellcheck=0.7.0 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (HBASE-23079) SCP fails when PE executes rollback of ORP
[
https://issues.apache.org/jira/browse/HBASE-23079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939283#comment-16939283
]
Xiaolin Ha commented on HBASE-23079:
No logs can help to find out what happened on RS. Will be back.
> SCP fails when PE executes rollback of ORP
> ---
>
> Key: HBASE-23079
> URL: https://issues.apache.org/jira/browse/HBASE-23079
> Project: HBase
> Issue Type: Bug
>Reporter: Xiaolin Ha
>Assignee: Xiaolin Ha
>Priority: Major
>
> In our test environment, we found that, when OpenRegionProcedure failed by
> timeout, though it doesn't support rollback, ProcedureExcutor will delete the
> OPR node in the store in its own rollback steps and make the RIT stay there
> looping rollback exception. ServerCrashProcedure will detect this RIT but
> cannot deal with it, because no node in the store and it will encounters NPE,
> as a result, SCP aborts.
>
> Logs are as follows,
> {quote}2019-08-29,07:27:35,710 INFO [PEWorker-15]
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure: pid=70206,
> state=RUNNABLE:SERVER_CRASH_ASSIGN, locked=true; ServerCrashProcedure
> server=c3-hadoop-srv-st297.bj,21600,1567012736142, splitWal=true, meta=true
> found RIT pid=56043, ppid=55625, state=ROLLEDBACK;
> TransitRegionStateProcedure table=c3srv_galaxy:tsdb,
> region=c1ccd94593bf1b87269cec98d6ffaaae, ASSIGN; rit=OPEN,
> location=c3-hadoop-srv-st297.bj,21600,1567012736142, table=c3srv_galaxy:tsdb,
> region=c1ccd94593bf1b87269cec98d6ffaaae
> 2019-08-29,07:27:35,711 ERROR [PEWorker-15]
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception: pid=70206, state=RUNNABLE:SERVER_CRASH_ASSIGN,
> locked=true; ServerCrashProcedure
> server=c3-hadoop-srv-st297.bj,21600,1567012736142, splitWal=true, meta=true
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker.update(ProcedureStoreTracker.java:140)
> at
> org.apache.hadoop.hbase.procedure2.store.ProcedureStoreTracker.update(ProcedureStoreTracker.java:133)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.updateStoreTracker(WALProcedureStore.java:782)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.pushData(WALProcedureStore.java:737)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.update(WALProcedureStore.java:604)
> at
> org.apache.hadoop.hbase.master.assignment.RegionRemoteProcedureBase.persistAndWake(RegionRemoteProcedureBase.java:182)
> at
> org.apache.hadoop.hbase.master.assignment.RegionRemoteProcedureBase.serverCrashed(RegionRemoteProcedureBase.java:239)
> at
> org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure.serverCrashed(TransitRegionStateProcedure.java:398)
> at
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.assignRegions(ServerCrashProcedure.java:461)
> at
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.executeFromState(ServerCrashProcedure.java:221)
> at
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.executeFromState(ServerCrashProcedure.java:64)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:189)
> at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1645)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:78)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1962)
> 2019-08-29,07:27:35,714 ERROR [PEWorker-15]
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor: CODE-BUG: Uncaught
> runtime exception for pid=70206, state=FAILED:SERVER_CRASH_SPLIT_LOGS,
> locked=true, exception=java.lang.NullPointerException via CODE-BUG: Uncaught
> runtime exception: pid=70206, state=RUNNABLE:SERVER_CRASH_ASSIGN,
> locked=true; ServerCrashProcedure
> server=c3-hadoop-srv-st297.bj,21600,1567012736142, splitWal=true,
> meta=true:java.lang.NullPointerException; ServerCrashProcedure
> server=c3-hadoop-srv-st297.bj,21600,1567012736142, splitWal=true, meta=true
> java.lang.UnsupportedOperationException: unhandled state=SERVER_CRASH_ASSIGN
> at
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.rollbackState(ServerCrashProcedure.java:333)
> at
> org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure.rollbackState(ServerCrashProcedure.java:64)
> at
> org.apache.hadoop.hbase.procedure2.StateMachineProcedure.rollback(StateMachineProcedure.java:208)
> at
>
[GitHub] [hbase] Tikiri16 opened a new pull request #665: Update start-hbase.sh
Tikiri16 opened a new pull request #665: Update start-hbase.sh URL: https://github.com/apache/hbase/pull/665 removed extra '=' sign which gave an error: "55: [: false: unexpected operator" This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] sunhelly commented on a change in pull request #657: Hbase 22932 Add rs group management methods in Admin and AsyncAdmin
sunhelly commented on a change in pull request #657: Hbase 22932 Add rs group management methods in Admin and AsyncAdmin URL: https://github.com/apache/hbase/pull/657#discussion_r328926634 ## File path: hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java ## @@ -2254,4 +2256,70 @@ boolean snapshotCleanupSwitch(final boolean on, final boolean synchronous) */ boolean isSnapshotCleanupEnabled() throws IOException; + /** + * Gets group info for the given group name + * @param groupName the group name + * @return group info + * @throws IOException if a remote or network exception occurs + */ + RSGroupInfo getRSGroupInfo(String groupName) throws IOException; Review comment: Yes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] brfrn169 commented on issue #647: HBASE-22988 Backport HBASE-11062 "hbtop" to branch-1
brfrn169 commented on issue #647: HBASE-22988 Backport HBASE-11062 "hbtop" to branch-1 URL: https://github.com/apache/hbase/pull/647#issuecomment-535860663 Actually, I'm not sure if we can go ahead and merge the patch or not as the compile and javac errors occurred in the last QA. @Reidd @apurtell This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (HBASE-23084) Assertion "Added a key not lexically larger than previous" or "prevKey followed by a smaller key" caused abortion of RS
ranpanfeng created HBASE-23084:
--
Summary: Assertion "Added a key not lexically larger than
previous" or "prevKey followed by a smaller key" caused abortion of RS
Key: HBASE-23084
URL: https://issues.apache.org/jira/browse/HBASE-23084
Project: HBase
Issue Type: Bug
Components: regionserver
Environment: We store many phoenix tables in hbase, only one table
trigger this issues.
When two jacent keys in wrong order, the observation as follows.
1. the prevKey is written into hbase via Put rpc
2. the current key via phoenix upsert statment( Multi rpc in underlying hbase)
3. the current Cell extends 1MB.
4. the error is easy to re-ocurr.
Reporter: ranpanfeng
when auto flush and manual flush is invoked, cell in immutable memstore
segment is scanned one by one to build data block and bloom filter block,
during this time, two adjacent keys are compared to assert that they are in
lexicographical order, but regretfully, the assertion fails in some scenarios,
when RS encounter these assertion failure, always call abort function to
terminate abnormally.
the version is 0ba7cc01a13dbb7fec8e6c5d599fc8b4936bae61 tag: rel/2.1.5
At least three cases are found.
*error case#1: getQualifierArray throws
java.lang.ArrayIndexOutOfBoundsException*
{code:java}
17:57:50.328 [MemStoreFlusher.0] ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer - * ABORTING region
server ${regionserver}: Replay of WAL required. Forcing server shutdown *
org.apache.hadoop.hbase.DroppedSnapshotException: region: ${region}.
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2788)
[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2467)
[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:2439)
[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:2329)
[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:612)
[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.flushRegion(MemStoreFlusher.java:581)
[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher.access$1000(MemStoreFlusher.java:68)
[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.MemStoreFlusher$FlushHandler.run(MemStoreFlusher.java:361)
[hbase-server-2.1.5.jar:2.1.5]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_144]
Caused by: java.lang.ArrayIndexOutOfBoundsException
at
org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1155)
~[hbase-common-2.1.5.jar:2.1.5]
at org.apache.hadoop.hbase.CellUtil.copyQualifierTo(CellUtil.java:309)
~[hbase-common-2.1.5.jar:2.1.5]
at org.apache.hadoop.hbase.CellUtil.cloneQualifier(CellUtil.java:121)
~[hbase-common-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.ByteBufferKeyValue.getQualifierArray(ByteBufferKeyValue.java:112)
~[hbase-common-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.CellUtil.getCellKeyAsString(CellUtil.java:1306)
~[hbase-common-2.1.5.jar:2.1.5]
at org.apache.hadoop.hbase.CellUtil.toString(CellUtil.java:1483)
~[hbase-common-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.ByteBufferKeyValue.toString(ByteBufferKeyValue.java:296)
~[hbase-common-2.1.5.jar:2.1.5]
at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_144]
at java.lang.StringBuilder.append(StringBuilder.java:131) ~[?:1.8.0_144]
at
org.apache.hadoop.hbase.regionserver.StoreScanner.checkScanOrder(StoreScanner.java:1007)
~[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:582)
~[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.StoreFlusher.performFlush(StoreFlusher.java:127)
~[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.DefaultStoreFlusher.flushSnapshot(DefaultStoreFlusher.java:69)
~[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:1021)
~[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.HStore$StoreFlusherImpl.flushCache(HStore.java:2259)
~[hbase-server-2.1.5.jar:2.1.5]
at
org.apache.hadoop.hbase.regionserver.HRegion.internalFlushCacheAndCommit(HRegion.java:2733)
~[hbase-server-2.1.5.jar:2.1.5]
... 8 more
17:57:50.332 [MemStoreFlusher.0] ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer - RegionServer abort: loaded
coprocessors are: [org.apache.phoenix.coprocessor.ScanRegionObserver,
[GitHub] [hbase] Apache-HBase commented on issue #652: HBASE-23035 Retain region to the last RegionServer make the failover …
Apache-HBase commented on issue #652: HBASE-23035 Retain region to the last RegionServer make the failover … URL: https://github.com/apache/hbase/pull/652#issuecomment-535844090 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 3m 30s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 7 new or modified test files. | ||| _ master Compile Tests _ | | :green_heart: | mvninstall | 5m 38s | master passed | | :green_heart: | compile | 0m 54s | master passed | | :green_heart: | checkstyle | 1m 17s | master passed | | :green_heart: | shadedjars | 4m 46s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 40s | master passed | | :blue_heart: | spotbugs | 4m 11s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 9s | master passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 5m 11s | the patch passed | | :green_heart: | compile | 0m 59s | the patch passed | | :green_heart: | javac | 0m 59s | the patch passed | | :broken_heart: | checkstyle | 1m 18s | hbase-server: The patch generated 3 new + 2 unchanged - 0 fixed = 5 total (was 2) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 48s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 16m 33s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 38s | the patch passed | | :green_heart: | findbugs | 4m 38s | the patch passed | ||| _ Other Tests _ | | :broken_heart: | unit | 331m 56s | hbase-server in the patch failed. | | :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 394m 11s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestSnapshotTemporaryDirectoryWithRegionReplicas | | | hadoop.hbase.tool.TestSecureBulkLoadHFiles | | | hadoop.hbase.replication.TestReplicationKillSlaveRSWithSeparateOldWALs | | | hadoop.hbase.client.TestSnapshotTemporaryDirectory | | | hadoop.hbase.replication.TestReplicationSmallTestsSync | | | hadoop.hbase.tool.TestBulkLoadHFiles | | | hadoop.hbase.client.TestAsyncTableAdminApi | | | hadoop.hbase.regionserver.TestRegionServerAbortTimeout | | | hadoop.hbase.replication.TestReplicationDisableInactivePeer | | | hadoop.hbase.replication.TestReplicationSmallTests | | | hadoop.hbase.client.TestCloneSnapshotFromClientNormal | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-652/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/652 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 1a2feb9e656a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-652/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | checkstyle | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-652/3/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-652/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-652/3/testReport/ | | Max. process+thread count | 4950 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-652/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For
[jira] [Commented] (HBASE-23083) Collect Executor status info periodically and report to metrics system
[ https://issues.apache.org/jira/browse/HBASE-23083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939200#comment-16939200 ] chenxu commented on HBASE-23083: FYI [~stack] > Collect Executor status info periodically and report to metrics system > -- > > Key: HBASE-23083 > URL: https://issues.apache.org/jira/browse/HBASE-23083 > Project: HBase > Issue Type: Improvement >Reporter: chenxu >Assignee: chenxu >Priority: Major > > HRegionServer#startServiceThreads will start some Executors, but we don't > have a good way to know their status, such as how many threads pending, and > how many threads running. Can add an ScheduledChore to collect the > information periodically and report to metrics system. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [hbase] chenxu14 opened a new pull request #664: HBASE-23083 Collect Executor status info periodically and report to
chenxu14 opened a new pull request #664: HBASE-23083 Collect Executor status info periodically and report to URL: https://github.com/apache/hbase/pull/664 metrics system This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [hbase] Apache-HBase commented on issue #663: HBASE-23078 BaseLoadBalancer should consider region replicas when ran…
Apache-HBase commented on issue #663: HBASE-23078 BaseLoadBalancer should consider region replicas when ran… URL: https://github.com/apache/hbase/pull/663#issuecomment-535826643 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 2m 28s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ master Compile Tests _ | | :green_heart: | mvninstall | 8m 3s | master passed | | :green_heart: | compile | 1m 14s | master passed | | :green_heart: | checkstyle | 1m 40s | master passed | | :green_heart: | shadedjars | 6m 15s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 52s | master passed | | :blue_heart: | spotbugs | 5m 22s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 5m 20s | master passed | ||| _ Patch Compile Tests _ | | :green_heart: | mvninstall | 7m 27s | the patch passed | | :green_heart: | compile | 1m 18s | the patch passed | | :green_heart: | javac | 1m 18s | the patch passed | | :green_heart: | checkstyle | 1m 38s | the patch passed | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 6m 16s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 23m 2s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 48s | the patch passed | | :green_heart: | findbugs | 6m 27s | the patch passed | ||| _ Other Tests _ | | :broken_heart: | unit | 240m 23s | hbase-server in the patch failed. | | :green_heart: | asflicense | 0m 26s | The patch does not generate ASF License warnings. | | | | 322m 18s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.master.TestRegionPlacement2 | | | hadoop.hbase.master.TestMasterShutdown | | | hadoop.hbase.master.balancer.TestBaseLoadBalancer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/663 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux b9e768eaa24c 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-663/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/2/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/2/testReport/ | | Max. process+thread count | 5097 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-663/2/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (HBASE-23062) Use TableInputFormat to read data from Hbase, when Scan.setCaching(size) the size is too big, some rowkeys will lost without exctpions.
[
https://issues.apache.org/jira/browse/HBASE-23062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhanxiongWang updated HBASE-23062:
--
Description:
Hbase server cluster version is 1.2.5 . Data will be lost when the hbase client
version is 0.98.6.1. When the hbase client version is 4, no data will be lost.
I did the experiment in three ways. One way I use spark to read hbase, second
way I use mapreduce to read hbase. In both cases, when I increase the Scan
Caching size, some data will be lost. To be more accurately, When I set
scan.setCaching(500), I can receive 7622 rows of data, but when I set
scan.setCaching(5), I can receive only 4226 rows of data. Third way I use
Scan to read hbase directly, caching size does not affect the result, I can
always receive 7622 rows of data.
The seriousness of the problem is that the data is lost but there is no
exceptions, it is difficult to find the reason.
My spark code is like this:
{code:java}
Configuration hbaseConfiguration = HBaseConfiguration.create();
hbaseConfiguration.set("hbase.zookeeper.property.clientPort", zkPort);
hbaseConfiguration.set("hbase.zookeeper.quorum", zkMaster);
hbaseConfiguration.set("zookeeper.znode.parent", zkPath);
hbaseConfiguration.set(TableInputFormat.INPUT_TABLE,hbaseTableName);
hbaseConfiguration.setLong("hbase.client.scanner.timeout.period",600);
hbaseConfiguration.setLong("hbase.rpc.timeout",600);
final Scan hbaseScan = new Scan();
hbaseScan.addFamily(familyName);
hbaseScan.setCaching(5);//if Caching is too big, some rowkeys will lost!
for(String[] cell:cellNames){
String column = cell[0];
hbaseScan.addColumn(familyName,Bytes.toBytes(column));
}
hbaseScan.setStartRow(Bytes.toBytes(startRowkeyStr));
hbaseScan.setStopRow(Bytes.toBytes(endRowkeyStr));
try {
ClientProtos.Scan scanProto = ProtobufUtil.toScan(hbaseScan);
hbaseConfiguration.set(TableInputFormat.SCAN,
Base64.encodeBytes(scanProto.toByteArray()));
JavaPairRDD pairRDD =
jsc.newAPIHadoopRDD(
hbaseConfiguration,TableInputFormat.class, ImmutableBytesWritable.class,
Result.class );
System.out.println("pairRDD.count(): " + pairRDD.count());
}
catch (IOException e) {
System.out.println("Scan Exception!! " + e.getMessage());
}
{code}
My mapreduce code is like this:
{code:java}
static class HbaseMapper extends TableMapper {
@Override protected void map(ImmutableBytesWritable key, Result
value,Mapper.Context context) throws IOException, InterruptedException {
for(Cell cell :value.rawCells()){
context.write(new ImmutableBytesWritable("A".getBytes()),new
Text("max"));
}
}
}
public static void main(String[] args) throws Exception {
org.apache.hadoop.conf.Configuration hbaseConfiguration =
HBaseConfiguration.create();
hbaseConfiguration.set("hbase.zookeeper.property.clientPort", zkPort);
hbaseConfiguration.set("hbase.zookeeper.quorum", zkMaster);
hbaseConfiguration.set("zookeeper.znode.parent", zkPath);
hbaseConfiguration.setLong("hbase.client.scanner.timeout.period",600);
hbaseConfiguration.setLong("hbase.rpc.timeout",600);
Job job = Job.getInstance(hbaseConfiguration);
job.setJarByClass(App.class);
List list = new ArrayList();
Scan scan = new Scan();
scan.addFamily(Bytes.toBytes(familyName));
scan.setCaching(5);//if Caching is too big, some rowkeys will lost!
for (String[] cell : cellNames) {
String column = cell[0];
scan.addColumn(familyName,Bytes.toBytes(column));
}
scan.setStartRow(Bytes.toBytes(startRowkeyStr));
scan.setStopRow(Bytes.toBytes(endRowkeyStr));
scan.setAttribute(Scan.SCAN_ATTRIBUTES_TABLE_NAME,
Bytes.toBytes(hbaseTableName));
list.add(scan);
System.out.println("size: "+list.size());
TableMapReduceUtil.initTableMapperJob(list,HbaseMapper.class,ImmutableBytesWritable.class,Text.class,
job);
job.setMapOutputKeyClass(ImmutableBytesWritable.class);
job.setMapOutputValueClass(Text.class);
job.setOutputKeyClass(ImmutableBytesWritable.class);
job.setOutputValueClass(Text.class);
FileOutputFormat.setOutputPath(job, new Path("maxTestOutput"));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}{code}
The pom.xml for mapreduce code is like this:
[^pom.xml]
Third way code is like this:
{code:java}
public static void main(String[] args) throws Exception{
org.apache.hadoop.conf.Configuration hbaseConfiguration =
HBaseConfiguration.create();
hbaseConfiguration.set("hbase.zookeeper.property.clientPort", zkPort);
hbaseConfiguration.set("hbase.zookeeper.quorum", zkMaster);
hbaseConfiguration.set("zookeeper.znode.parent", zkPath);
hbaseConfiguration.setLong("hbase.client.scanner.timeout.period",600);
hbaseConfiguration.setLong("hbase.rpc.timeout",600);
Connection conn = ConnectionFactory.createConnection(hbaseConfiguration);
HTable table = (HTable) conn.getTable(TableName.valueOf(hbaseTableName));
Long res = 0l;
final Scan hbaseScan = new Scan();
[jira] [Updated] (HBASE-23062) Use TableInputFormat to read data from Hbase, when Scan.setCaching(size) the size is too big, some rowkeys will lost without exctpions.
[
https://issues.apache.org/jira/browse/HBASE-23062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhanxiongWang updated HBASE-23062:
--
Description:
Hbase server cluster version is 1.2.5 . Data will be lost when the hbase client
version is 0.98.6.1. When the hbase client version is 1.2.5, no data will be
lost.
I did the experiment in three ways. One way I use spark to read hbase, second
way I use mapreduce to read hbase. In both cases, when I increase the Scan
Caching size, some data will be lost. To be more accurately, When I set
scan.setCaching(500), I can receive 7622 rows of data, but when I set
scan.setCaching(5), I can receive only 4226 rows of data. Third way I use
Scan to read hbase directly, caching size does not affect the result, I can
always receive 7622 rows of data.
The seriousness of the problem is that the data is lost but there is no
exceptions, it is difficult to find the reason.
My spark code is like this:
{code:java}
Configuration hbaseConfiguration = HBaseConfiguration.create();
hbaseConfiguration.set("hbase.zookeeper.property.clientPort", zkPort);
hbaseConfiguration.set("hbase.zookeeper.quorum", zkMaster);
hbaseConfiguration.set("zookeeper.znode.parent", zkPath);
hbaseConfiguration.set(TableInputFormat.INPUT_TABLE,hbaseTableName);
hbaseConfiguration.setLong("hbase.client.scanner.timeout.period",600);
hbaseConfiguration.setLong("hbase.rpc.timeout",600);
final Scan hbaseScan = new Scan();
hbaseScan.addFamily(familyName);
hbaseScan.setCaching(5);//if Caching is too big, some rowkeys will lost!
for(String[] cell:cellNames){
String column = cell[0];
hbaseScan.addColumn(familyName,Bytes.toBytes(column));
}
hbaseScan.setStartRow(Bytes.toBytes(startRowkeyStr));
hbaseScan.setStopRow(Bytes.toBytes(endRowkeyStr));
try {
ClientProtos.Scan scanProto = ProtobufUtil.toScan(hbaseScan);
hbaseConfiguration.set(TableInputFormat.SCAN,
Base64.encodeBytes(scanProto.toByteArray()));
JavaPairRDD pairRDD =
jsc.newAPIHadoopRDD(
hbaseConfiguration,TableInputFormat.class, ImmutableBytesWritable.class,
Result.class );
System.out.println("pairRDD.count(): " + pairRDD.count());
}
catch (IOException e) {
System.out.println("Scan Exception!! " + e.getMessage());
}
{code}
My mapreduce code is like this:
{code:java}
static class HbaseMapper extends TableMapper {
@Override protected void map(ImmutableBytesWritable key, Result
value,Mapper.Context context) throws IOException, InterruptedException {
for(Cell cell :value.rawCells()){
context.write(new ImmutableBytesWritable("A".getBytes()),new
Text("max"));
}
}
}
public static void main(String[] args) throws Exception {
org.apache.hadoop.conf.Configuration hbaseConfiguration =
HBaseConfiguration.create();
hbaseConfiguration.set("hbase.zookeeper.property.clientPort", zkPort);
hbaseConfiguration.set("hbase.zookeeper.quorum", zkMaster);
hbaseConfiguration.set("zookeeper.znode.parent", zkPath);
hbaseConfiguration.setLong("hbase.client.scanner.timeout.period",600);
hbaseConfiguration.setLong("hbase.rpc.timeout",600);
Job job = Job.getInstance(hbaseConfiguration);
job.setJarByClass(App.class);
List list = new ArrayList();
Scan scan = new Scan();
scan.addFamily(Bytes.toBytes(familyName));
scan.setCaching(5);//if Caching is too big, some rowkeys will lost!
for (String[] cell : cellNames) {
String column = cell[0];
scan.addColumn(familyName,Bytes.toBytes(column));
}
scan.setStartRow(Bytes.toBytes(startRowkeyStr));
scan.setStopRow(Bytes.toBytes(endRowkeyStr));
scan.setAttribute(Scan.SCAN_ATTRIBUTES_TABLE_NAME,
Bytes.toBytes(hbaseTableName));
list.add(scan);
System.out.println("size: "+list.size());
TableMapReduceUtil.initTableMapperJob(list,HbaseMapper.class,ImmutableBytesWritable.class,Text.class,
job);
job.setMapOutputKeyClass(ImmutableBytesWritable.class);
job.setMapOutputValueClass(Text.class);
job.setOutputKeyClass(ImmutableBytesWritable.class);
job.setOutputValueClass(Text.class);
FileOutputFormat.setOutputPath(job, new Path("maxTestOutput"));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}{code}
The pom.xml for mapreduce code is like this:
[^pom.xml]
Third way code is like this:
{code:java}
public static void main(String[] args) throws Exception{
org.apache.hadoop.conf.Configuration hbaseConfiguration =
HBaseConfiguration.create();
hbaseConfiguration.set("hbase.zookeeper.property.clientPort", zkPort);
hbaseConfiguration.set("hbase.zookeeper.quorum", zkMaster);
hbaseConfiguration.set("zookeeper.znode.parent", zkPath);
hbaseConfiguration.setLong("hbase.client.scanner.timeout.period",600);
hbaseConfiguration.setLong("hbase.rpc.timeout",600);
Connection conn = ConnectionFactory.createConnection(hbaseConfiguration);
HTable table = (HTable) conn.getTable(TableName.valueOf(hbaseTableName));
Long res = 0l;
final Scan hbaseScan = new Scan();
[jira] [Updated] (HBASE-23062) Use TableInputFormat to read data from Hbase, when Scan.setCaching(size) the size is too big, some rowkeys will lost without exctpions.
[
https://issues.apache.org/jira/browse/HBASE-23062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhanxiongWang updated HBASE-23062:
--
Description:
Hbase server cluster version is
I did the experiment in three ways. One way I use spark to read hbase, second
way I use mapreduce to read hbase. In both cases, when I increase the Scan
Caching size, some data will be lost. To be more accurately, When I set
scan.setCaching(500), I can receive 7622 rows of data, but when I set
scan.setCaching(5), I can receive only 4226 rows of data. Third way I use
Scan to read hbase directly, caching size does not affect the result, I can
always receive 7622 rows of data.
The seriousness of the problem is that the data is lost but there is no
exceptions, it is difficult to find the reason.
My spark code is like this:
{code:java}
Configuration hbaseConfiguration = HBaseConfiguration.create();
hbaseConfiguration.set("hbase.zookeeper.property.clientPort", zkPort);
hbaseConfiguration.set("hbase.zookeeper.quorum", zkMaster);
hbaseConfiguration.set("zookeeper.znode.parent", zkPath);
hbaseConfiguration.set(TableInputFormat.INPUT_TABLE,hbaseTableName);
hbaseConfiguration.setLong("hbase.client.scanner.timeout.period",600);
hbaseConfiguration.setLong("hbase.rpc.timeout",600);
final Scan hbaseScan = new Scan();
hbaseScan.addFamily(familyName);
hbaseScan.setCaching(5);//if Caching is too big, some rowkeys will lost!
for(String[] cell:cellNames){
String column = cell[0];
hbaseScan.addColumn(familyName,Bytes.toBytes(column));
}
hbaseScan.setStartRow(Bytes.toBytes(startRowkeyStr));
hbaseScan.setStopRow(Bytes.toBytes(endRowkeyStr));
try {
ClientProtos.Scan scanProto = ProtobufUtil.toScan(hbaseScan);
hbaseConfiguration.set(TableInputFormat.SCAN,
Base64.encodeBytes(scanProto.toByteArray()));
JavaPairRDD pairRDD =
jsc.newAPIHadoopRDD(
hbaseConfiguration,TableInputFormat.class, ImmutableBytesWritable.class,
Result.class );
System.out.println("pairRDD.count(): " + pairRDD.count());
}
catch (IOException e) {
System.out.println("Scan Exception!! " + e.getMessage());
}
{code}
My mapreduce code is like this:
{code:java}
static class HbaseMapper extends TableMapper {
@Override protected void map(ImmutableBytesWritable key, Result
value,Mapper.Context context) throws IOException, InterruptedException {
for(Cell cell :value.rawCells()){
context.write(new ImmutableBytesWritable("A".getBytes()),new
Text("max"));
}
}
}
public static void main(String[] args) throws Exception {
org.apache.hadoop.conf.Configuration hbaseConfiguration =
HBaseConfiguration.create();
hbaseConfiguration.set("hbase.zookeeper.property.clientPort", zkPort);
hbaseConfiguration.set("hbase.zookeeper.quorum", zkMaster);
hbaseConfiguration.set("zookeeper.znode.parent", zkPath);
hbaseConfiguration.setLong("hbase.client.scanner.timeout.period",600);
hbaseConfiguration.setLong("hbase.rpc.timeout",600);
Job job = Job.getInstance(hbaseConfiguration);
job.setJarByClass(App.class);
List list = new ArrayList();
Scan scan = new Scan();
scan.addFamily(Bytes.toBytes(familyName));
scan.setCaching(5);//if Caching is too big, some rowkeys will lost!
for (String[] cell : cellNames) {
String column = cell[0];
scan.addColumn(familyName,Bytes.toBytes(column));
}
scan.setStartRow(Bytes.toBytes(startRowkeyStr));
scan.setStopRow(Bytes.toBytes(endRowkeyStr));
scan.setAttribute(Scan.SCAN_ATTRIBUTES_TABLE_NAME,
Bytes.toBytes(hbaseTableName));
list.add(scan);
System.out.println("size: "+list.size());
TableMapReduceUtil.initTableMapperJob(list,HbaseMapper.class,ImmutableBytesWritable.class,Text.class,
job);
job.setMapOutputKeyClass(ImmutableBytesWritable.class);
job.setMapOutputValueClass(Text.class);
job.setOutputKeyClass(ImmutableBytesWritable.class);
job.setOutputValueClass(Text.class);
FileOutputFormat.setOutputPath(job, new Path("maxTestOutput"));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}{code}
The pom.xml for mapreduce code is like this:
[^pom.xml]
Third way code is like this:
{code:java}
public static void main(String[] args) throws Exception{
org.apache.hadoop.conf.Configuration hbaseConfiguration =
HBaseConfiguration.create();
hbaseConfiguration.set("hbase.zookeeper.property.clientPort", zkPort);
hbaseConfiguration.set("hbase.zookeeper.quorum", zkMaster);
hbaseConfiguration.set("zookeeper.znode.parent", zkPath);
hbaseConfiguration.setLong("hbase.client.scanner.timeout.period",600);
hbaseConfiguration.setLong("hbase.rpc.timeout",600);
Connection conn = ConnectionFactory.createConnection(hbaseConfiguration);
HTable table = (HTable) conn.getTable(TableName.valueOf(hbaseTableName));
Long res = 0l;
final Scan hbaseScan = new Scan();
hbaseScan.addFamily(Bytes.toBytes(familyName));
hbaseScan.setCaching(5);//if Caching is too big, some rowkeys will lost!
for (String[] cell :
[GitHub] [hbase] Apache-HBase commented on issue #656: HBASE-23063 Add an option to enable multiget in parallel
Apache-HBase commented on issue #656: HBASE-23063 Add an option to enable multiget in parallel URL: https://github.com/apache/hbase/pull/656#issuecomment-535815261 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | :blue_heart: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | :green_heart: | dupname | 0m 0s | No case conflicting files found. | | :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | :green_heart: | test4tests | 0m 0s | The patch appears to include 2 new or modified test files. | ||| _ master Compile Tests _ | | :blue_heart: | mvndep | 0m 38s | Maven dependency ordering for branch | | :green_heart: | mvninstall | 5m 13s | master passed | | :green_heart: | compile | 1m 19s | master passed | | :green_heart: | checkstyle | 1m 50s | master passed | | :green_heart: | shadedjars | 4m 35s | branch has no errors when building our shaded downstream artifacts. | | :green_heart: | javadoc | 0m 57s | master passed | | :blue_heart: | spotbugs | 4m 4s | Used deprecated FindBugs config; considering switching to SpotBugs. | | :green_heart: | findbugs | 4m 54s | master passed | ||| _ Patch Compile Tests _ | | :blue_heart: | mvndep | 0m 15s | Maven dependency ordering for patch | | :green_heart: | mvninstall | 4m 52s | the patch passed | | :green_heart: | compile | 1m 18s | the patch passed | | :green_heart: | javac | 1m 18s | the patch passed | | :green_heart: | checkstyle | 0m 27s | The patch passed checkstyle in hbase-common | | :green_heart: | checkstyle | 1m 22s | hbase-server: The patch generated 0 new + 195 unchanged - 3 fixed = 195 total (was 198) | | :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | :green_heart: | shadedjars | 4m 33s | patch has no errors when building our shaded downstream artifacts. | | :green_heart: | hadoopcheck | 15m 31s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2. | | :green_heart: | javadoc | 0m 59s | the patch passed | | :green_heart: | findbugs | 5m 4s | the patch passed | ||| _ Other Tests _ | | :green_heart: | unit | 3m 6s | hbase-common in the patch passed. | | :broken_heart: | unit | 159m 47s | hbase-server in the patch failed. | | :green_heart: | asflicense | 1m 5s | The patch does not generate ASF License warnings. | | | | 224m 58s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.quotas.TestQuotaAdmin | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-656/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/656 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux d6998b28a38a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-656/out/precommit/personality/provided.sh | | git revision | master / 3250a8021a | | Default Java | 1.8.0_181 | | unit | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-656/3/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-656/3/testReport/ | | Max. process+thread count | 4627 (vs. ulimit of 1) | | modules | C: hbase-common hbase-server U: . | | Console output | https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-656/3/console | | versions | git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11 | | Powered by | Apache Yetus 0.11.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
