Re: [PR] HBASE-28174 [hbase]
Apache-HBase commented on PR #5501: URL: https://github.com/apache/hbase/pull/5501#issuecomment-1803381662 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 18s | master passed | | +1 :green_heart: | compile | 0m 17s | master passed | | +1 :green_heart: | shadedjars | 4m 49s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 13s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 20s | the patch passed | | +1 :green_heart: | compile | 0m 17s | the patch passed | | +1 :green_heart: | javac | 0m 17s | the patch passed | | +1 :green_heart: | shadedjars | 4m 50s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 12s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 3m 21s | hbase-rest in the patch failed. | | | | 20m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5501 | | JIRA Issue | HBASE-28174 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 81703b568a5e 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 4b5db21f3f | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-rest.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/testReport/ | | Max. process+thread count | 1537 (vs. ulimit of 3) | | modules | C: hbase-rest U: hbase-rest | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28174 [hbase]
Apache-HBase commented on PR #5501: URL: https://github.com/apache/hbase/pull/5501#issuecomment-1803385410 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 48s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 12s | master passed | | +1 :green_heart: | compile | 0m 21s | master passed | | +1 :green_heart: | shadedjars | 5m 12s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 15s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 44s | the patch passed | | +1 :green_heart: | compile | 0m 19s | the patch passed | | +1 :green_heart: | javac | 0m 19s | the patch passed | | +1 :green_heart: | shadedjars | 4m 55s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 15s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 3m 52s | hbase-rest in the patch failed. | | | | 22m 55s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5501 | | JIRA Issue | HBASE-28174 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6b3ddf0d3843 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 4b5db21f3f | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-rest.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/testReport/ | | Max. process+thread count | 1590 (vs. ulimit of 3) | | modules | C: hbase-rest U: hbase-rest | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28174 [hbase]
Apache-HBase commented on PR #5501: URL: https://github.com/apache/hbase/pull/5501#issuecomment-1803388388 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 41s | master passed | | +1 :green_heart: | compile | 0m 23s | master passed | | +1 :green_heart: | checkstyle | 0m 8s | master passed | | +1 :green_heart: | spotless | 0m 40s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 39s | the patch passed | | +1 :green_heart: | compile | 0m 23s | the patch passed | | -0 :warning: | javac | 0m 23s | hbase-rest generated 11 new + 154 unchanged - 0 fixed = 165 total (was 154) | | -0 :warning: | checkstyle | 0m 7s | hbase-rest: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 45s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 38s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 7s | The patch does not generate ASF License warnings. | | | | 24m 58s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5501 | | JIRA Issue | HBASE-28174 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 34817033ce5e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 4b5db21f3f | | Default Java | Eclipse Adoptium-11.0.17+8 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/artifact/yetus-general-check/output/diff-compile-javac-hbase-rest.txt | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/artifact/yetus-general-check/output/diff-checkstyle-hbase-rest.txt | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-rest U: hbase-rest | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/6/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28174 [hbase]
judilsteve commented on PR #5501: URL: https://github.com/apache/hbase/pull/5501#issuecomment-1803397021 ``` [ERROR] org.apache.hadoop.hbase.rest.TestGetAndPutResource.testUrlB64EncodedKeyHeader Time elapsed: 1.214 s <<< FAILURE! org.junit.ComparisonFailure: expected:<[text/xml]> but was:<[application/octet-stream]> at org.junit.Assert.assertEquals(Assert.java:117) at org.junit.Assert.assertEquals(Assert.java:146) at org.apache.hadoop.hbase.rest.TestGetAndPutResource.checkValue1(TestGetAndPutResource.java:351) at org.apache.hadoop.hbase.rest.TestGetAndPutResource.testUrlB64EncodedKeyHeader(TestGetAndPutResource.java:401) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.lang.Thread.run(Thread.java:829) ``` That's a bit more of a clue. Will keep digging tomorrow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28189 Fix the miss count in one of CombinedBlockCache getBlock implementations [hbase]
wchevreuil merged PR #5506: URL: https://github.com/apache/hbase/pull/5506 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28189) Fix the miss count in one of CombinedBlockCache getBlock implementations
[ https://issues.apache.org/jira/browse/HBASE-28189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-28189: - Affects Version/s: 2.5.6 2.4.17 3.0.0-alpha-4 2.6.0 4.0.0-alpha-1 > Fix the miss count in one of CombinedBlockCache getBlock implementations > > > Key: HBASE-28189 > URL: https://issues.apache.org/jira/browse/HBASE-28189 > Project: HBase > Issue Type: Bug >Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6, 4.0.0-alpha-1 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > > In one of the > CombinedBlockCache.getBlock(getBlock(cacheKey,cachingrepeat,updateCacheMetrics) > we always compute a miss in L1 if the passed block is of type DATA. We > should compute the miss in one of the caches only, not both. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28189) Fix the miss count in one of CombinedBlockCache getBlock implementations
[ https://issues.apache.org/jira/browse/HBASE-28189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-28189: - Fix Version/s: 4.0.0-alpha-1 > Fix the miss count in one of CombinedBlockCache getBlock implementations > > > Key: HBASE-28189 > URL: https://issues.apache.org/jira/browse/HBASE-28189 > Project: HBase > Issue Type: Bug >Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6, 4.0.0-alpha-1 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 4.0.0-alpha-1 > > > In one of the > CombinedBlockCache.getBlock(getBlock(cacheKey,cachingrepeat,updateCacheMetrics) > we always compute a miss in L1 if the passed block is of type DATA. We > should compute the miss in one of the caches only, not both. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28189) Fix the miss count in one of CombinedBlockCache getBlock implementations
[ https://issues.apache.org/jira/browse/HBASE-28189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-28189: - Fix Version/s: 3.0.0-beta-1 > Fix the miss count in one of CombinedBlockCache getBlock implementations > > > Key: HBASE-28189 > URL: https://issues.apache.org/jira/browse/HBASE-28189 > Project: HBase > Issue Type: Bug >Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6, 4.0.0-alpha-1 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 3.0.0-beta-1, 4.0.0-alpha-1 > > > In one of the > CombinedBlockCache.getBlock(getBlock(cacheKey,cachingrepeat,updateCacheMetrics) > we always compute a miss in L1 if the passed block is of type DATA. We > should compute the miss in one of the caches only, not both. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28189) Fix the miss count in one of CombinedBlockCache getBlock implementations
[ https://issues.apache.org/jira/browse/HBASE-28189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-28189: - Fix Version/s: 2.6.0 2.4.18 2.5.7 > Fix the miss count in one of CombinedBlockCache getBlock implementations > > > Key: HBASE-28189 > URL: https://issues.apache.org/jira/browse/HBASE-28189 > Project: HBase > Issue Type: Bug >Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6, 4.0.0-alpha-1 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 4.0.0-alpha-1, 2.5.7 > > > In one of the > CombinedBlockCache.getBlock(getBlock(cacheKey,cachingrepeat,updateCacheMetrics) > we always compute a miss in L1 if the passed block is of type DATA. We > should compute the miss in one of the caches only, not both. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28189) Fix the miss count in one of CombinedBlockCache getBlock implementations
[ https://issues.apache.org/jira/browse/HBASE-28189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil resolved HBASE-28189. -- Resolution: Fixed Thanks for reviewing this, [~psomogyi]. I had merged this to master, branch-3, branch-2, branch-2.5 and branch-2.4. > Fix the miss count in one of CombinedBlockCache getBlock implementations > > > Key: HBASE-28189 > URL: https://issues.apache.org/jira/browse/HBASE-28189 > Project: HBase > Issue Type: Bug >Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6, 4.0.0-alpha-1 >Reporter: Wellington Chevreuil >Assignee: Wellington Chevreuil >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 4.0.0-alpha-1, 2.5.7 > > > In one of the > CombinedBlockCache.getBlock(getBlock(cacheKey,cachingrepeat,updateCacheMetrics) > we always compute a miss in L1 if the passed block is of type DATA. We > should compute the miss in one of the caches only, not both. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28043 Reduce seeks from beginning of block in StoreFileScanner.seekToPreviousRow [hbase]
bbeaudreault merged PR #5373: URL: https://github.com/apache/hbase/pull/5373 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28120) Provide the switch to avoid reopening regions in the alter sync command
[ https://issues.apache.org/jira/browse/HBASE-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784477#comment-17784477 ] Bryan Beaudreault commented on HBASE-28120: --- Can we simply backport in the original jira? I think it is easier on everyone if we do backports as part of the original jira, unless a backport happens long after a Jira is closed. This is because the fixVersions are consolidated in one place, rather than having to check multiple jiras to see where a feature landed. > Provide the switch to avoid reopening regions in the alter sync command > --- > > Key: HBASE-28120 > URL: https://issues.apache.org/jira/browse/HBASE-28120 > Project: HBase > Issue Type: Sub-task > Components: master, shell >Affects Versions: 2.0.0-alpha-1 >Reporter: Gourab Taparia >Assignee: Gourab Taparia >Priority: Major > Fix For: 2.6.0 > > > As part of the sub-task, as HBase 2 supports both Async and Sync API, this > task is to add this support/feature to HBase 2's Sync API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HBASE-28120) Provide the switch to avoid reopening regions in the alter sync command
[ https://issues.apache.org/jira/browse/HBASE-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784477#comment-17784477 ] Bryan Beaudreault edited comment on HBASE-28120 at 11/9/23 2:31 PM: Can we simply backport in the original jira? I think it is easier on everyone if we do backports as part of the original jira, unless a backport happens long after a Jira is closed. This is because the fixVersions are consolidated in one place, rather than having to check multiple jiras to see where a feature landed. So my ask would be if Gourab would be willing to close this Jira, and someone push a branch-2 PR for the original parent jira. We can get that merged before closing it. was (Author: bbeaudreault): Can we simply backport in the original jira? I think it is easier on everyone if we do backports as part of the original jira, unless a backport happens long after a Jira is closed. This is because the fixVersions are consolidated in one place, rather than having to check multiple jiras to see where a feature landed. > Provide the switch to avoid reopening regions in the alter sync command > --- > > Key: HBASE-28120 > URL: https://issues.apache.org/jira/browse/HBASE-28120 > Project: HBase > Issue Type: Sub-task > Components: master, shell >Affects Versions: 2.0.0-alpha-1 >Reporter: Gourab Taparia >Assignee: Gourab Taparia >Priority: Major > Fix For: 2.6.0 > > > As part of the sub-task, as HBase 2 supports both Async and Sync API, this > task is to add this support/feature to HBase 2's Sync API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28120) Provide the switch to avoid reopening regions in the alter sync command
[ https://issues.apache.org/jira/browse/HBASE-28120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784488#comment-17784488 ] Bryan Beaudreault commented on HBASE-28120: --- We could re-purpose the existing PR, but I agree with above that we should start over with a fresh cherry-pick of the master commit so that we don't lose feedback from that PR. > Provide the switch to avoid reopening regions in the alter sync command > --- > > Key: HBASE-28120 > URL: https://issues.apache.org/jira/browse/HBASE-28120 > Project: HBase > Issue Type: Sub-task > Components: master, shell >Affects Versions: 2.0.0-alpha-1 >Reporter: Gourab Taparia >Assignee: Gourab Taparia >Priority: Major > Fix For: 2.6.0 > > > As part of the sub-task, as HBase 2 supports both Async and Sync API, this > task is to add this support/feature to HBase 2's Sync API. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-27276 Reduce reflection overhead in Filter deserialization [hbase]
Apache-HBase commented on PR #5488: URL: https://github.com/apache/hbase/pull/5488#issuecomment-1804022713 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 41s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 13s | master passed | | +1 :green_heart: | compile | 4m 11s | master passed | | +1 :green_heart: | checkstyle | 1m 10s | master passed | | +1 :green_heart: | spotless | 0m 46s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 3m 15s | master passed | | -0 :warning: | patch | 2m 9s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 27s | the patch passed | | +1 :green_heart: | compile | 3m 59s | the patch passed | | +1 :green_heart: | javac | 3m 59s | the patch passed | | +1 :green_heart: | checkstyle | 1m 6s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 56s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 1m 5s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 4m 38s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 52m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/8/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5488 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 71db3c3988ea 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5dc4467e6c | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/8/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28043) Reduce seeks from beginning of block in StoreFileScanner.seekToPreviousRow
[ https://issues.apache.org/jira/browse/HBASE-28043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Beaudreault updated HBASE-28043: -- Fix Version/s: 2.6.0 3.0.0-beta-1 Release Note: Optimizes StoreFileScanner.seekToPreviousRow to use keep track of a hint which allows us to eliminate one seek per call, resulting in 40% or more throughput increase for reverse scans. External users (Phoenix) of this LimitedPrivate method should be aware of the change in semantics. It is expected that seekToPreviousRow is used for scanning backwards in the StoreFile. Calling with an originalKey greater than the previously passed key (i.e. forward in the StoreFile), the originalKey will not be honored. Instead use seek for this case. Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for all the work here [~bewing]! Big optimization > Reduce seeks from beginning of block in StoreFileScanner.seekToPreviousRow > -- > > Key: HBASE-28043 > URL: https://issues.apache.org/jira/browse/HBASE-28043 > Project: HBase > Issue Type: Improvement >Reporter: Becker Ewing >Assignee: Becker Ewing >Priority: Major > Fix For: 2.6.0, 3.0.0-beta-1 > > Attachments: Current_SeekToPreviousRowBehavior.png, > Proposed_SeekToPreviousRowBehavior.png > > > Currently, for non-RIV1 DBE encodings, each call to > [StoreFileScanner.seekToPreviousRow|https://github.com/apache/hbase/blob/89ca7f4ade84c84a246281c71898543b6161c099/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java#L493-L506] > (a common operation in reverse scans) results in two seeks: > # Seek from the beginning of the block to before the given row to find the > prior row > # Seek from the beginning of the block to the first cell of the prior row > So if there are "N" rows in a block, a reverse scan through each row results > in seeking past summation from i=1 to N (2(i-1)) rows. > > This is a particularly expensive operation for tall tables that have many > rows in a block. > > By introducing a state variable "previousRow" to StoreFileScanner, I believe > that we could modify the seeking algorithm to be: > # Seek from the beginning of the block to before the given row to find the > prior row > # Seek from the beginning of the block to before the row that is before the > row that was just seeked to (i.e. 2 rows back). _Save_ this as a hint for > where the prior row is in "previousRow" > # Reseek from "previousRow" (2 rows back from start) to 1 row back from > start (to the actual previousRow) > Then the rest of the calls where a "previousRow" is present, you just need to > seek to the beginning of the block once instead of twice, i.e. > # seek from the beginning of the block to right before the beginning of your > "previousRow" marker. Save this as the new "previousRow" marker > # Reseek to the next row (i.e. your previous "previousRow" marker) > > If there are "N" rows in a block, a reverse scan from row N to row 0 results > in seeking past approximately summation from i=1 to N (i-1) rows i.e. 50% > less than the current behavior. > > See the attached diagrams for the current and proposed behavior. > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-27276 Reduce reflection overhead in Filter deserialization [hbase]
bbeaudreault commented on code in PR #5488:
URL: https://github.com/apache/hbase/pull/5488#discussion_r1384834533
##
hbase-client/src/main/java/org/apache/hadoop/hbase/shaded/protobuf/ProtobufUtil.java:
##
@@ -1552,13 +1571,23 @@ public static ComparatorProtos.Comparator
toComparator(ByteArrayComparable compa
public static ByteArrayComparable toComparator(ComparatorProtos.Comparator
proto)
throws IOException {
String type = proto.getName();
-String funcName = "parseFrom";
byte[] value = proto.getSerializedComparator().toByteArray();
+
try {
+ ByteArrayComparable result = COMPARATORS.getAndCallByName(type, value);
+ if (result != null) {
+return result;
+ }
+
+ if (!ALLOW_FAST_REFLECTION_FALLTHROUGH) {
+throw new IllegalStateException("Failed to deserialize comparator " +
type
+ + " because fast reflection returned null and fallthrough is
disabled");
+ }
Review Comment:
I only added this so that I could write tests, since this is all static
methods.
I don't think we want a warn or counter here. How often it happens will
depend on the usage of custom filters. If they don't use custom filters, this
will never fail. If they use exclusively custom filters, then it will fail
every time. It's not really a failure mode, more backwards compatibility
handling.
In fact, this specific exception will never fire outside of tests. The above
call to getAndCallByName will "fail" (return null) for custom filters, and
cleanly fallback to the below older handling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-27276 Reduce reflection overhead in Filter deserialization [hbase]
bbeaudreault commented on PR #5488: URL: https://github.com/apache/hbase/pull/5488#issuecomment-1804131667 I think this is ready for merge, if anyone has any other feedback since the move to on-demand loading. Note I decided to use a ConcurrentHashMap rather than LoadingCache or Caffeine Cache, because we don't need any of the extra features of those. The total number of cached items is small and constant, with no eviction or expiration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
guluo2016 opened a new pull request, #5510: URL: https://github.com/apache/hbase/pull/5510 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
guluo2016 opened a new pull request, #5511: URL: https://github.com/apache/hbase/pull/5511 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
guluo2016 opened a new pull request, #5512: URL: https://github.com/apache/hbase/pull/5512 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28191) Meta browser can happen NPE when the server or target server of region is null
[
https://issues.apache.org/jira/browse/HBASE-28191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784550#comment-17784550
]
guluo commented on HBASE-28191:
---
Thank you [~zhangduo] for your reply
I have already committed the change to branch-2, branch-2.4, branch-2.5 and
master
> Meta browser can happen NPE when the server or target server of region is null
> --
>
> Key: HBASE-28191
> URL: https://issues.apache.org/jira/browse/HBASE-28191
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 2.4.13, 2.5.6
> Environment: hbase2.4.13
> centos
>Reporter: guluo
>Assignee: guluo
>Priority: Major
>
> Reproduction Steps:
> 1. We create a new table with many regions , for example:
> {code:java}
> //代码占位符
> create 't01' , 'info' , SPLITS => Array(1..999).map{|e|e.to_s}{code}
> 2. At the same time, we scan meta entries of this table on meta browser by
> hbase UI
> 3. We would get NPE if those regions is initializing and the server or target
> server is null
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
Apache-HBase commented on PR #5512: URL: https://github.com/apache/hbase/pull/5512#issuecomment-1804209423 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 46s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.5 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 3s | branch-2.5 passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 45s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 11s | The patch does not generate ASF License warnings. | | | | 9m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5512/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5512 | | Optional Tests | dupname asflicense javac spotless | | uname | Linux 3cf99542027b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.5 / ce309a7c15 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5512/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
Apache-HBase commented on PR #5511: URL: https://github.com/apache/hbase/pull/5511#issuecomment-1804214052 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 56s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2.4 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 50s | branch-2.4 passed | | +1 :green_heart: | spotless | 1m 0s | branch has no errors when running spotless:check. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 35s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | spotless | 1m 1s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 14s | The patch does not generate ASF License warnings. | | | | 12m 8s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5511/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5511 | | Optional Tests | dupname asflicense javac spotless | | uname | Linux beb17f821f68 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / 43a109098d | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 82 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5511/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
Apache-HBase commented on PR #5510: URL: https://github.com/apache/hbase/pull/5510#issuecomment-1804214572 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 17s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 25s | branch-2 passed | | +1 :green_heart: | spotless | 1m 2s | branch has no errors when running spotless:check. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 51s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | spotless | 1m 1s | patch has no errors when running spotless:check. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 21s | The patch does not generate ASF License warnings. | | | | 12m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5510/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5510 | | Optional Tests | dupname asflicense javac spotless | | uname | Linux 379a6b51fc86 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91a5eae61c | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5510/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28175: Deep copy RpcLogDetails' param field [hbase]
bbeaudreault merged PR #5481: URL: https://github.com/apache/hbase/pull/5481 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-28175) RpcLogDetails' Message can become corrupt before log is consumed
[ https://issues.apache.org/jira/browse/HBASE-28175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Beaudreault resolved HBASE-28175. --- Fix Version/s: 2.6.0 3.0.0-beta-1 2.5.7 Resolution: Fixed > RpcLogDetails' Message can become corrupt before log is consumed > > > Key: HBASE-28175 > URL: https://issues.apache.org/jira/browse/HBASE-28175 > Project: HBase > Issue Type: Bug >Reporter: Ray Mattingly >Assignee: Ray Mattingly >Priority: Major > Fix For: 2.6.0, 3.0.0-beta-1, 2.5.7 > > > The RpcLogDetails class represents a slow (or large) log event which will > later be consumed by the SlowLogQueueService. > Right now the RpcLogDetails' param field points to the slow call's Message, > and this Message is backed by a CodedInputStream which may be overwritten > before the given log is consumed. This overwriting of the Message may result > in slow query payloads for which the metadata derived post-consumption is > inaccurate. > To solve this bug I think we need to copy the Message in the RpcLogDetails > constructor. I have this bug reproduced in a QA environment and will test out > this idea and open a PR shortly if the test results are promising. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-27276 Reduce reflection overhead in Filter deserialization [hbase]
Apache-HBase commented on PR #5488: URL: https://github.com/apache/hbase/pull/5488#issuecomment-1804361882 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 47s | master passed | | +1 :green_heart: | compile | 1m 25s | master passed | | +1 :green_heart: | shadedjars | 4m 59s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | master passed | | -0 :warning: | patch | 6m 23s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 41s | the patch passed | | +1 :green_heart: | compile | 1m 27s | the patch passed | | +1 :green_heart: | javac | 1m 26s | the patch passed | | +1 :green_heart: | shadedjars | 4m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 29s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 28s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 227m 13s | hbase-server in the patch passed. | | | | 257m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/8/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5488 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 6e2cc5f02825 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5dc4467e6c | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/8/testReport/ | | Max. process+thread count | 4722 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/8/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-27276 Reduce reflection overhead in Filter deserialization [hbase]
Apache-HBase commented on PR #5488: URL: https://github.com/apache/hbase/pull/5488#issuecomment-1804373350 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 39s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 40s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 46s | master passed | | +1 :green_heart: | compile | 1m 23s | master passed | | +1 :green_heart: | shadedjars | 6m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 6s | master passed | | -0 :warning: | patch | 7m 40s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 6s | the patch passed | | +1 :green_heart: | compile | 1m 33s | the patch passed | | +1 :green_heart: | javac | 1m 33s | the patch passed | | +1 :green_heart: | shadedjars | 5m 58s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 3s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 11s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 26s | hbase-client in the patch passed. | | -1 :x: | unit | 227m 52s | hbase-server in the patch failed. | | | | 261m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/8/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5488 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b8cc55d6cb1f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 5dc4467e6c | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/8/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/8/testReport/ | | Max. process+thread count | 4507 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/8/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-27276 Reduce reflection overhead in Filter deserialization [hbase]
Apache-HBase commented on PR #5488: URL: https://github.com/apache/hbase/pull/5488#issuecomment-1804509255 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 24s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 39s | master passed | | +1 :green_heart: | compile | 4m 13s | master passed | | +1 :green_heart: | checkstyle | 1m 8s | master passed | | +1 :green_heart: | spotless | 0m 47s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 3m 7s | master passed | | -0 :warning: | patch | 2m 6s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 31s | the patch passed | | +1 :green_heart: | compile | 4m 7s | the patch passed | | +1 :green_heart: | javac | 4m 7s | the patch passed | | +1 :green_heart: | checkstyle | 1m 7s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 30s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 1m 3s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 4m 42s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 51m 36s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/9/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5488 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux bdd5c473181d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/9/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-28192) Master should recover if meta region state is inconsistent
Viraj Jasani created HBASE-28192:
Summary: Master should recover if meta region state is inconsistent
Key: HBASE-28192
URL: https://issues.apache.org/jira/browse/HBASE-28192
Project: HBase
Issue Type: Improvement
Affects Versions: 2.5.6, 2.4.17
Reporter: Viraj Jasani
Assignee: Viraj Jasani
Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
During active master initialization, before we set master as active (i.e.
{_}setInitialized(true){_}), we need both meta and namespace regions online. If
the region state of meta or namespace is inconsistent, active master can get
stuck in the initialization step:
{code:java}
private boolean isRegionOnline(RegionInfo ri) {
RetryCounter rc = null;
while (!isStopped()) {
...
...
...
// Check once-a-minute.
if (rc == null) {
rc = new RetryCounterFactory(Integer.MAX_VALUE, 1000, 60_000).create();
}
Threads.sleep(rc.getBackoffTimeAndIncrementAttempts());
}
return false;
}
{code}
In one of the recent outage, we observed that meta was online on a server,
which was correctly reflected in meta znode, but the server starttime was
different. This means that as per the latest transition record, meta was marked
online on old server (same server with old start time). This kept active master
initialization waiting forever and some SCPs got stuck in initial stage where
they need to access meta table before getting candidate for region moves.
The only way out of this outage is for operator to schedule recoveries using
hbck for old server, which triggers SCP for old server address of meta. Since
many SCPs were stuck, the processing of new SCP too was taking some time and
manual restart of active master triggered failover, and new master was able to
complete SCP for old meta server, correcting the meta assignment details, which
eventually marked master as active and only after this, we were able to see
real large num of RITs that were hidden so far.
We need to let master recover from this state to avoid manual intervention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
Apache-HBase commented on PR #5511: URL: https://github.com/apache/hbase/pull/5511#issuecomment-1804567611 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.4 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 16s | branch-2.4 passed | | +1 :green_heart: | javadoc | 0m 24s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 0s | the patch passed | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 180m 6s | hbase-server in the patch passed. | | | | 189m 51s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5511/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5511 | | Optional Tests | javac javadoc unit | | uname | Linux b5c751dc7b90 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / 43a109098d | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5511/1/testReport/ | | Max. process+thread count | 4287 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5511/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
Apache-HBase commented on PR #5512: URL: https://github.com/apache/hbase/pull/5512#issuecomment-1804605439 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.5 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 58s | branch-2.5 passed | | +1 :green_heart: | javadoc | 0m 24s | branch-2.5 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 38s | the patch passed | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 194m 2s | hbase-server in the patch passed. | | | | 205m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5512/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5512 | | Optional Tests | javac javadoc unit | | uname | Linux fff98ebd84b0 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.5 / ce309a7c15 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5512/1/testReport/ | | Max. process+thread count | 4639 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5512/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
Apache-HBase commented on PR #5511: URL: https://github.com/apache/hbase/pull/5511#issuecomment-1804610607 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 47s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.4 Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 12s | branch-2.4 passed | | +1 :green_heart: | javadoc | 0m 33s | branch-2.4 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 59s | the patch passed | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 195m 29s | hbase-server in the patch failed. | | | | 207m 32s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5511/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5511 | | Optional Tests | javac javadoc unit | | uname | Linux 00e5978b7f21 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.4 / 43a109098d | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5511/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5511/1/testReport/ | | Max. process+thread count | 3884 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5511/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
Apache-HBase commented on PR #5512: URL: https://github.com/apache/hbase/pull/5512#issuecomment-1804645316 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 46s | Docker mode activated. | | -0 :warning: | yetus | 0m 6s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2.5 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 10s | branch-2.5 passed | | +1 :green_heart: | javadoc | 0m 34s | branch-2.5 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 7s | the patch passed | | +1 :green_heart: | javadoc | 0m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 210m 48s | hbase-server in the patch passed. | | | | 224m 18s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5512/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5512 | | Optional Tests | javac javadoc unit | | uname | Linux 68e2640e5c12 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2.5 / ce309a7c15 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5512/1/testReport/ | | Max. process+thread count | 4034 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5512/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
Apache-HBase commented on PR #5510: URL: https://github.com/apache/hbase/pull/5510#issuecomment-1804648171 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 20s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 23s | branch-2 passed | | +1 :green_heart: | javadoc | 0m 28s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 59s | the patch passed | | +1 :green_heart: | javadoc | 0m 32s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 212m 29s | hbase-server in the patch passed. | | | | 226m 39s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5510/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5510 | | Optional Tests | javac javadoc unit | | uname | Linux 907b9e5efee4 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91a5eae61c | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5510/1/testReport/ | | Max. process+thread count | 4299 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5510/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28122) TLSv1.3-compatible cipher suites are not used by default
[
https://issues.apache.org/jira/browse/HBASE-28122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784604#comment-17784604
]
Hudson commented on HBASE-28122:
Results for branch branch-3
[build #81 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/81/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/81/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/81/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/81/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> TLSv1.3-compatible cipher suites are not used by default
>
>
> Key: HBASE-28122
> URL: https://issues.apache.org/jira/browse/HBASE-28122
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.5.5, 3.0.0
>Reporter: Charles Connell
>Assignee: Charles Connell
>Priority: Major
> Fix For: 2.6.0, 3.0.0-beta-1
>
>
> The X509Util class has a system for choosing the cipher suites to support on
> TLS connections, if {{hbase.rpc.tls.ciphersuites}} is not provided. It also
> allows you choose what protocol you want via the
> {{hbase.rpc.tls.enabledProtocols}} config. If
> {{hbase.rpc.tls.enabledProtocols}} is set to {{{}TLSv1.3{}}}, and
> {{hbase.rpc.tls.ciphersuites}} is not set, the user of X509Util cannot form
> any working TLS connections.
> This is because all the cipher suites chosen by X509Utils are pre-TLSv1.3
> suites, and so are rejected during connection handshakes. [TLSv1.3 requires
> or suggests support|https://www.rfc-editor.org/rfc/rfc8446#section-9.1] for
> these suites, none of which are shared in common with TLSv1.2:
> * {{TLS_AES_256_GCM_SHA384}}
> * {{TLS_CHACHA20_POLY1305_SHA256}}
> * {{TLS_AES_128_GCM_SHA256}}
> Of these, the intersection of
> [BoringSSL|https://github.com/google/boringssl/blob/master/ssl/test/runner/cipher_suites.go]
> and [Java
> 11+|https://docs.oracle.com/en/java/javase/11/docs/specs/security/standard-names.html#jsse-cipher-suite-names]
> support {{TLS_AES_128_GCM_SHA256}} and {{{}TLS_AES_256_GCM_SHA384{}}}, so
> those should be added to the defaults in X509Util. BoringSSL and the JVM are
> the two crypto providers used here.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28191) Meta browser can happen NPE when the server or target server of region is null
[
https://issues.apache.org/jira/browse/HBASE-28191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784603#comment-17784603
]
Hudson commented on HBASE-28191:
Results for branch branch-3
[build #81 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/81/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/81/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/81/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/81/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Meta browser can happen NPE when the server or target server of region is null
> --
>
> Key: HBASE-28191
> URL: https://issues.apache.org/jira/browse/HBASE-28191
> Project: HBase
> Issue Type: Bug
> Components: UI
>Affects Versions: 2.4.13, 2.5.6
> Environment: hbase2.4.13
> centos
>Reporter: guluo
>Assignee: guluo
>Priority: Major
>
> Reproduction Steps:
> 1. We create a new table with many regions , for example:
> {code:java}
> //代码占位符
> create 't01' , 'info' , SPLITS => Array(1..999).map{|e|e.to_s}{code}
> 2. At the same time, we scan meta entries of this table on meta browser by
> hbase UI
> 3. We would get NPE if those regions is initializing and the server or target
> server is null
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28191 Meta browser can happen NPE when the server or target server of region is null [hbase]
Apache-HBase commented on PR #5510: URL: https://github.com/apache/hbase/pull/5510#issuecomment-1804664427 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 36s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 27s | branch-2 passed | | +1 :green_heart: | javadoc | 0m 26s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 14s | the patch passed | | +1 :green_heart: | javadoc | 0m 25s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 227m 21s | hbase-server in the patch passed. | | | | 238m 1s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5510/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5510 | | Optional Tests | javac javadoc unit | | uname | Linux 4ceefd27d69d 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 91a5eae61c | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5510/1/testReport/ | | Max. process+thread count | 4279 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5510/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
Apache-HBase commented on PR #5513: URL: https://github.com/apache/hbase/pull/5513#issuecomment-1804684123 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 8s | master passed | | +1 :green_heart: | compile | 3m 5s | master passed | | +1 :green_heart: | checkstyle | 0m 56s | master passed | | +1 :green_heart: | spotless | 1m 4s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 11s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 28s | the patch passed | | +1 :green_heart: | compile | 2m 57s | the patch passed | | +1 :green_heart: | javac | 2m 57s | the patch passed | | +1 :green_heart: | checkstyle | 0m 48s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 37s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 1m 3s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 2m 17s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 44m 46s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/1/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5513 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 85bbcd0830eb 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 83 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/1/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
apurtell commented on code in PR #5513:
URL: https://github.com/apache/hbase/pull/5513#discussion_r1388603796
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java:
##
@@ -1399,23 +1399,46 @@ private void
createMissingCFsInMetaDuringUpgrade(TableDescriptor metaDescriptor)
* Check hbase:meta is up and ready for reading. For use during Master
startup only.
* @return True if meta is UP and online and startup can progress.
Otherwise, meta is not online
* and we will hold here until operator intervention.
+ * @throws IOException If the master restart is required.
*/
@InterfaceAudience.Private
- public boolean waitForMetaOnline() {
+ public boolean waitForMetaOnline() throws IOException {
return isRegionOnline(RegionInfoBuilder.FIRST_META_REGIONINFO);
}
/**
+ * Wait until the region is reported online on a live regionserver.
+ * @param ri Region info.
* @return True if region is online and scannable else false if an error or
shutdown (Otherwise we
* just block in here holding up all forward-progess).
+ * @throws IOException If the master restart is required.
*/
- private boolean isRegionOnline(RegionInfo ri) {
+ private boolean isRegionOnline(RegionInfo ri) throws IOException {
RetryCounter rc = null;
while (!isStopped()) {
RegionState rs =
this.assignmentManager.getRegionStates().getRegionState(ri);
if (rs != null && rs.isOpened()) {
if (this.getServerManager().isServerOnline(rs.getServerName())) {
return true;
+} else {
+ LOG.info("{} has state {} but the server {} is not online,
scheduling recovery.",
+ri.getRegionNameAsString(), rs, rs.getServerName());
+ this.getServerManager().expireServer(rs.getServerName(), true);
+ // If already many SCPs are scheduled, but they are not progressing
because of
+ // meta's unavailability, the best action item is to throw
PleaseRestartMasterException
+ // and let new active master init take care of on-lining meta and
process all other
+ // pending SCPs. It's worth waiting for ~20s before arriving at the
conclusion, rather
Review Comment:
Is this true? Have we really thought about prioritizing execution of the
procedures related to meta and namespace before the others? I agree this could
lead to complex code.
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java:
##
@@ -1399,23 +1399,46 @@ private void
createMissingCFsInMetaDuringUpgrade(TableDescriptor metaDescriptor)
* Check hbase:meta is up and ready for reading. For use during Master
startup only.
* @return True if meta is UP and online and startup can progress.
Otherwise, meta is not online
* and we will hold here until operator intervention.
+ * @throws IOException If the master restart is required.
*/
@InterfaceAudience.Private
- public boolean waitForMetaOnline() {
+ public boolean waitForMetaOnline() throws IOException {
return isRegionOnline(RegionInfoBuilder.FIRST_META_REGIONINFO);
}
/**
+ * Wait until the region is reported online on a live regionserver.
+ * @param ri Region info.
* @return True if region is online and scannable else false if an error or
shutdown (Otherwise we
* just block in here holding up all forward-progess).
+ * @throws IOException If the master restart is required.
*/
- private boolean isRegionOnline(RegionInfo ri) {
+ private boolean isRegionOnline(RegionInfo ri) throws IOException {
RetryCounter rc = null;
while (!isStopped()) {
RegionState rs =
this.assignmentManager.getRegionStates().getRegionState(ri);
if (rs != null && rs.isOpened()) {
if (this.getServerManager().isServerOnline(rs.getServerName())) {
return true;
+} else {
+ LOG.info("{} has state {} but the server {} is not online,
scheduling recovery.",
+ri.getRegionNameAsString(), rs, rs.getServerName());
+ this.getServerManager().expireServer(rs.getServerName(), true);
+ // If already many SCPs are scheduled, but they are not progressing
because of
+ // meta's unavailability, the best action item is to throw
PleaseRestartMasterException
+ // and let new active master init take care of on-lining meta and
process all other
+ // pending SCPs. It's worth waiting for ~20s before arriving at the
conclusion, rather
+ // than looping through procedures to figure out how/when/why they
are able to or not
+ // able to make any progress and eventually abort master
initialization anyway.
+ Threads.sleep(2);
Review Comment:
Interrupt is not handled here.
But, more generally, do not unconditionally wait for so long. Use the Waiter
pattern. Why 20 seconds and not some other value? Why not make this
configurable?
##
h
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
apurtell commented on code in PR #5513:
URL: https://github.com/apache/hbase/pull/5513#discussion_r1388610043
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java:
##
@@ -1399,23 +1399,46 @@ private void
createMissingCFsInMetaDuringUpgrade(TableDescriptor metaDescriptor)
* Check hbase:meta is up and ready for reading. For use during Master
startup only.
* @return True if meta is UP and online and startup can progress.
Otherwise, meta is not online
* and we will hold here until operator intervention.
+ * @throws IOException If the master restart is required.
*/
@InterfaceAudience.Private
- public boolean waitForMetaOnline() {
+ public boolean waitForMetaOnline() throws IOException {
return isRegionOnline(RegionInfoBuilder.FIRST_META_REGIONINFO);
}
/**
+ * Wait until the region is reported online on a live regionserver.
+ * @param ri Region info.
* @return True if region is online and scannable else false if an error or
shutdown (Otherwise we
* just block in here holding up all forward-progess).
+ * @throws IOException If the master restart is required.
*/
- private boolean isRegionOnline(RegionInfo ri) {
+ private boolean isRegionOnline(RegionInfo ri) throws IOException {
RetryCounter rc = null;
while (!isStopped()) {
RegionState rs =
this.assignmentManager.getRegionStates().getRegionState(ri);
if (rs != null && rs.isOpened()) {
if (this.getServerManager().isServerOnline(rs.getServerName())) {
return true;
+} else {
+ LOG.info("{} has state {} but the server {} is not online,
scheduling recovery.",
+ri.getRegionNameAsString(), rs, rs.getServerName());
+ this.getServerManager().expireServer(rs.getServerName(), true);
+ // If already many SCPs are scheduled, but they are not progressing
because of
+ // meta's unavailability, the best action item is to throw
PleaseRestartMasterException
+ // and let new active master init take care of on-lining meta and
process all other
+ // pending SCPs. It's worth waiting for ~20s before arriving at the
conclusion, rather
+ // than looping through procedures to figure out how/when/why they
are able to or not
+ // able to make any progress and eventually abort master
initialization anyway.
+ Threads.sleep(2);
+ rs = this.assignmentManager.getRegionStates().getRegionState(ri);
+ if (rs != null && rs.isOpened()) {
+if (this.getServerManager().isServerOnline(rs.getServerName())) {
+ return true;
+} else {
+ throw new PleaseRestartMasterException("meta is still not online
on live server yet");
Review Comment:
What you check here, for this particular region, is if it is supposed to be
online on a server, that the server is considered online. This does not seem to
check anything specifically pertaining to the meta table. If you want to break
out of this loop because meta is not available you should make an explicit
check of the region state for the meta regions.
Also, this is the same logic as if we would just loop around again, so why
not do that? D.R.Y.
Finally, do we really need to restart the master here or is there some other
way to address this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
virajjasani commented on code in PR #5513:
URL: https://github.com/apache/hbase/pull/5513#discussion_r1388621113
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java:
##
@@ -1399,23 +1399,46 @@ private void
createMissingCFsInMetaDuringUpgrade(TableDescriptor metaDescriptor)
* Check hbase:meta is up and ready for reading. For use during Master
startup only.
* @return True if meta is UP and online and startup can progress.
Otherwise, meta is not online
* and we will hold here until operator intervention.
+ * @throws IOException If the master restart is required.
*/
@InterfaceAudience.Private
- public boolean waitForMetaOnline() {
+ public boolean waitForMetaOnline() throws IOException {
return isRegionOnline(RegionInfoBuilder.FIRST_META_REGIONINFO);
}
/**
+ * Wait until the region is reported online on a live regionserver.
+ * @param ri Region info.
* @return True if region is online and scannable else false if an error or
shutdown (Otherwise we
* just block in here holding up all forward-progess).
+ * @throws IOException If the master restart is required.
*/
- private boolean isRegionOnline(RegionInfo ri) {
+ private boolean isRegionOnline(RegionInfo ri) throws IOException {
RetryCounter rc = null;
while (!isStopped()) {
RegionState rs =
this.assignmentManager.getRegionStates().getRegionState(ri);
if (rs != null && rs.isOpened()) {
if (this.getServerManager().isServerOnline(rs.getServerName())) {
return true;
+} else {
+ LOG.info("{} has state {} but the server {} is not online,
scheduling recovery.",
+ri.getRegionNameAsString(), rs, rs.getServerName());
+ this.getServerManager().expireServer(rs.getServerName(), true);
+ // If already many SCPs are scheduled, but they are not progressing
because of
+ // meta's unavailability, the best action item is to throw
PleaseRestartMasterException
+ // and let new active master init take care of on-lining meta and
process all other
+ // pending SCPs. It's worth waiting for ~20s before arriving at the
conclusion, rather
+ // than looping through procedures to figure out how/when/why they
are able to or not
+ // able to make any progress and eventually abort master
initialization anyway.
+ Threads.sleep(2);
+ rs = this.assignmentManager.getRegionStates().getRegionState(ri);
+ if (rs != null && rs.isOpened()) {
+if (this.getServerManager().isServerOnline(rs.getServerName())) {
+ return true;
+} else {
+ throw new PleaseRestartMasterException("meta is still not online
on live server yet");
Review Comment:
> This does not seem to check anything specifically pertaining to the meta
table.
This utility is used by both meta and namespace regions only.
> Finally, do we really need to restart the master here or is there some
other way to address this?
That's what will guarantee prioritizing SCP of the server where meta was
hosted. So we schedule SCP of the server hosting meta and abort master, new
master failover is guaranteed to process SCP of that server, and then continue
processing other SCPs that were stuck due to unavailable meta (due to
inconsistent meta state).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-28085) Configurably use scanner timeout as rpc timeout for scanner next calls
[
https://issues.apache.org/jira/browse/HBASE-28085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault updated HBASE-28085:
--
Fix Version/s: 2.6.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Configurably use scanner timeout as rpc timeout for scanner next calls
> --
>
> Key: HBASE-28085
> URL: https://issues.apache.org/jira/browse/HBASE-28085
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
> Fix For: 2.6.0
>
>
> In the AsyncTable implementation, scanner next() calls use
> "hbase.client.scanner.timeout.period" as the rpc timeout. The reason is
> described in comments, and makes a lot of sense:
> {code:java}
> // As we have a call sequence for scan, it is useless to have a different rpc
> timeout which is
> // less than the scan timeout. If the server does not respond in time(usually
> this will not
> // happen as we have heartbeat now), we will get an
> OutOfOrderScannerNextException when
> // resending the next request and the only way to fix this is to close the
> scanner and open a
> // new one. {code}
> The branch-2 HTable implementation still uses the old behavior – the next()
> call passes the read rpc timeout as the rpc timeout, and uses the scanner
> timeout period as the operation timeout. This can lead to the above behavior.
> It would be nice to provide users a migration path to AsyncTable's behavior,
> in the form of a config flag which causes HTable to use
> "hbase.client.scanner.timeout.period" as rpc timeout for next() calls like
> AsyncTable does.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28085) Configurably use scanner timeout as rpc timeout for scanner next calls
[
https://issues.apache.org/jira/browse/HBASE-28085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Beaudreault updated HBASE-28085:
--
Release Note: Adds a new configuration key:
hbase.client.use.scanner.timeout.period.for.next.calls. The default value is
false to preserve original behavior. When set to true, ClientScanner will use
hbase.client.scanner.timeout.period for the RPC timeout of individual next()
RPC calls. This is in-line with the behavior of AsyncTable and future 3.0
release.
> Configurably use scanner timeout as rpc timeout for scanner next calls
> --
>
> Key: HBASE-28085
> URL: https://issues.apache.org/jira/browse/HBASE-28085
> Project: HBase
> Issue Type: Improvement
>Reporter: Bryan Beaudreault
>Assignee: Bryan Beaudreault
>Priority: Major
> Labels: patch-available
> Fix For: 2.6.0
>
>
> In the AsyncTable implementation, scanner next() calls use
> "hbase.client.scanner.timeout.period" as the rpc timeout. The reason is
> described in comments, and makes a lot of sense:
> {code:java}
> // As we have a call sequence for scan, it is useless to have a different rpc
> timeout which is
> // less than the scan timeout. If the server does not respond in time(usually
> this will not
> // happen as we have heartbeat now), we will get an
> OutOfOrderScannerNextException when
> // resending the next request and the only way to fix this is to close the
> scanner and open a
> // new one. {code}
> The branch-2 HTable implementation still uses the old behavior – the next()
> call passes the read rpc timeout as the rpc timeout, and uses the scanner
> timeout period as the operation timeout. This can lead to the above behavior.
> It would be nice to provide users a migration path to AsyncTable's behavior,
> in the form of a config flag which causes HTable to use
> "hbase.client.scanner.timeout.period" as rpc timeout for next() calls like
> AsyncTable does.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-27276 Reduce reflection overhead in Filter deserialization [hbase]
Apache-HBase commented on PR #5488: URL: https://github.com/apache/hbase/pull/5488#issuecomment-1804823842 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 27s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 3m 30s | master passed | | +1 :green_heart: | compile | 1m 26s | master passed | | +1 :green_heart: | shadedjars | 6m 9s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 5s | master passed | | -0 :warning: | patch | 7m 37s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 11s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 3m 15s | the patch passed | | +1 :green_heart: | compile | 1m 27s | the patch passed | | +1 :green_heart: | javac | 1m 27s | the patch passed | | +1 :green_heart: | shadedjars | 6m 6s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 57s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 3s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 27s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 230m 2s | hbase-server in the patch passed. | | | | 263m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/9/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5488 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 2d17e04470bb 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/9/testReport/ | | Max. process+thread count | 4534 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/9/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-27276 Reduce reflection overhead in Filter deserialization [hbase]
Apache-HBase commented on PR #5488: URL: https://github.com/apache/hbase/pull/5488#issuecomment-1804838158 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 2m 42s | master passed | | +1 :green_heart: | compile | 1m 28s | master passed | | +1 :green_heart: | shadedjars | 4m 58s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 58s | master passed | | -0 :warning: | patch | 6m 22s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 2m 41s | the patch passed | | +1 :green_heart: | compile | 1m 26s | the patch passed | | +1 :green_heart: | javac | 1m 26s | the patch passed | | +1 :green_heart: | shadedjars | 4m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 59s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 2m 30s | hbase-common in the patch passed. | | +1 :green_heart: | unit | 1m 29s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 250m 46s | hbase-server in the patch passed. | | | | 281m 17s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/9/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5488 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 18cffe96c7f3 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/9/testReport/ | | Max. process+thread count | 4759 (vs. ulimit of 3) | | modules | C: hbase-common hbase-client hbase-server U: . | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5488/9/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
virajjasani commented on code in PR #5513:
URL: https://github.com/apache/hbase/pull/5513#discussion_r1388702442
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java:
##
@@ -1399,23 +1399,46 @@ private void
createMissingCFsInMetaDuringUpgrade(TableDescriptor metaDescriptor)
* Check hbase:meta is up and ready for reading. For use during Master
startup only.
* @return True if meta is UP and online and startup can progress.
Otherwise, meta is not online
* and we will hold here until operator intervention.
+ * @throws IOException If the master restart is required.
*/
@InterfaceAudience.Private
- public boolean waitForMetaOnline() {
+ public boolean waitForMetaOnline() throws IOException {
return isRegionOnline(RegionInfoBuilder.FIRST_META_REGIONINFO);
}
/**
+ * Wait until the region is reported online on a live regionserver.
+ * @param ri Region info.
* @return True if region is online and scannable else false if an error or
shutdown (Otherwise we
* just block in here holding up all forward-progess).
+ * @throws IOException If the master restart is required.
*/
- private boolean isRegionOnline(RegionInfo ri) {
+ private boolean isRegionOnline(RegionInfo ri) throws IOException {
RetryCounter rc = null;
while (!isStopped()) {
RegionState rs =
this.assignmentManager.getRegionStates().getRegionState(ri);
if (rs != null && rs.isOpened()) {
if (this.getServerManager().isServerOnline(rs.getServerName())) {
return true;
+} else {
+ LOG.info("{} has state {} but the server {} is not online,
scheduling recovery.",
+ri.getRegionNameAsString(), rs, rs.getServerName());
+ this.getServerManager().expireServer(rs.getServerName(), true);
+ // If already many SCPs are scheduled, but they are not progressing
because of
+ // meta's unavailability, the best action item is to throw
PleaseRestartMasterException
+ // and let new active master init take care of on-lining meta and
process all other
+ // pending SCPs. It's worth waiting for ~20s before arriving at the
conclusion, rather
+ // than looping through procedures to figure out how/when/why they
are able to or not
+ // able to make any progress and eventually abort master
initialization anyway.
+ Threads.sleep(2);
Review Comment:
introduced 2 configs to improve the retry logic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
virajjasani commented on code in PR #5513:
URL: https://github.com/apache/hbase/pull/5513#discussion_r1388702683
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java:
##
@@ -1399,23 +1399,46 @@ private void
createMissingCFsInMetaDuringUpgrade(TableDescriptor metaDescriptor)
* Check hbase:meta is up and ready for reading. For use during Master
startup only.
* @return True if meta is UP and online and startup can progress.
Otherwise, meta is not online
* and we will hold here until operator intervention.
+ * @throws IOException If the master restart is required.
*/
@InterfaceAudience.Private
- public boolean waitForMetaOnline() {
+ public boolean waitForMetaOnline() throws IOException {
return isRegionOnline(RegionInfoBuilder.FIRST_META_REGIONINFO);
}
/**
+ * Wait until the region is reported online on a live regionserver.
+ * @param ri Region info.
* @return True if region is online and scannable else false if an error or
shutdown (Otherwise we
* just block in here holding up all forward-progess).
+ * @throws IOException If the master restart is required.
*/
- private boolean isRegionOnline(RegionInfo ri) {
+ private boolean isRegionOnline(RegionInfo ri) throws IOException {
RetryCounter rc = null;
while (!isStopped()) {
RegionState rs =
this.assignmentManager.getRegionStates().getRegionState(ri);
if (rs != null && rs.isOpened()) {
if (this.getServerManager().isServerOnline(rs.getServerName())) {
return true;
+} else {
+ LOG.info("{} has state {} but the server {} is not online,
scheduling recovery.",
+ri.getRegionNameAsString(), rs, rs.getServerName());
+ this.getServerManager().expireServer(rs.getServerName(), true);
+ // If already many SCPs are scheduled, but they are not progressing
because of
+ // meta's unavailability, the best action item is to throw
PleaseRestartMasterException
+ // and let new active master init take care of on-lining meta and
process all other
+ // pending SCPs. It's worth waiting for ~20s before arriving at the
conclusion, rather
Review Comment:
added proc state based waiting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
virajjasani commented on code in PR #5513:
URL: https://github.com/apache/hbase/pull/5513#discussion_r1388705399
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java:
##
@@ -1399,23 +1399,46 @@ private void
createMissingCFsInMetaDuringUpgrade(TableDescriptor metaDescriptor)
* Check hbase:meta is up and ready for reading. For use during Master
startup only.
* @return True if meta is UP and online and startup can progress.
Otherwise, meta is not online
* and we will hold here until operator intervention.
+ * @throws IOException If the master restart is required.
*/
@InterfaceAudience.Private
- public boolean waitForMetaOnline() {
+ public boolean waitForMetaOnline() throws IOException {
return isRegionOnline(RegionInfoBuilder.FIRST_META_REGIONINFO);
}
/**
+ * Wait until the region is reported online on a live regionserver.
+ * @param ri Region info.
* @return True if region is online and scannable else false if an error or
shutdown (Otherwise we
* just block in here holding up all forward-progess).
+ * @throws IOException If the master restart is required.
*/
- private boolean isRegionOnline(RegionInfo ri) {
+ private boolean isRegionOnline(RegionInfo ri) throws IOException {
RetryCounter rc = null;
while (!isStopped()) {
RegionState rs =
this.assignmentManager.getRegionStates().getRegionState(ri);
if (rs != null && rs.isOpened()) {
if (this.getServerManager().isServerOnline(rs.getServerName())) {
return true;
+} else {
+ LOG.info("{} has state {} but the server {} is not online,
scheduling recovery.",
+ri.getRegionNameAsString(), rs, rs.getServerName());
+ this.getServerManager().expireServer(rs.getServerName(), true);
+ // If already many SCPs are scheduled, but they are not progressing
because of
+ // meta's unavailability, the best action item is to throw
PleaseRestartMasterException
+ // and let new active master init take care of on-lining meta and
process all other
+ // pending SCPs. It's worth waiting for ~20s before arriving at the
conclusion, rather
+ // than looping through procedures to figure out how/when/why they
are able to or not
+ // able to make any progress and eventually abort master
initialization anyway.
+ Threads.sleep(2);
+ rs = this.assignmentManager.getRegionStates().getRegionState(ri);
+ if (rs != null && rs.isOpened()) {
+if (this.getServerManager().isServerOnline(rs.getServerName())) {
+ return true;
+} else {
+ throw new PleaseRestartMasterException("meta is still not online
on live server yet");
Review Comment:
Tried improving this overall in the latest revision.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
Apache-HBase commented on PR #5513: URL: https://github.com/apache/hbase/pull/5513#issuecomment-1804911062 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 27s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 36s | master passed | | +1 :green_heart: | compile | 0m 37s | master passed | | +1 :green_heart: | shadedjars | 5m 23s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 19s | the patch passed | | +1 :green_heart: | compile | 0m 38s | the patch passed | | +1 :green_heart: | javac | 0m 38s | the patch passed | | +1 :green_heart: | shadedjars | 5m 22s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 256m 21s | hbase-server in the patch failed. | | | | 278m 16s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5513 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 8e84fa2d2746 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/1/testReport/ | | Max. process+thread count | 4518 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
Apache-HBase commented on PR #5513: URL: https://github.com/apache/hbase/pull/5513#issuecomment-1804913627 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 43s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 32s | master passed | | +1 :green_heart: | compile | 1m 2s | master passed | | +1 :green_heart: | shadedjars | 5m 48s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 31s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 14s | the patch passed | | +1 :green_heart: | compile | 0m 49s | the patch passed | | +1 :green_heart: | javac | 0m 49s | the patch passed | | +1 :green_heart: | shadedjars | 4m 57s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 26s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 256m 42s | hbase-server in the patch failed. | | | | 282m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5513 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 0ba8245296d2 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/1/testReport/ | | Max. process+thread count | 5086 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/1/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28192) Master should recover if meta region state is inconsistent
[
https://issues.apache.org/jira/browse/HBASE-28192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784639#comment-17784639
]
Duo Zhang commented on HBASE-28192:
---
{quote}
In one of the recent outage, we observed that meta was online on a server,
which was correctly reflected in meta znode, but the server starttime was
different. This means that as per the latest transition record, meta was marked
online on old server (same server with old start time).
{quote}
How could this happen?
> Master should recover if meta region state is inconsistent
> --
>
> Key: HBASE-28192
> URL: https://issues.apache.org/jira/browse/HBASE-28192
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.4.17, 2.5.6
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> During active master initialization, before we set master as active (i.e.
> {_}setInitialized(true){_}), we need both meta and namespace regions online.
> If the region state of meta or namespace is inconsistent, active master can
> get stuck in the initialization step:
> {code:java}
> private boolean isRegionOnline(RegionInfo ri) {
> RetryCounter rc = null;
> while (!isStopped()) {
> ...
> ...
> ...
> // Check once-a-minute.
> if (rc == null) {
> rc = new RetryCounterFactory(Integer.MAX_VALUE, 1000, 60_000).create();
> }
> Threads.sleep(rc.getBackoffTimeAndIncrementAttempts());
> }
> return false;
> }
> {code}
> In one of the recent outage, we observed that meta was online on a server,
> which was correctly reflected in meta znode, but the server starttime was
> different. This means that as per the latest transition record, meta was
> marked online on old server (same server with old start time). This kept
> active master initialization waiting forever and some SCPs got stuck in
> initial stage where they need to access meta table before getting candidate
> for region moves.
> The only way out of this outage is for operator to schedule recoveries using
> hbck for old server, which triggers SCP for old server address of meta. Since
> many SCPs were stuck, the processing of new SCP too was taking some time and
> manual restart of active master triggered failover, and new master was able
> to complete SCP for old meta server, correcting the meta assignment details,
> which eventually marked master as active and only after this, we were able to
> see real large num of RITs that were hidden so far.
> We need to let master recover from this state to avoid manual intervention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
Apache-HBase commented on PR #5513: URL: https://github.com/apache/hbase/pull/5513#issuecomment-1804938722 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 32s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 53s | master passed | | +1 :green_heart: | compile | 2m 26s | master passed | | +1 :green_heart: | checkstyle | 0m 36s | master passed | | +1 :green_heart: | spotless | 0m 43s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 33s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 35s | the patch passed | | +1 :green_heart: | compile | 2m 25s | the patch passed | | +1 :green_heart: | javac | 2m 25s | the patch passed | | +1 :green_heart: | checkstyle | 0m 35s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 27s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 41s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 12s | The patch does not generate ASF License warnings. | | | | 31m 53s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5513 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux e069f5f58e6c 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28174 [hbase]
Apache-HBase commented on PR #5501: URL: https://github.com/apache/hbase/pull/5501#issuecomment-1804967830 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 23s | master passed | | +1 :green_heart: | compile | 0m 17s | master passed | | +1 :green_heart: | shadedjars | 4m 52s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 13s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 19s | the patch passed | | +1 :green_heart: | compile | 0m 17s | the patch passed | | +1 :green_heart: | javac | 0m 17s | the patch passed | | +1 :green_heart: | shadedjars | 4m 49s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 13s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 3m 19s | hbase-rest in the patch passed. | | | | 20m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5501 | | JIRA Issue | HBASE-28174 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux d0f18b290560 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/7/testReport/ | | Max. process+thread count | 1621 (vs. ulimit of 3) | | modules | C: hbase-rest U: hbase-rest | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/7/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28192) Master should recover if meta region state is inconsistent
[
https://issues.apache.org/jira/browse/HBASE-28192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784650#comment-17784650
]
Viraj Jasani commented on HBASE-28192:
--
[~zhangduo] i am not aware of the exact root cause but this was hbase 2.4 to
2.5 upgrade and HBASE-26193 might be suspect, i am not sure, need to dig in,
but let's say we do know the reason and there could be something else tomorrow
that can make active master init stuck in the loop, it's not good anyway right?
If meta is online but not on live server, master should be able to recover. Any
cause should be handled separately too, but right now we let master stuck in
infinite loop for this edge case, which is also not reliable IMO.
> Master should recover if meta region state is inconsistent
> --
>
> Key: HBASE-28192
> URL: https://issues.apache.org/jira/browse/HBASE-28192
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.4.17, 2.5.6
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> During active master initialization, before we set master as active (i.e.
> {_}setInitialized(true){_}), we need both meta and namespace regions online.
> If the region state of meta or namespace is inconsistent, active master can
> get stuck in the initialization step:
> {code:java}
> private boolean isRegionOnline(RegionInfo ri) {
> RetryCounter rc = null;
> while (!isStopped()) {
> ...
> ...
> ...
> // Check once-a-minute.
> if (rc == null) {
> rc = new RetryCounterFactory(Integer.MAX_VALUE, 1000, 60_000).create();
> }
> Threads.sleep(rc.getBackoffTimeAndIncrementAttempts());
> }
> return false;
> }
> {code}
> In one of the recent outage, we observed that meta was online on a server,
> which was correctly reflected in meta znode, but the server starttime was
> different. This means that as per the latest transition record, meta was
> marked online on old server (same server with old start time). This kept
> active master initialization waiting forever and some SCPs got stuck in
> initial stage where they need to access meta table before getting candidate
> for region moves.
> The only way out of this outage is for operator to schedule recoveries using
> hbck for old server, which triggers SCP for old server address of meta. Since
> many SCPs were stuck, the processing of new SCP too was taking some time and
> manual restart of active master triggered failover, and new master was able
> to complete SCP for old meta server, correcting the meta assignment details,
> which eventually marked master as active and only after this, we were able to
> see real large num of RITs that were hidden so far.
> We need to let master recover from this state to avoid manual intervention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28174 [hbase]
Apache-HBase commented on PR #5501: URL: https://github.com/apache/hbase/pull/5501#issuecomment-1804971303 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 31s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 50s | master passed | | +1 :green_heart: | compile | 0m 24s | master passed | | +1 :green_heart: | checkstyle | 0m 8s | master passed | | +1 :green_heart: | spotless | 0m 39s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 28s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 38s | the patch passed | | +1 :green_heart: | compile | 0m 24s | the patch passed | | -0 :warning: | javac | 0m 24s | hbase-rest generated 11 new + 154 unchanged - 0 fixed = 165 total (was 154) | | -0 :warning: | checkstyle | 0m 7s | hbase-rest: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 53s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 0m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 7s | The patch does not generate ASF License warnings. | | | | 25m 22s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5501 | | JIRA Issue | HBASE-28174 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 8e7ddd655d8a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Eclipse Adoptium-11.0.17+8 | | javac | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/7/artifact/yetus-general-check/output/diff-compile-javac-hbase-rest.txt | | checkstyle | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/7/artifact/yetus-general-check/output/diff-checkstyle-hbase-rest.txt | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-rest U: hbase-rest | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/7/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28174 [hbase]
Apache-HBase commented on PR #5501: URL: https://github.com/apache/hbase/pull/5501#issuecomment-1804972679 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 2s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 32s | master passed | | +1 :green_heart: | compile | 0m 24s | master passed | | +1 :green_heart: | shadedjars | 6m 7s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 16s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 25s | the patch passed | | +1 :green_heart: | compile | 0m 21s | the patch passed | | +1 :green_heart: | javac | 0m 21s | the patch passed | | +1 :green_heart: | shadedjars | 6m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 4m 36s | hbase-rest in the patch passed. | | | | 27m 12s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5501 | | JIRA Issue | HBASE-28174 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 10324b3607bc 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/7/testReport/ | | Max. process+thread count | 1594 (vs. ulimit of 3) | | modules | C: hbase-rest U: hbase-rest | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5501/7/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-28192) Master should recover if meta region state is inconsistent
[
https://issues.apache.org/jira/browse/HBASE-28192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784650#comment-17784650
]
Viraj Jasani edited comment on HBASE-28192 at 11/10/23 2:41 AM:
[~zhangduo] i am not aware of the exact root cause but this was hbase 2.4 to
2.5 upgrade and HBASE-26193 might be suspect, i am not sure, need to dig in,
but let's say we get to know about the cause and resolve it, but there could be
something else tomorrow that can make active master init stuck in the loop,
maybe during upgrade or maybe during usual restarts, it's not good anyway right?
If meta is online but not on live server, master should be able to recover. Any
cause should be handled separately too, but right now we let master get stuck
in infinite loop for this edge case, which is also not reliable IMO. At least
we should not expect operator to perform hbck recovery for meta and/or
namespace regions while master stay stuck forever in loop.
was (Author: vjasani):
[~zhangduo] i am not aware of the exact root cause but this was hbase 2.4 to
2.5 upgrade and HBASE-26193 might be suspect, i am not sure, need to dig in,
but let's say we do know the reason and there could be something else tomorrow
that can make active master init stuck in the loop, it's not good anyway right?
If meta is online but not on live server, master should be able to recover. Any
cause should be handled separately too, but right now we let master stuck in
infinite loop for this edge case, which is also not reliable IMO.
> Master should recover if meta region state is inconsistent
> --
>
> Key: HBASE-28192
> URL: https://issues.apache.org/jira/browse/HBASE-28192
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.4.17, 2.5.6
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> During active master initialization, before we set master as active (i.e.
> {_}setInitialized(true){_}), we need both meta and namespace regions online.
> If the region state of meta or namespace is inconsistent, active master can
> get stuck in the initialization step:
> {code:java}
> private boolean isRegionOnline(RegionInfo ri) {
> RetryCounter rc = null;
> while (!isStopped()) {
> ...
> ...
> ...
> // Check once-a-minute.
> if (rc == null) {
> rc = new RetryCounterFactory(Integer.MAX_VALUE, 1000, 60_000).create();
> }
> Threads.sleep(rc.getBackoffTimeAndIncrementAttempts());
> }
> return false;
> }
> {code}
> In one of the recent outage, we observed that meta was online on a server,
> which was correctly reflected in meta znode, but the server starttime was
> different. This means that as per the latest transition record, meta was
> marked online on old server (same server with old start time). This kept
> active master initialization waiting forever and some SCPs got stuck in
> initial stage where they need to access meta table before getting candidate
> for region moves.
> The only way out of this outage is for operator to schedule recoveries using
> hbck for old server, which triggers SCP for old server address of meta. Since
> many SCPs were stuck, the processing of new SCP too was taking some time and
> manual restart of active master triggered failover, and new master was able
> to complete SCP for old meta server, correcting the meta assignment details,
> which eventually marked master as active and only after this, we were able to
> see real large num of RITs that were hidden so far.
> We need to let master recover from this state to avoid manual intervention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28192) Master should recover if meta region state is inconsistent
[
https://issues.apache.org/jira/browse/HBASE-28192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784659#comment-17784659
]
Viraj Jasani commented on HBASE-28192:
--
Let me add some logs:
regionserver where meta is online:
{code:java}
2023-11-08 18:10:31,079 INFO [MemStoreFlusher.1] regionserver.HStore - Added
hdfs://{cluster}/hbase/data/hbase/meta/1588230740/rep_barrier/3e5faf652f1e4c6db1c4ba1ae676c3ee,
entries=1630, sequenceid=94325525, filesize=362.1 K {code}
master server 4 which thought it was active:
{code:java}
2023-11-08 18:14:34,563 DEBUG [0:becomeActiveMaster]
assignment.AssignmentManager - Loaded hbase:meta state=OPEN,
location=server3-1,61020,1699456864765, table=hbase:meta, region=1588230740
2023-11-08 18:14:34,609 INFO [0:becomeActiveMaster] master.ServerManager -
Registering regionserver=server3-1,61020,1699456864765 {code}
master server 1 which thought it was active:
{code:java}
2023-11-08 18:15:50,350 DEBUG [aster/server1:61000:becomeActiveMaster]
assignment.AssignmentManager - Loaded hbase:meta state=OPEN,
location=server3-1,61020,1698687384632, table=hbase:meta, region=1588230740
2023-11-08 18:15:50,399 INFO [aster/server1:61000:becomeActiveMaster]
master.ServerManager - Registering regionserver=server3-1,61020,1699456864765
{code}
master server 4 gave up:
{code:java}
2023-11-08 18:16:22,776 INFO [aster/server4:61000:becomeActiveMaster]
master.ActiveMasterManager - Another master is the active master,
server1,61000,1699467212235; waiting to become the next active master {code}
When server 4 was trying to be active master and loaded meta, it retrieved the
correct location of meta i.e. server3-1,61020,1699456864765
However, when server 1 (eventual active master) loaded meta, it retrieved
incorrect location i.e. server3-1,61020,1698687384632
For hbase 2.5, i see that HBASE-26193 no longer relies on zookeeper and rather
relies on scanning master region:
{code:java}
// Start the Assignment Thread
startAssignmentThread();
// load meta region states.
// here we are still in the early steps of active master startup. There is
only one thread(us)
// can access AssignmentManager and create region node, so here we do not
need to lock the
// region node.
try (ResultScanner scanner =
masterRegion.getScanner(new Scan().addFamily(HConstants.CATALOG_FAMILY))) {
for (;;) {
Result result = scanner.next();
if (result == null) {
break;
}
RegionStateStore
.visitMetaEntry((r, regionInfo, state, regionLocation, lastHost,
openSeqNum) -> {
RegionStateNode regionNode =
regionStates.getOrCreateRegionStateNode(regionInfo);
regionNode.setState(state);
regionNode.setLastHost(lastHost);
regionNode.setRegionLocation(regionLocation);
regionNode.setOpenSeqNum(openSeqNum);
if (regionNode.getProcedure() != null) {
regionNode.getProcedure().stateLoaded(this, regionNode);
}
if (regionLocation != null) {
regionStates.addRegionToServer(regionNode);
}
if (RegionReplicaUtil.isDefaultReplica(regionInfo.getReplicaId())) {
setMetaAssigned(regionInfo, state == State.OPEN);
}
LOG.debug("Loaded hbase:meta {}", regionNode);
}, result);
}
}
mirrorMetaLocations();
}
{code}
> Master should recover if meta region state is inconsistent
> --
>
> Key: HBASE-28192
> URL: https://issues.apache.org/jira/browse/HBASE-28192
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.4.17, 2.5.6
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> During active master initialization, before we set master as active (i.e.
> {_}setInitialized(true){_}), we need both meta and namespace regions online.
> If the region state of meta or namespace is inconsistent, active master can
> get stuck in the initialization step:
> {code:java}
> private boolean isRegionOnline(RegionInfo ri) {
> RetryCounter rc = null;
> while (!isStopped()) {
> ...
> ...
> ...
> // Check once-a-minute.
> if (rc == null) {
> rc = new RetryCounterFactory(Integer.MAX_VALUE, 1000, 60_000).create();
> }
> Threads.sleep(rc.getBackoffTimeAndIncrementAttempts());
> }
> return false;
> }
> {code}
> In one of the recent outage, we observed that meta was online on a server,
> which was correctly reflected in meta znode, but the server starttime was
> different. This means that as per the latest transition record, meta was
> marked online on old server (same server with old start time). This kept
> active master initialization waiting forever and some SCPs got stuck in
> initial
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
virajjasani commented on PR #5513: URL: https://github.com/apache/hbase/pull/5513#issuecomment-1805005442 > Unless we know what is root cause I'm always -1 for doing things like this in our normal code logic. HBCK is the correct way for fixing the incosistency which is caused by a code bug. > > So why there is no SCP for the old server after it is already dead? Added some comments on Jira, still it's suspicious, not a guaranteed root cause and maybe this can happen only during upgrade from 2.4 to 2.5? Let me check what happened to SCP of old server. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (HBASE-28192) Master should recover if meta region state is inconsistent
[
https://issues.apache.org/jira/browse/HBASE-28192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784659#comment-17784659
]
Viraj Jasani edited comment on HBASE-28192 at 11/10/23 3:15 AM:
Let me add some logs:
regionserver where meta is online:
{code:java}
2023-11-08 18:10:31,079 INFO [MemStoreFlusher.1] regionserver.HStore - Added
hdfs://{cluster}/hbase/data/hbase/meta/1588230740/rep_barrier/3e5faf652f1e4c6db1c4ba1ae676c3ee,
entries=1630, sequenceid=94325525, filesize=362.1 K {code}
master server 4 which thought it was active:
{code:java}
2023-11-08 18:14:34,563 DEBUG [0:becomeActiveMaster]
assignment.AssignmentManager - Loaded hbase:meta state=OPEN,
location=server3-1,61020,1699456864765, table=hbase:meta, region=1588230740
2023-11-08 18:14:34,609 INFO [0:becomeActiveMaster] master.ServerManager -
Registering regionserver=server3-1,61020,1699456864765 {code}
master server 1 which thought it was active:
{code:java}
2023-11-08 18:15:50,350 DEBUG [aster/server1:61000:becomeActiveMaster]
assignment.AssignmentManager - Loaded hbase:meta state=OPEN,
location=server3-1,61020,1698687384632, table=hbase:meta, region=1588230740
2023-11-08 18:15:50,399 INFO [aster/server1:61000:becomeActiveMaster]
master.ServerManager - Registering regionserver=server3-1,61020,1699456864765
{code}
master server 4 gave up:
{code:java}
2023-11-08 18:16:22,776 INFO [aster/server4:61000:becomeActiveMaster]
master.ActiveMasterManager - Another master is the active master,
server1,61000,1699467212235; waiting to become the next active master {code}
When server 4 was trying to be active master and loaded meta, it retrieved the
correct location of meta i.e. server3-1,61020,1699456864765
However, when server 1 (eventual active master) loaded meta, it retrieved
incorrect location i.e. server3-1,61020,1698687384632
For hbase 2.5, i see that HBASE-26193 no longer relies on zookeeper and rather
relies on scanning master region:
{code:java}
// Start the Assignment Thread
startAssignmentThread();
// load meta region states.
// here we are still in the early steps of active master startup. There is
only one thread(us)
// can access AssignmentManager and create region node, so here we do not
need to lock the
// region node.
try (ResultScanner scanner =
masterRegion.getScanner(new Scan().addFamily(HConstants.CATALOG_FAMILY))) {
for (;;) {
Result result = scanner.next();
if (result == null) {
break;
}
RegionStateStore
.visitMetaEntry((r, regionInfo, state, regionLocation, lastHost,
openSeqNum) -> {
RegionStateNode regionNode =
regionStates.getOrCreateRegionStateNode(regionInfo);
regionNode.setState(state);
regionNode.setLastHost(lastHost);
regionNode.setRegionLocation(regionLocation);
regionNode.setOpenSeqNum(openSeqNum);
if (regionNode.getProcedure() != null) {
regionNode.getProcedure().stateLoaded(this, regionNode);
}
if (regionLocation != null) {
regionStates.addRegionToServer(regionNode);
}
if (RegionReplicaUtil.isDefaultReplica(regionInfo.getReplicaId())) {
setMetaAssigned(regionInfo, state == State.OPEN);
}
LOG.debug("Loaded hbase:meta {}", regionNode);
}, result);
}
}
mirrorMetaLocations();
}
{code}
Maybe this incident was one-off case, maybe only happens during hbase 2.4 to
2.5 upgrade. Once meta location is only read from master region (for 2.5+
releases), there should not be any inconsistency I think.
was (Author: vjasani):
Let me add some logs:
regionserver where meta is online:
{code:java}
2023-11-08 18:10:31,079 INFO [MemStoreFlusher.1] regionserver.HStore - Added
hdfs://{cluster}/hbase/data/hbase/meta/1588230740/rep_barrier/3e5faf652f1e4c6db1c4ba1ae676c3ee,
entries=1630, sequenceid=94325525, filesize=362.1 K {code}
master server 4 which thought it was active:
{code:java}
2023-11-08 18:14:34,563 DEBUG [0:becomeActiveMaster]
assignment.AssignmentManager - Loaded hbase:meta state=OPEN,
location=server3-1,61020,1699456864765, table=hbase:meta, region=1588230740
2023-11-08 18:14:34,609 INFO [0:becomeActiveMaster] master.ServerManager -
Registering regionserver=server3-1,61020,1699456864765 {code}
master server 1 which thought it was active:
{code:java}
2023-11-08 18:15:50,350 DEBUG [aster/server1:61000:becomeActiveMaster]
assignment.AssignmentManager - Loaded hbase:meta state=OPEN,
location=server3-1,61020,1698687384632, table=hbase:meta, region=1588230740
2023-11-08 18:15:50,399 INFO [aster/server1:61000:becomeActiveMaster]
master.ServerManager - Registering regionserver=server3-1,61020,1699456864765
{code}
master server 4 gave up:
{code:java}
2023-11-08 18:16:22,776 INFO [aster/server4:61000:becomeActiveMaster]
master.ActiveMasterManage
[jira] [Comment Edited] (HBASE-28192) Master should recover if meta region state is inconsistent
[
https://issues.apache.org/jira/browse/HBASE-28192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784659#comment-17784659
]
Viraj Jasani edited comment on HBASE-28192 at 11/10/23 3:15 AM:
Let me add some logs:
regionserver where meta is online:
{code:java}
2023-11-08 18:10:31,079 INFO [MemStoreFlusher.1] regionserver.HStore - Added
hdfs://{cluster}/hbase/data/hbase/meta/1588230740/rep_barrier/3e5faf652f1e4c6db1c4ba1ae676c3ee,
entries=1630, sequenceid=94325525, filesize=362.1 K {code}
master server 4 which thought it was active:
{code:java}
2023-11-08 18:14:34,563 DEBUG [0:becomeActiveMaster]
assignment.AssignmentManager - Loaded hbase:meta state=OPEN,
location=server3-1,61020,1699456864765, table=hbase:meta, region=1588230740
2023-11-08 18:14:34,609 INFO [0:becomeActiveMaster] master.ServerManager -
Registering regionserver=server3-1,61020,1699456864765 {code}
master server 1 which thought it was active:
{code:java}
2023-11-08 18:15:50,350 DEBUG [aster/server1:61000:becomeActiveMaster]
assignment.AssignmentManager - Loaded hbase:meta state=OPEN,
location=server3-1,61020,1698687384632, table=hbase:meta, region=1588230740
2023-11-08 18:15:50,399 INFO [aster/server1:61000:becomeActiveMaster]
master.ServerManager - Registering regionserver=server3-1,61020,1699456864765
{code}
master server 4 gave up:
{code:java}
2023-11-08 18:16:22,776 INFO [aster/server4:61000:becomeActiveMaster]
master.ActiveMasterManager - Another master is the active master,
server1,61000,1699467212235; waiting to become the next active master {code}
When server 4 was trying to be active master and loaded meta, it retrieved the
correct location of meta i.e. server3-1,61020,1699456864765
However, when server 1 (eventual active master) loaded meta, it retrieved
incorrect location i.e. server3-1,61020,1698687384632
For hbase 2.5, i see that HBASE-26193 no longer relies on zookeeper and rather
relies on scanning master region:
{code:java}
// Start the Assignment Thread
startAssignmentThread();
// load meta region states.
// here we are still in the early steps of active master startup. There is
only one thread(us)
// can access AssignmentManager and create region node, so here we do not
need to lock the
// region node.
try (ResultScanner scanner =
masterRegion.getScanner(new Scan().addFamily(HConstants.CATALOG_FAMILY))) {
for (;;) {
Result result = scanner.next();
if (result == null) {
break;
}
RegionStateStore
.visitMetaEntry((r, regionInfo, state, regionLocation, lastHost,
openSeqNum) -> {
RegionStateNode regionNode =
regionStates.getOrCreateRegionStateNode(regionInfo);
regionNode.setState(state);
regionNode.setLastHost(lastHost);
regionNode.setRegionLocation(regionLocation);
regionNode.setOpenSeqNum(openSeqNum);
if (regionNode.getProcedure() != null) {
regionNode.getProcedure().stateLoaded(this, regionNode);
}
if (regionLocation != null) {
regionStates.addRegionToServer(regionNode);
}
if (RegionReplicaUtil.isDefaultReplica(regionInfo.getReplicaId())) {
setMetaAssigned(regionInfo, state == State.OPEN);
}
LOG.debug("Loaded hbase:meta {}", regionNode);
}, result);
}
}
mirrorMetaLocations();
}
{code}
Maybe this incident was one-off case, maybe only happens during hbase 2.4 to
2.5 upgrade. Once meta location is only read from master region, there should
not be inconsistency I think.
was (Author: vjasani):
Let me add some logs:
regionserver where meta is online:
{code:java}
2023-11-08 18:10:31,079 INFO [MemStoreFlusher.1] regionserver.HStore - Added
hdfs://{cluster}/hbase/data/hbase/meta/1588230740/rep_barrier/3e5faf652f1e4c6db1c4ba1ae676c3ee,
entries=1630, sequenceid=94325525, filesize=362.1 K {code}
master server 4 which thought it was active:
{code:java}
2023-11-08 18:14:34,563 DEBUG [0:becomeActiveMaster]
assignment.AssignmentManager - Loaded hbase:meta state=OPEN,
location=server3-1,61020,1699456864765, table=hbase:meta, region=1588230740
2023-11-08 18:14:34,609 INFO [0:becomeActiveMaster] master.ServerManager -
Registering regionserver=server3-1,61020,1699456864765 {code}
master server 1 which thought it was active:
{code:java}
2023-11-08 18:15:50,350 DEBUG [aster/server1:61000:becomeActiveMaster]
assignment.AssignmentManager - Loaded hbase:meta state=OPEN,
location=server3-1,61020,1698687384632, table=hbase:meta, region=1588230740
2023-11-08 18:15:50,399 INFO [aster/server1:61000:becomeActiveMaster]
master.ServerManager - Registering regionserver=server3-1,61020,1699456864765
{code}
master server 4 gave up:
{code:java}
2023-11-08 18:16:22,776 INFO [aster/server4:61000:becomeActiveMaster]
master.ActiveMasterManager - Another master is th
[jira] [Commented] (HBASE-28192) Master should recover if meta region state is inconsistent
[
https://issues.apache.org/jira/browse/HBASE-28192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784672#comment-17784672
]
Duo Zhang commented on HBASE-28192:
---
{quote}
but let's say we get to know about the cause and resolve it, but there could be
something else tomorrow that can make active master init stuck in the loop,
maybe during upgrade or maybe during usual restarts, it's not good anyway right?
{quote}
Actually, the most dangerous thing is always that, people think they can fix
something without knowing the root cause and then they just make thing worse...
If you even do not know what is the root cause which causes the meta region can
not be online, how can you make sure that the code you added can fix the
problem? What if it just make the meta data corruption? This is the point
here...
> Master should recover if meta region state is inconsistent
> --
>
> Key: HBASE-28192
> URL: https://issues.apache.org/jira/browse/HBASE-28192
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.4.17, 2.5.6
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> During active master initialization, before we set master as active (i.e.
> {_}setInitialized(true){_}), we need both meta and namespace regions online.
> If the region state of meta or namespace is inconsistent, active master can
> get stuck in the initialization step:
> {code:java}
> private boolean isRegionOnline(RegionInfo ri) {
> RetryCounter rc = null;
> while (!isStopped()) {
> ...
> ...
> ...
> // Check once-a-minute.
> if (rc == null) {
> rc = new RetryCounterFactory(Integer.MAX_VALUE, 1000, 60_000).create();
> }
> Threads.sleep(rc.getBackoffTimeAndIncrementAttempts());
> }
> return false;
> }
> {code}
> In one of the recent outage, we observed that meta was online on a server,
> which was correctly reflected in meta znode, but the server starttime was
> different. This means that as per the latest transition record, meta was
> marked online on old server (same server with old start time). This kept
> active master initialization waiting forever and some SCPs got stuck in
> initial stage where they need to access meta table before getting candidate
> for region moves.
> The only way out of this outage is for operator to schedule recoveries using
> hbck for old server, which triggers SCP for old server address of meta. Since
> many SCPs were stuck, the processing of new SCP too was taking some time and
> manual restart of active master triggered failover, and new master was able
> to complete SCP for old meta server, correcting the meta assignment details,
> which eventually marked master as active and only after this, we were able to
> see real large num of RITs that were hidden so far.
> We need to let master recover from this state to avoid manual intervention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28192) Master should recover if meta region state is inconsistent
[
https://issues.apache.org/jira/browse/HBASE-28192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784673#comment-17784673
]
Duo Zhang commented on HBASE-28192:
---
So here, meta is already online on server3-1,61020,1699456864765, but after
server1 becomes active, the loaded meta location is
server3-1,61020,1698687384632, which is a dead server?
And this happens on a rolling upgrading from 2.4 to 2.5? What is the version
for server1 and server4? Server4 is 2.4.x and server and server1 is 2.5.x?
> Master should recover if meta region state is inconsistent
> --
>
> Key: HBASE-28192
> URL: https://issues.apache.org/jira/browse/HBASE-28192
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.4.17, 2.5.6
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> During active master initialization, before we set master as active (i.e.
> {_}setInitialized(true){_}), we need both meta and namespace regions online.
> If the region state of meta or namespace is inconsistent, active master can
> get stuck in the initialization step:
> {code:java}
> private boolean isRegionOnline(RegionInfo ri) {
> RetryCounter rc = null;
> while (!isStopped()) {
> ...
> ...
> ...
> // Check once-a-minute.
> if (rc == null) {
> rc = new RetryCounterFactory(Integer.MAX_VALUE, 1000, 60_000).create();
> }
> Threads.sleep(rc.getBackoffTimeAndIncrementAttempts());
> }
> return false;
> }
> {code}
> In one of the recent outage, we observed that meta was online on a server,
> which was correctly reflected in meta znode, but the server starttime was
> different. This means that as per the latest transition record, meta was
> marked online on old server (same server with old start time). This kept
> active master initialization waiting forever and some SCPs got stuck in
> initial stage where they need to access meta table before getting candidate
> for region moves.
> The only way out of this outage is for operator to schedule recoveries using
> hbck for old server, which triggers SCP for old server address of meta. Since
> many SCPs were stuck, the processing of new SCP too was taking some time and
> manual restart of active master triggered failover, and new master was able
> to complete SCP for old meta server, correcting the meta assignment details,
> which eventually marked master as active and only after this, we were able to
> see real large num of RITs that were hidden so far.
> We need to let master recover from this state to avoid manual intervention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28193 Update plugin for SBOM generation to 2.7.10 [hbase]
VinodAnandan commented on PR #5485: URL: https://github.com/apache/hbase/pull/5485#issuecomment-1805083445 > Hi, @VinodAnandan , thanks for the PR. > > HBase uses JIRA to track issues, so please follow the guide on our readme page to request a jira account and then file a jira issue for this problem. @Apache9 Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-28193) Update plugin for SBOM generation to 2.7.10
Vinod Anandan created HBASE-28193: - Summary: Update plugin for SBOM generation to 2.7.10 Key: HBASE-28193 URL: https://issues.apache.org/jira/browse/HBASE-28193 Project: HBase Issue Type: Improvement Reporter: Vinod Anandan Update the CycloneDX Maven plugin for SBOM generation to 2.7.10 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28192) Master should recover if meta region state is inconsistent
[
https://issues.apache.org/jira/browse/HBASE-28192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784677#comment-17784677
]
Viraj Jasani commented on HBASE-28192:
--
{quote}Actually, the most dangerous thing is always that, people think they can
fix something without knowing the root cause and then they just make thing
worse...
{quote}
I agree, but this case is quite particular. I am not suggesting we schedule
recovery for any inconsistent state of meta, I just meant to say that if meta
is already online as per AssignmentManager but the server it is online on is
not even live, we already have a problem that we will likely not recover unless
that dead server SCP is being processed. The only way out for this case is for
operator to schedule recovery of the old server, the more it takes for operator
to understand what the current state of the cluster is, higher are the chances
of client requests failures in that duration and higher num of stuck procedures
will be accumulated.
If meta state is not online, we don't need any change in the current logic.
{quote}So here, meta is already online on server3-1,61020,1699456864765, but
after server1 becomes active, the loaded meta location is
server3-1,61020,1698687384632, which is a dead server?
{quote}
Correct.
{quote}And this happens on a rolling upgrading from 2.4 to 2.5? What is the
version for server1 and server4? Server4 is 2.4.x and server and server1 is
2.5.x?
{quote}
Yes, so far we observed this only during 2.4 to 2.5 upgrade. Let me get back
with the version details of masters (server4 and server1) in sometime.
> Master should recover if meta region state is inconsistent
> --
>
> Key: HBASE-28192
> URL: https://issues.apache.org/jira/browse/HBASE-28192
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.4.17, 2.5.6
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> During active master initialization, before we set master as active (i.e.
> {_}setInitialized(true){_}), we need both meta and namespace regions online.
> If the region state of meta or namespace is inconsistent, active master can
> get stuck in the initialization step:
> {code:java}
> private boolean isRegionOnline(RegionInfo ri) {
> RetryCounter rc = null;
> while (!isStopped()) {
> ...
> ...
> ...
> // Check once-a-minute.
> if (rc == null) {
> rc = new RetryCounterFactory(Integer.MAX_VALUE, 1000, 60_000).create();
> }
> Threads.sleep(rc.getBackoffTimeAndIncrementAttempts());
> }
> return false;
> }
> {code}
> In one of the recent outage, we observed that meta was online on a server,
> which was correctly reflected in meta znode, but the server starttime was
> different. This means that as per the latest transition record, meta was
> marked online on old server (same server with old start time). This kept
> active master initialization waiting forever and some SCPs got stuck in
> initial stage where they need to access meta table before getting candidate
> for region moves.
> The only way out of this outage is for operator to schedule recoveries using
> hbck for old server, which triggers SCP for old server address of meta. Since
> many SCPs were stuck, the processing of new SCP too was taking some time and
> manual restart of active master triggered failover, and new master was able
> to complete SCP for old meta server, correcting the meta assignment details,
> which eventually marked master as active and only after this, we were able to
> see real large num of RITs that were hidden so far.
> We need to let master recover from this state to avoid manual intervention.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-28189) Fix the miss count in one of CombinedBlockCache getBlock implementations
[
https://issues.apache.org/jira/browse/HBASE-28189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17784679#comment-17784679
]
Hudson commented on HBASE-28189:
Results for branch branch-2.4
[build #649 on
builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/649/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/649/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/649/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/649/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/649/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Fix the miss count in one of CombinedBlockCache getBlock implementations
>
>
> Key: HBASE-28189
> URL: https://issues.apache.org/jira/browse/HBASE-28189
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6, 4.0.0-alpha-1
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 4.0.0-alpha-1, 2.5.7
>
>
> In one of the
> CombinedBlockCache.getBlock(getBlock(cacheKey,cachingrepeat,updateCacheMetrics)
> we always compute a miss in L1 if the passed block is of type DATA. We
> should compute the miss in one of the caches only, not both.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
Apache-HBase commented on PR #5513: URL: https://github.com/apache/hbase/pull/5513#issuecomment-1805111075 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 25s | Docker mode activated. | | -0 :warning: | yetus | 0m 2s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 8s | master passed | | +1 :green_heart: | compile | 0m 46s | master passed | | +1 :green_heart: | shadedjars | 5m 29s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 52s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed | | +1 :green_heart: | javac | 0m 46s | the patch passed | | +1 :green_heart: | shadedjars | 5m 29s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 236m 15s | hbase-server in the patch failed. | | | | 260m 3s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5513 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 04f2cef806db 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/2/testReport/ | | Max. process+thread count | 4748 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28192 Master should recover if meta region state is inconsistent [hbase]
Apache-HBase commented on PR #5513: URL: https://github.com/apache/hbase/pull/5513#issuecomment-1805124676 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 12s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 32s | master passed | | +1 :green_heart: | compile | 0m 41s | master passed | | +1 :green_heart: | shadedjars | 4m 51s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 20s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | shadedjars | 4m 52s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 244m 59s | hbase-server in the patch failed. | | | | 266m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5513 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux c9935834d86f 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 7f3921ae40 | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/2/testReport/ | | Max. process+thread count | 4661 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5513/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-28194) New Splittable Meta
ruanhui created HBASE-28194: --- Summary: New Splittable Meta Key: HBASE-28194 URL: https://issues.apache.org/jira/browse/HBASE-28194 Project: HBase Issue Type: New Feature Components: meta, Region Assignment Reporter: ruanhui This issue is used to try to land to solution on splittable meta. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28195) set start row as prefix if a scan with PrefixFilter
ruanhui created HBASE-28195: --- Summary: set start row as prefix if a scan with PrefixFilter Key: HBASE-28195 URL: https://issues.apache.org/jira/browse/HBASE-28195 Project: HBase Issue Type: Improvement Components: Filters Affects Versions: 3.0.0-alpha-4 Reporter: ruanhui Assignee: ruanhui Fix For: 3.0.0-beta-1 If a scan with PrefixFilter, we can set start row as the prefix. This will help reduce filtered data. -- This message was sent by Atlassian Jira (v8.20.10#820010)
