[jira] [Updated] (HBASE-28151) hbck -o should not allow bypassing pre transit check by default
[ https://issues.apache.org/jira/browse/HBASE-28151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-28151: - Description: When operator uses hbck assigns or unassigns with "-o", the override will also skip pre transit checks. While this is one of the intentions with "-o", the primary purpose should still be to only unattach existing procedure from RegionStateNode so that newly scheduled assign proc can take exclusive region level lock. We should restrict bypassing preTransitCheck by only providing it as site config. If bypassing preTransitCheck is configured, only then any hbck "-o" should be allowed to bypass this check, otherwise by default they should go through the check. It is important to keep "unset of the procedure from RegionStateNode" and "bypassing preTransitCheck" separate so that when the cluster state is bad, we don't explicitly deteriorate it further e.g. if a region was successfully split and now if operator performs "hbck assigns \{region} -o" and if it bypasses the transit check, master would bring the region online and it could compact store files and archive the store file which is referenced by daughter region. This would not allow daughter region to come online. Let's introduce hbase site config to allow bypassing preTransitCheck, it should not be doable only by operator using hbck alone. "-o" should mean "override" the procedure that is attached to the RegionStateNode, it should not mean forcefully skip any region transition validation checks. was: When operator uses hbck assigns or unassigns with "-o", the override will also skip pre transit checks. While this is one of the intentions with "-o", the primary purpose should still be to only unattach existing procedure from RegionStateNode so that newly scheduled assign proc can take exclusive region level lock. We should restrict bypassing preTransitCheck by only providing it as site config. If bypassing preTransitCheck is configured, only then any hbck "-o" should be allowed to bypass this check, otherwise by default they should go through the check. It is important to keep "unset of the procedure from RegionStateNode" and "bypassing preTransitCheck" separate so that when the cluster state is bad, we don't explicitly deteriorate it further e.g. if a region was successfully split and now if operator performs "hbck assigns \{region} -o" and if it bypasses the transit check, master would bring the region online and it could compact store files and archive the store file which is referenced by daughter region. This would not allow daughter region to come online. Let's introduce hbase site config to allow bypassing preTransitCheck, it should not be doable only by operator using hbck alone. "-o" should mean "override" the procedure that is attached to the RegionStateNode, it should not mean forcefully skip any region transition validation checks and perform the region assignments. > hbck -o should not allow bypassing pre transit check by default > --- > > Key: HBASE-28151 > URL: https://issues.apache.org/jira/browse/HBASE-28151 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.4.17, 2.5.5 >Reporter: Viraj Jasani >Priority: Major > > When operator uses hbck assigns or unassigns with "-o", the override will > also skip pre transit checks. While this is one of the intentions with "-o", > the primary purpose should still be to only unattach existing procedure from > RegionStateNode so that newly scheduled assign proc can take exclusive region > level lock. > We should restrict bypassing preTransitCheck by only providing it as site > config. > If bypassing preTransitCheck is configured, only then any hbck "-o" should be > allowed to bypass this check, otherwise by default they should go through the > check. > > It is important to keep "unset of the procedure from RegionStateNode" and > "bypassing preTransitCheck" separate so that when the cluster state is bad, > we don't explicitly deteriorate it further e.g. if a region was successfully > split and now if operator performs "hbck assigns \{region} -o" and if it > bypasses the transit check, master would bring the region online and it could > compact store files and archive the store file which is referenced by > daughter region. This would not allow daughter region to come online. > Let's introduce hbase site config to allow bypassing preTransitCheck, it > should not be doable only by operator using hbck alone. > > "-o" should mean "override" the procedure that is attached to the > RegionStateNode, it should not mean forcefully skip any region transition > validation checks. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28151) hbck -o should not allow bypassing pre transit check by default
[ https://issues.apache.org/jira/browse/HBASE-28151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-28151: - Description: When operator uses hbck assigns or unassigns with "-o", the override will also skip pre transit checks. While this is one of the intentions with "-o", the primary purpose should still be to only unattach existing procedure from RegionStateNode so that newly scheduled assign proc can take exclusive region level lock. We should restrict bypassing preTransitCheck by only providing it as site config. If bypassing preTransitCheck is configured, only then any hbck "-o" should be allowed to bypass this check, otherwise by default they should go through the check. It is important to keep "unset of the procedure from RegionStateNode" and "bypassing preTransitCheck" separate so that when the cluster state is bad, we don't explicitly deteriorate it further e.g. if a region was successfully split and now if operator performs "hbck assigns \{region} -o" and if it bypasses the transit check, master would bring the region online and it could compact store files and archive the store file which is referenced by daughter region. This would not allow daughter region to come online. Let's introduce hbase site config to allow bypassing preTransitCheck, it should not be doable only by operator using hbck alone. "-o" should mean "override" the procedure that is attached to the RegionStateNode, it should not mean forcefully skip any region transition validation checks and perform the region assignments. was: When operator uses hbck assigns or unassigns with "-o", the override will also skip pre transit checks. While this is one of the intentions with "-o", the primary purpose should still be to only unattach existing procedure from RegionStateNode so that newly scheduled assign proc can take exclusive region level lock. We should restrict bypassing preTransitCheck by only providing it as site config. If bypassing preTransitCheck is configured, only then any hbck "-o" should be allowed to bypass this check, otherwise by default they should go through the check. It is important to keep "unset of the procedure from RegionStateNode" and "bypassing preTransitCheck" separate so that when the cluster state is bad, we don't explicitly deteriorate it further e.g. if a region was successfully split and now if operator performs "hbck assigns \{region} -o" and if it bypasses the transit check, master would bring the region online and it could compact store files and archive the store file which is referenced by daughter region. This would not allow daughter region to come online. Let's introduce hbase site config to allow bypassing preTransitCheck, it should not be doable only by operator using hbck alone. > hbck -o should not allow bypassing pre transit check by default > --- > > Key: HBASE-28151 > URL: https://issues.apache.org/jira/browse/HBASE-28151 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.4.17, 2.5.5 >Reporter: Viraj Jasani >Priority: Major > > When operator uses hbck assigns or unassigns with "-o", the override will > also skip pre transit checks. While this is one of the intentions with "-o", > the primary purpose should still be to only unattach existing procedure from > RegionStateNode so that newly scheduled assign proc can take exclusive region > level lock. > We should restrict bypassing preTransitCheck by only providing it as site > config. > If bypassing preTransitCheck is configured, only then any hbck "-o" should be > allowed to bypass this check, otherwise by default they should go through the > check. > > It is important to keep "unset of the procedure from RegionStateNode" and > "bypassing preTransitCheck" separate so that when the cluster state is bad, > we don't explicitly deteriorate it further e.g. if a region was successfully > split and now if operator performs "hbck assigns \{region} -o" and if it > bypasses the transit check, master would bring the region online and it could > compact store files and archive the store file which is referenced by > daughter region. This would not allow daughter region to come online. > Let's introduce hbase site config to allow bypassing preTransitCheck, it > should not be doable only by operator using hbck alone. > > "-o" should mean "override" the procedure that is attached to the > RegionStateNode, it should not mean forcefully skip any region transition > validation checks and perform the region assignments. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[ https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774786#comment-17774786 ] Hudson commented on HBASE-28147: Results for branch branch-2 [build #903 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/General_20Nightly_20Build_20Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests > > > Key: HBASE-28147 > URL: https://issues.apache.org/jira/browse/HBASE-28147 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28135) Specify -Xms for tests
[ https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774787#comment-17774787 ] Hudson commented on HBASE-28135: Results for branch branch-2 [build #903 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/General_20Nightly_20Build_20Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2/903/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Specify -Xms for tests > -- > > Key: HBASE-28135 > URL: https://issues.apache.org/jira/browse/HBASE-28135 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1 > > > The default -Xms value is JVM dependent, but the host memory size is usually > included in the calculation. > -Xms in turn is used to calculate some GC parameters, for example NewSize and > OldSize, which affect the behaviour of tests. > As the memory consumption on the tests is not dependent on the host VM size, > we could set -Xms for the tests explictly, and enjoy more consistent test > results. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[ https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774780#comment-17774780 ] Hudson commented on HBASE-28147: Results for branch branch-2.4 [build #635 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/General_20Nightly_20Build_20Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests > > > Key: HBASE-28147 > URL: https://issues.apache.org/jira/browse/HBASE-28147 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28135) Specify -Xms for tests
[ https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774781#comment-17774781 ] Hudson commented on HBASE-28135: Results for branch branch-2.4 [build #635 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/General_20Nightly_20Build_20Report/] (x) {color:red}-1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.4/635/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Specify -Xms for tests > -- > > Key: HBASE-28135 > URL: https://issues.apache.org/jira/browse/HBASE-28135 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1 > > > The default -Xms value is JVM dependent, but the host memory size is usually > included in the calculation. > -Xms in turn is used to calculate some GC parameters, for example NewSize and > OldSize, which affect the behaviour of tests. > As the memory consumption on the tests is not dependent on the host VM size, > we could set -Xms for the tests explictly, and enjoy more consistent test > results. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28151) hbck -o should not allow bypassing pre transit check by default
Viraj Jasani created HBASE-28151: Summary: hbck -o should not allow bypassing pre transit check by default Key: HBASE-28151 URL: https://issues.apache.org/jira/browse/HBASE-28151 Project: HBase Issue Type: Improvement Reporter: Viraj Jasani When operator uses hbck assigns or unassigns with "-o", the override will also skip pre transit checks. While this is one of the intentions with "-o", the primary purpose should still be to only unattach existing procedure from RegionStateNode so that newly scheduled assign proc can take exclusive region level lock. We should restrict bypassing preTransitCheck by only providing it as site config. If bypassing preTransitCheck is configured, only then any hbck "-o" should be allowed to bypass this check, otherwise by default they should go through the check. It is important to keep "unset of the procedure from RegionStateNode" and "bypassing preTransitCheck" separate so that when the cluster state is bad, we don't explicitly deteriorate it further e.g. if a region was successfully split and now if operator performs "hbck assigns \{region} -o" and if it bypasses the transit check, master would bring the region online and it could compact store files and archive the store file which is referenced by daughter region. This would not allow daughter region to come online. Let's introduce hbase site config to allow bypassing preTransitCheck, it should not be doable only by operator using hbck alone. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28151) hbck -o should not allow bypassing pre transit check by default
[ https://issues.apache.org/jira/browse/HBASE-28151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HBASE-28151: - Affects Version/s: 2.5.5 2.4.17 > hbck -o should not allow bypassing pre transit check by default > --- > > Key: HBASE-28151 > URL: https://issues.apache.org/jira/browse/HBASE-28151 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.4.17, 2.5.5 >Reporter: Viraj Jasani >Priority: Major > > When operator uses hbck assigns or unassigns with "-o", the override will > also skip pre transit checks. While this is one of the intentions with "-o", > the primary purpose should still be to only unattach existing procedure from > RegionStateNode so that newly scheduled assign proc can take exclusive region > level lock. > We should restrict bypassing preTransitCheck by only providing it as site > config. > If bypassing preTransitCheck is configured, only then any hbck "-o" should be > allowed to bypass this check, otherwise by default they should go through the > check. > > It is important to keep "unset of the procedure from RegionStateNode" and > "bypassing preTransitCheck" separate so that when the cluster state is bad, > we don't explicitly deteriorate it further e.g. if a region was successfully > split and now if operator performs "hbck assigns \{region} -o" and if it > bypasses the transit check, master would bring the region online and it could > compact store files and archive the store file which is referenced by > daughter region. This would not allow daughter region to come online. > Let's introduce hbase site config to allow bypassing preTransitCheck, it > should not be doable only by operator using hbck alone. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-25222) Add a cost function to move the daughter regions of a recent split to different region servers
[ https://issues.apache.org/jira/browse/HBASE-25222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774707#comment-17774707 ] David Manning commented on HBASE-25222: --- Seems like in hbase 2.x, this is already less of an issue, because the SplitTableRegionProcedure will choose new servers to open the daughter regions. In hbase 1.x, the daughter regions would open on the same server as the parent region. > Add a cost function to move the daughter regions of a recent split to > different region servers > --- > > Key: HBASE-25222 > URL: https://issues.apache.org/jira/browse/HBASE-25222 > Project: HBase > Issue Type: Improvement >Reporter: Sandeep Pal >Assignee: Sandeep Pal >Priority: Major > > In HBase, hotspot regions are easily formed whenever there is skew and there > is high write volume. Few regions grow really fast which also becomes the > bottleneck on the few region servers. > It would be beneficial to add a cost function to move the regions after the > split to differetn region servers. In this way the writes to hot key range > will be distributed to multiple region servers. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1760443461 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 40s | branch-2 passed | | +1 :green_heart: | compile | 0m 43s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 15s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 30s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | shadedjars | 5m 16s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 227m 25s | hbase-server in the patch failed. | | | | 250m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b9f01b6807d3 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/testReport/ | | Max. process+thread count | 4800 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1760409599 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 11s | branch-2 passed | | +1 :green_heart: | compile | 0m 40s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 31s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 11s | the patch passed | | +1 :green_heart: | compile | 0m 40s | the patch passed | | +1 :green_heart: | javac | 0m 40s | the patch passed | | +1 :green_heart: | shadedjars | 4m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 208m 20s | hbase-server in the patch passed. | | | | 228m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux b63454fe1b7a 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/testReport/ | | Max. process+thread count | 4355 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28133) TestSyncTimeRangeTracker fails with OOM with small -Xms values
[ https://issues.apache.org/jira/browse/HBASE-28133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774680#comment-17774680 ] Hudson commented on HBASE-28133: Results for branch branch-2.5 [build #418 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/]: (/) *{color:green}+1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > TestSyncTimeRangeTracker fails with OOM with small -Xms values > -- > > Key: HBASE-28133 > URL: https://issues.apache.org/jira/browse/HBASE-28133 > Project: HBase > Issue Type: Bug >Affects Versions: 2.4.17 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: Arm64, test > Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1 > > > Edit2: It's not the OS, it's the -Xms value determined from the host memory > size. > Edit: It's related to the OS and it's default java 8 , not to the processor > architecture. > This test seems to be cutting real close to the heap size. > On ARM, it consistently fails on my RHEL8.8 Aarch64 VM with Java 8. > {noformat} > mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G > -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server > ... > [ERROR] > org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker.testConcurrentIncludeTimestampCorrectness > Time elapsed: 1.969 s <<< ERROR! > java.lang.OutOfMemoryError: Java heap space > {noformat} > It seems that Java on ARM has some higher memory overhead than x86_64. > Simply bumping -Xmx from the default 2200m to 2300m allows it to pass. > {noformat} > mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G > -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server > -Dsurefire.Xmx=2300m > ... > [INFO] Running org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker > [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.395 > s - in org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker > {noformat} > However, the real solution should be reducing the memory usage for this test. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[ https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774681#comment-17774681 ] Hudson commented on HBASE-28147: Results for branch branch-2.5 [build #418 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/]: (/) *{color:green}+1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests > > > Key: HBASE-28147 > URL: https://issues.apache.org/jira/browse/HBASE-28147 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28135) Specify -Xms for tests
[ https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774682#comment-17774682 ] Hudson commented on HBASE-28135: Results for branch branch-2.5 [build #418 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/]: (/) *{color:green}+1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop2 checks{color} -- For more information [see jdk8 (hadoop2) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-2.5/418/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Specify -Xms for tests > -- > > Key: HBASE-28135 > URL: https://issues.apache.org/jira/browse/HBASE-28135 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1 > > > The default -Xms value is JVM dependent, but the host memory size is usually > included in the calculation. > -Xms in turn is used to calculate some GC parameters, for example NewSize and > OldSize, which affect the behaviour of tests. > As the memory consumption on the tests is not dependent on the host VM size, > we could set -Xms for the tests explictly, and enjoy more consistent test > results. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28133) TestSyncTimeRangeTracker fails with OOM with small -Xms values
[ https://issues.apache.org/jira/browse/HBASE-28133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774676#comment-17774676 ] Hudson commented on HBASE-28133: Results for branch branch-3 [build #65 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/]: (/) *{color:green}+1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > TestSyncTimeRangeTracker fails with OOM with small -Xms values > -- > > Key: HBASE-28133 > URL: https://issues.apache.org/jira/browse/HBASE-28133 > Project: HBase > Issue Type: Bug >Affects Versions: 2.4.17 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: Arm64, test > Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1 > > > Edit2: It's not the OS, it's the -Xms value determined from the host memory > size. > Edit: It's related to the OS and it's default java 8 , not to the processor > architecture. > This test seems to be cutting real close to the heap size. > On ARM, it consistently fails on my RHEL8.8 Aarch64 VM with Java 8. > {noformat} > mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G > -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server > ... > [ERROR] > org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker.testConcurrentIncludeTimestampCorrectness > Time elapsed: 1.969 s <<< ERROR! > java.lang.OutOfMemoryError: Java heap space > {noformat} > It seems that Java on ARM has some higher memory overhead than x86_64. > Simply bumping -Xmx from the default 2200m to 2300m allows it to pass. > {noformat} > mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G > -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server > -Dsurefire.Xmx=2300m > ... > [INFO] Running org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker > [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.395 > s - in org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker > {noformat} > However, the real solution should be reducing the memory usage for this test. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[ https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774677#comment-17774677 ] Hudson commented on HBASE-28147: Results for branch branch-3 [build #65 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/]: (/) *{color:green}+1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests > > > Key: HBASE-28147 > URL: https://issues.apache.org/jira/browse/HBASE-28147 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28135) Specify -Xms for tests
[ https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774678#comment-17774678 ] Hudson commented on HBASE-28135: Results for branch branch-3 [build #65 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/]: (/) *{color:green}+1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/General_20Nightly_20Build_20Report/] (/) {color:green}+1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/branch-3/65/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Specify -Xms for tests > -- > > Key: HBASE-28135 > URL: https://issues.apache.org/jira/browse/HBASE-28135 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1 > > > The default -Xms value is JVM dependent, but the host memory size is usually > included in the calculation. > -Xms in turn is used to calculate some GC parameters, for example NewSize and > OldSize, which affect the behaviour of tests. > As the memory consumption on the tests is not dependent on the host VM size, > we could set -Xms for the tests explictly, and enjoy more consistent test > results. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[ https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774651#comment-17774651 ] Hudson commented on HBASE-28147: Results for branch master [build #925 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/General_20Nightly_20Build_20Report/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests > > > Key: HBASE-28147 > URL: https://issues.apache.org/jira/browse/HBASE-28147 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28148) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit
[ https://issues.apache.org/jira/browse/HBASE-28148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774652#comment-17774652 ] Hudson commented on HBASE-28148: Results for branch master [build #925 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/General_20Nightly_20Build_20Report/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit > --- > > Key: HBASE-28148 > URL: https://issues.apache.org/jira/browse/HBASE-28148 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28133) TestSyncTimeRangeTracker fails with OOM with small -Xms values
[ https://issues.apache.org/jira/browse/HBASE-28133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774650#comment-17774650 ] Hudson commented on HBASE-28133: Results for branch master [build #925 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/General_20Nightly_20Build_20Report/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > TestSyncTimeRangeTracker fails with OOM with small -Xms values > -- > > Key: HBASE-28133 > URL: https://issues.apache.org/jira/browse/HBASE-28133 > Project: HBase > Issue Type: Bug >Affects Versions: 2.4.17 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: Arm64, test > Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1 > > > Edit2: It's not the OS, it's the -Xms value determined from the host memory > size. > Edit: It's related to the OS and it's default java 8 , not to the processor > architecture. > This test seems to be cutting real close to the heap size. > On ARM, it consistently fails on my RHEL8.8 Aarch64 VM with Java 8. > {noformat} > mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G > -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server > ... > [ERROR] > org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker.testConcurrentIncludeTimestampCorrectness > Time elapsed: 1.969 s <<< ERROR! > java.lang.OutOfMemoryError: Java heap space > {noformat} > It seems that Java on ARM has some higher memory overhead than x86_64. > Simply bumping -Xmx from the default 2200m to 2300m allows it to pass. > {noformat} > mvn test -P runDevTests -Dtest.build.data.basedirectory=/ram2G > -Dhadoop.profile=3.0 -fn -B -Dtest=TestSyncTimeRangeTracker* -pl hbase-server > -Dsurefire.Xmx=2300m > ... > [INFO] Running org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker > [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.395 > s - in org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker > {noformat} > However, the real solution should be reducing the memory usage for this test. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-28135) Specify -Xms for tests
[ https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774653#comment-17774653 ] Hudson commented on HBASE-28135: Results for branch master [build #925 on builds.a.o|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/]: (x) *{color:red}-1 overall{color}* details (if available): (/) {color:green}+1 general checks{color} -- For more information [see general report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/General_20Nightly_20Build_20Report/] (x) {color:red}-1 jdk8 hadoop3 checks{color} -- For more information [see jdk8 (hadoop3) report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 jdk11 hadoop3 checks{color} -- For more information [see jdk11 report|https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/925/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/] (/) {color:green}+1 source release artifact{color} -- See build output for details. (/) {color:green}+1 client integration test{color} > Specify -Xms for tests > -- > > Key: HBASE-28135 > URL: https://issues.apache.org/jira/browse/HBASE-28135 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1 > > > The default -Xms value is JVM dependent, but the host memory size is usually > included in the calculation. > -Xms in turn is used to calculate some GC parameters, for example NewSize and > OldSize, which affect the behaviour of tests. > As the memory consumption on the tests is not dependent on the host VM size, > we could set -Xms for the tests explictly, and enjoy more consistent test > results. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1760163936 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 38s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 46s | branch-2 passed | | +1 :green_heart: | compile | 2m 24s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 38s | branch-2 passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 30s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 33s | the patch passed | | +1 :green_heart: | compile | 2m 24s | the patch passed | | +1 :green_heart: | javac | 2m 24s | the patch passed | | +1 :green_heart: | checkstyle | 0m 33s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 20s | Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 43s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 13s | The patch does not generate ASF License warnings. | | | | 31m 49s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux c7e216628d56 5.4.0-163-generic #180-Ubuntu SMP Tue Sep 5 13:21:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 78 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/3/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1760103112 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 15s | Docker mode activated. | | -0 :warning: | yetus | 0m 4s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 21s | branch-2 passed | | +1 :green_heart: | compile | 0m 40s | branch-2 passed | | +1 :green_heart: | shadedjars | 4m 36s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 25s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 14s | the patch passed | | +1 :green_heart: | compile | 0m 40s | the patch passed | | +1 :green_heart: | javac | 0m 40s | the patch passed | | +1 :green_heart: | shadedjars | 4m 30s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 24s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 207m 49s | hbase-server in the patch passed. | | | | 228m 34s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 493f497d3c0f 5.4.0-153-generic #170-Ubuntu SMP Fri Jun 16 13:43:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Temurin-1.8.0_352-b08 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/testReport/ | | Max. process+thread count | 4540 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1760098522 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 42s | Docker mode activated. | | -0 :warning: | yetus | 0m 5s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 47s | branch-2 passed | | +1 :green_heart: | compile | 0m 43s | branch-2 passed | | +1 :green_heart: | shadedjars | 5m 19s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 32s | the patch passed | | +1 :green_heart: | compile | 0m 43s | the patch passed | | +1 :green_heart: | javac | 0m 43s | the patch passed | | +1 :green_heart: | shadedjars | 5m 14s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 23s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 203m 0s | hbase-server in the patch passed. | | | | 225m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux fa5a8c149301 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/testReport/ | | Max. process+thread count | 4656 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-26974 Introduce a LogRollProcedure [hbase]
frostruan commented on code in PR #5408: URL: https://github.com/apache/hbase/pull/5408#discussion_r1357033196 ## hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/util/BackupUtils.java: ## @@ -762,4 +773,30 @@ public static String findMostRecentBackupId(String[] backupIds) { return BackupRestoreConstants.BACKUPID_PREFIX + recentTimestamp; } + public static void rollWALWriters(Admin admin, Map props) throws IOException { +byte[] ret = Review Comment: No, this is the last one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-26974 Introduce a LogRollProcedure [hbase]
frostruan commented on code in PR #5408: URL: https://github.com/apache/hbase/pull/5408#discussion_r1357032646 ## hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/LogRollProcedure.java: ## @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.backup.master; + +import static org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_DEFAULT; +import static org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_KEY; +import static org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED; +import static org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED_DEFAULT; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Optional; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.NamespaceDescriptor; +import org.apache.hadoop.hbase.ServerName; +import org.apache.hadoop.hbase.TableName; +import org.apache.hadoop.hbase.TableNotEnabledException; +import org.apache.hadoop.hbase.backup.impl.BackupSystemTable; +import org.apache.hadoop.hbase.client.RegionInfo; +import org.apache.hadoop.hbase.client.TableDescriptor; +import org.apache.hadoop.hbase.client.TableState; +import org.apache.hadoop.hbase.master.TableStateManager; +import org.apache.hadoop.hbase.master.procedure.CreateNamespaceProcedure; +import org.apache.hadoop.hbase.master.procedure.CreateTableProcedure; +import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv; +import org.apache.hadoop.hbase.master.procedure.TableProcedureInterface; +import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer; +import org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException; +import org.apache.hadoop.hbase.procedure2.ProcedureYieldException; +import org.apache.hadoop.hbase.procedure2.StateMachineProcedure; +import org.apache.hadoop.hbase.util.EnvironmentEdgeManager; +import org.apache.hadoop.hbase.util.ModifyRegionUtils; +import org.apache.yetus.audience.InterfaceAudience; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollData; +import org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollState; + +@InterfaceAudience.Private +public class LogRollProcedure extends StateMachineProcedure + implements TableProcedureInterface { + + private static final Logger LOG = LoggerFactory.getLogger(RSLogRollProcedure.class); + + private Configuration conf; + + private String backupRoot; + + public LogRollProcedure() { + } + + public LogRollProcedure(final String backupRoot, final Configuration conf) { +this.backupRoot = backupRoot; +this.conf = conf; + } + + @Override + protected Flow executeFromState(MasterProcedureEnv env, LogRollState state) +throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException { +LOG.info("{} execute state={}", this, state); + +try { + switch (state) { +case LOG_ROLL_PRE_CHECK_NAMESPACE: + // create backup namespace if not exists + if ( + !env.getMasterServices().getClusterSchema().getTableNamespaceManager() + .doesNamespaceExist(getTableName().getNamespaceAsString()) + ) { +NamespaceDescriptor desc = + NamespaceDescriptor.create(getTableName().getNamespaceAsString()).build(); +addChildProcedure(new CreateNamespaceProcedure(env, desc)); + } + setNextState(LogRollState.LOG_ROLL_PRE_CHECK_TABLES); + return Flow.HAS_MORE_STATE; +case LOG_ROLL_PRE_CHECK_TABLES: + // create backup system table and backup system bulkload table if not exists + List toCreate = new ArrayList<>(); + TableStateManager tsm = env.getMasterServices().getTableStateManager(); + createIfNeeded(env, tsm, BackupSystemTable.getTableName(conf), + BackupSystemTable.getSystemTableDescriptor(conf)).ifPresent(toCreate::add); + createIfNeeded(env, tsm, BackupSystemTable.getTableNameFo
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
bbeaudreault commented on code in PR #5461: URL: https://github.com/apache/hbase/pull/5461#discussion_r1356965047 ## hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java: ## @@ -718,14 +720,19 @@ public static void closeRegionSilentlyAndWait(ClusterConnection connection, Serv public AdminService.BlockingInterface getRsAdmin(final ServerName sn) throws IOException { AdminService.BlockingInterface admin = this.rsAdmins.get(sn); if (admin == null) { - LOG.debug("New admin connection to " + sn.toString()); - if (sn.equals(master.getServerName()) && master instanceof HRegionServer) { -// A master is also a region server now, see HBASE-10569 for details -admin = ((HRegionServer) master).getRSRpcServices(); - } else { -admin = this.connection.getAdmin(sn); - } - this.rsAdmins.put(sn, admin); + return this.rsAdmins.computeIfAbsent(sn, server -> { +LOG.debug("New admin connection to " + server.toString()); +if (server.equals(master.getServerName()) && master instanceof HRegionServer) { + // A master is also a region server now, see HBASE-10569 for details + return ((HRegionServer) master).getRSRpcServices(); +} else { + try { +return this.connection.getAdmin(server); + } catch (IOException e) { +throw new RuntimeException(e); Review Comment: As written, it seems we need _some_ synchronization. A concurrent map is pointless if you do unsyncrhonized writes on it. You could use ConcurrentMapUtils.computeIfAbsentEx to get around the exception issue. That said, looking at the impls of getRSRpcServices and getAdmin, I don't think we really need to cache these at all. getRSRpcServices directly returns an instance variable. getAdmin already does it's own caching. So it might make sense to simply remove the rsAdmins map. @Apache9 what do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache-HBase commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1759751872 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 49s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-2 Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 55s | branch-2 passed | | +1 :green_heart: | compile | 2m 26s | branch-2 passed | | +1 :green_heart: | checkstyle | 0m 34s | branch-2 passed | | +1 :green_heart: | spotless | 0m 44s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 27s | branch-2 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 39s | the patch passed | | +1 :green_heart: | compile | 2m 26s | the patch passed | | +1 :green_heart: | javac | 2m 26s | the patch passed | | +1 :green_heart: | checkstyle | 0m 34s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 13m 58s | Patch does not cause any errors with Hadoop 2.10.2 or 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 40s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 31s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 32m 45s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5461 | | JIRA Issue | HBASE-28146 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 416b440107d5 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | branch-2 / 212c7cb827 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5461/2/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
rmdmattingly commented on code in PR #5461: URL: https://github.com/apache/hbase/pull/5461#discussion_r1356892819 ## hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java: ## @@ -718,14 +720,19 @@ public static void closeRegionSilentlyAndWait(ClusterConnection connection, Serv public AdminService.BlockingInterface getRsAdmin(final ServerName sn) throws IOException { AdminService.BlockingInterface admin = this.rsAdmins.get(sn); if (admin == null) { - LOG.debug("New admin connection to " + sn.toString()); - if (sn.equals(master.getServerName()) && master instanceof HRegionServer) { -// A master is also a region server now, see HBASE-10569 for details -admin = ((HRegionServer) master).getRSRpcServices(); - } else { -admin = this.connection.getAdmin(sn); - } - this.rsAdmins.put(sn, admin); + return this.rsAdmins.computeIfAbsent(sn, server -> { +LOG.debug("New admin connection to " + server.toString()); +if (server.equals(master.getServerName()) && master instanceof HRegionServer) { + // A master is also a region server now, see HBASE-10569 for details + return ((HRegionServer) master).getRSRpcServices(); +} else { + try { +return this.connection.getAdmin(server); + } catch (IOException e) { +throw new RuntimeException(e); Review Comment: Yeah, good point -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache9 commented on code in PR #5461: URL: https://github.com/apache/hbase/pull/5461#discussion_r1356884191 ## hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java: ## @@ -718,14 +720,19 @@ public static void closeRegionSilentlyAndWait(ClusterConnection connection, Serv public AdminService.BlockingInterface getRsAdmin(final ServerName sn) throws IOException { AdminService.BlockingInterface admin = this.rsAdmins.get(sn); if (admin == null) { - LOG.debug("New admin connection to " + sn.toString()); - if (sn.equals(master.getServerName()) && master instanceof HRegionServer) { -// A master is also a region server now, see HBASE-10569 for details -admin = ((HRegionServer) master).getRSRpcServices(); - } else { -admin = this.connection.getAdmin(sn); - } - this.rsAdmins.put(sn, admin); + return this.rsAdmins.computeIfAbsent(sn, server -> { +LOG.debug("New admin connection to " + server.toString()); +if (server.equals(master.getServerName()) && master instanceof HRegionServer) { + // A master is also a region server now, see HBASE-10569 for details + return ((HRegionServer) master).getRSRpcServices(); +} else { + try { +return this.connection.getAdmin(server); + } catch (IOException e) { +throw new RuntimeException(e); Review Comment: We should throw this IOException out? This is a problem of computeIfAbsent, as the functional interface does not throw IOException... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-26974 Introduce a LogRollProcedure [hbase]
Apache9 commented on code in PR #5408: URL: https://github.com/apache/hbase/pull/5408#discussion_r1356848506 ## hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/util/BackupUtils.java: ## @@ -762,4 +773,30 @@ public static String findMostRecentBackupId(String[] backupIds) { return BackupRestoreConstants.BACKUPID_PREFIX + recentTimestamp; } + public static void rollWALWriters(Admin admin, Map props) throws IOException { +byte[] ret = Review Comment: The execProcedure call is for zk based procedures, do we still have other procedures besides the log roll one? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-26974 Introduce a LogRollProcedure [hbase]
Apache9 commented on code in PR #5408: URL: https://github.com/apache/hbase/pull/5408#discussion_r1356843363 ## hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/LogRollProcedure.java: ## @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.backup.master; + +import static org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_DEFAULT; +import static org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_KEY; +import static org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED; +import static org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED_DEFAULT; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Optional; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.NamespaceDescriptor; +import org.apache.hadoop.hbase.ServerName; +import org.apache.hadoop.hbase.TableName; +import org.apache.hadoop.hbase.TableNotEnabledException; +import org.apache.hadoop.hbase.backup.impl.BackupSystemTable; +import org.apache.hadoop.hbase.client.RegionInfo; +import org.apache.hadoop.hbase.client.TableDescriptor; +import org.apache.hadoop.hbase.client.TableState; +import org.apache.hadoop.hbase.master.TableStateManager; +import org.apache.hadoop.hbase.master.procedure.CreateNamespaceProcedure; +import org.apache.hadoop.hbase.master.procedure.CreateTableProcedure; +import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv; +import org.apache.hadoop.hbase.master.procedure.TableProcedureInterface; +import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer; +import org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException; +import org.apache.hadoop.hbase.procedure2.ProcedureYieldException; +import org.apache.hadoop.hbase.procedure2.StateMachineProcedure; +import org.apache.hadoop.hbase.util.EnvironmentEdgeManager; +import org.apache.hadoop.hbase.util.ModifyRegionUtils; +import org.apache.yetus.audience.InterfaceAudience; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollData; +import org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollState; + +@InterfaceAudience.Private +public class LogRollProcedure extends StateMachineProcedure + implements TableProcedureInterface { + + private static final Logger LOG = LoggerFactory.getLogger(RSLogRollProcedure.class); + + private Configuration conf; + + private String backupRoot; + + public LogRollProcedure() { + } + + public LogRollProcedure(final String backupRoot, final Configuration conf) { +this.backupRoot = backupRoot; +this.conf = conf; + } + + @Override + protected Flow executeFromState(MasterProcedureEnv env, LogRollState state) +throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException { +LOG.info("{} execute state={}", this, state); + +try { + switch (state) { +case LOG_ROLL_PRE_CHECK_NAMESPACE: + // create backup namespace if not exists + if ( + !env.getMasterServices().getClusterSchema().getTableNamespaceManager() + .doesNamespaceExist(getTableName().getNamespaceAsString()) + ) { +NamespaceDescriptor desc = + NamespaceDescriptor.create(getTableName().getNamespaceAsString()).build(); +addChildProcedure(new CreateNamespaceProcedure(env, desc)); + } + setNextState(LogRollState.LOG_ROLL_PRE_CHECK_TABLES); + return Flow.HAS_MORE_STATE; +case LOG_ROLL_PRE_CHECK_TABLES: + // create backup system table and backup system bulkload table if not exists + List toCreate = new ArrayList<>(); + TableStateManager tsm = env.getMasterServices().getTableStateManager(); + createIfNeeded(env, tsm, BackupSystemTable.getTableName(conf), + BackupSystemTable.getSystemTableDescriptor(conf)).ifPresent(toCreate::add); + createIfNeeded(env, tsm, BackupSystemTable.getTableNameForB
Re: [PR] HBASE-26974 Introduce a LogRollProcedure [hbase]
Apache9 commented on code in PR #5408: URL: https://github.com/apache/hbase/pull/5408#discussion_r1356842065 ## hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/master/LogRollProcedure.java: ## @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.backup.master; + +import static org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_DEFAULT; +import static org.apache.hadoop.hbase.backup.BackupRestoreConstants.BACKUP_ENABLE_KEY; +import static org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED; +import static org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager.LOGROLL_PROCEDURE_ENABLED_DEFAULT; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Optional; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.NamespaceDescriptor; +import org.apache.hadoop.hbase.ServerName; +import org.apache.hadoop.hbase.TableName; +import org.apache.hadoop.hbase.TableNotEnabledException; +import org.apache.hadoop.hbase.backup.impl.BackupSystemTable; +import org.apache.hadoop.hbase.client.RegionInfo; +import org.apache.hadoop.hbase.client.TableDescriptor; +import org.apache.hadoop.hbase.client.TableState; +import org.apache.hadoop.hbase.master.TableStateManager; +import org.apache.hadoop.hbase.master.procedure.CreateNamespaceProcedure; +import org.apache.hadoop.hbase.master.procedure.CreateTableProcedure; +import org.apache.hadoop.hbase.master.procedure.MasterProcedureEnv; +import org.apache.hadoop.hbase.master.procedure.TableProcedureInterface; +import org.apache.hadoop.hbase.procedure2.ProcedureStateSerializer; +import org.apache.hadoop.hbase.procedure2.ProcedureSuspendedException; +import org.apache.hadoop.hbase.procedure2.ProcedureYieldException; +import org.apache.hadoop.hbase.procedure2.StateMachineProcedure; +import org.apache.hadoop.hbase.util.EnvironmentEdgeManager; +import org.apache.hadoop.hbase.util.ModifyRegionUtils; +import org.apache.yetus.audience.InterfaceAudience; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollData; +import org.apache.hadoop.hbase.shaded.protobuf.generated.BackupProtos.LogRollState; + +@InterfaceAudience.Private +public class LogRollProcedure extends StateMachineProcedure + implements TableProcedureInterface { + + private static final Logger LOG = LoggerFactory.getLogger(RSLogRollProcedure.class); + + private Configuration conf; + + private String backupRoot; + + public LogRollProcedure() { + } + + public LogRollProcedure(final String backupRoot, final Configuration conf) { +this.backupRoot = backupRoot; +this.conf = conf; + } + + @Override + protected Flow executeFromState(MasterProcedureEnv env, LogRollState state) +throws ProcedureSuspendedException, ProcedureYieldException, InterruptedException { +LOG.info("{} execute state={}", this, state); + +try { + switch (state) { +case LOG_ROLL_PRE_CHECK_NAMESPACE: + // create backup namespace if not exists + if ( + !env.getMasterServices().getClusterSchema().getTableNamespaceManager() + .doesNamespaceExist(getTableName().getNamespaceAsString()) + ) { +NamespaceDescriptor desc = + NamespaceDescriptor.create(getTableName().getNamespaceAsString()).build(); +addChildProcedure(new CreateNamespaceProcedure(env, desc)); + } + setNextState(LogRollState.LOG_ROLL_PRE_CHECK_TABLES); + return Flow.HAS_MORE_STATE; +case LOG_ROLL_PRE_CHECK_TABLES: + // create backup system table and backup system bulkload table if not exists + List toCreate = new ArrayList<>(); + TableStateManager tsm = env.getMasterServices().getTableStateManager(); + createIfNeeded(env, tsm, BackupSystemTable.getTableName(conf), + BackupSystemTable.getSystemTableDescriptor(conf)).ifPresent(toCreate::add); + createIfNeeded(env, tsm, BackupSystemTable.getTableNameForB
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache9 commented on code in PR #5461: URL: https://github.com/apache/hbase/pull/5461#discussion_r1356835441 ## hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java: ## @@ -128,7 +128,8 @@ public class ServerManager { * Map of admin interfaces per registered regionserver; these interfaces we use to control * regionservers out on the cluster */ - private final Map rsAdmins = new HashMap<>(); + private final Map rsAdmins = Review Comment: Better also change the declaration type to ConcurrentMap -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
Apache9 commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1759622141 Ah, OK, we change to use async connection for master and branch-3... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28146: Make ServerManager rsAdmins map thread safe [hbase]
bbeaudreault commented on PR #5461: URL: https://github.com/apache/hbase/pull/5461#issuecomment-1759553079 > This problem only affects 2.x? Master and branch-3 are safe? It looks like master was fixed via your commit https://github.com/apache/hbase/commit/5d872d34225157bfbc18a6600c958c93d96636fd. Should we make a similar change here? Otherwise if we intend to just fix this bug here, then looking at ServerManager.getRsAdmin, it is not thread safe. We should probably use computeIfAbsent if the get returns null. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28140 AbstractWALProvider may miss the WAL which is under creat… [hbase]
Apache9 commented on PR #5455: URL: https://github.com/apache/hbase/pull/5455#issuecomment-1759397697 The failed UTs are not related. Not sure why TestZooKeeper can fail with time out... Saw it several times recently... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28140 AbstractWALProvider may miss the WAL which is under creat… [hbase]
Apache-HBase commented on PR #5455: URL: https://github.com/apache/hbase/pull/5455#issuecomment-1759390434 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 29s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 26s | master passed | | +1 :green_heart: | compile | 0m 36s | master passed | | +1 :green_heart: | shadedjars | 5m 10s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 14s | the patch passed | | +1 :green_heart: | compile | 0m 36s | the patch passed | | +1 :green_heart: | javac | 0m 36s | the patch passed | | +1 :green_heart: | shadedjars | 5m 9s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 21s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 222m 56s | hbase-server in the patch failed. | | | | 244m 13s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5455 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 497bcea4f47e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 991d4aab51 | | Default Java | Temurin-1.8.0_352-b08 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/testReport/ | | Max. process+thread count | 4637 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28140 AbstractWALProvider may miss the WAL which is under creat… [hbase]
Apache-HBase commented on PR #5455: URL: https://github.com/apache/hbase/pull/5455#issuecomment-1759381079 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 26s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 17s | master passed | | +1 :green_heart: | compile | 1m 2s | master passed | | +1 :green_heart: | shadedjars | 5m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 35s | the patch passed | | +1 :green_heart: | compile | 0m 42s | the patch passed | | +1 :green_heart: | javac | 0m 42s | the patch passed | | +1 :green_heart: | shadedjars | 5m 8s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 22s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 214m 21s | hbase-server in the patch failed. | | | | 237m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5455 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 89eb6b214063 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 991d4aab51 | | Default Java | Eclipse Adoptium-11.0.17+8 | | unit | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt | | Test Results | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/testReport/ | | Max. process+thread count | 4552 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/console | | versions | git=2.34.1 maven=3.8.6 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-28138) Make the connection idle timeout configurable for the embedded HTTP servers
[ https://issues.apache.org/jira/browse/HBASE-28138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17774427#comment-17774427 ] Duo Zhang commented on HBASE-28138: --- This can not be applied to other branches? > Make the connection idle timeout configurable for the embedded HTTP servers > --- > > Key: HBASE-28138 > URL: https://issues.apache.org/jira/browse/HBASE-28138 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-4 >Reporter: qiuwei >Assignee: qiuwei >Priority: Minor > Fix For: 4.0.0-alpha-1 > > Attachments: > 0001-HBASE-28138-Slow-HTTP-Denial-of-Service-Attack-of-HB.patch > > > The Jetty Server may be slow HTTP denial of service attacks in hbase master > web ui due to the default value of jetty connector idle timeout is 3. > This may cause such an attack. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28138) Make the connection idle timeout configurable for the embedded HTTP servers
[ https://issues.apache.org/jira/browse/HBASE-28138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-28138: - Resolution: Fixed Status: Resolved (was: Patch Available) Thanks for the contribution [~qiuwei6]! I had now merged this into master branch. > Make the connection idle timeout configurable for the embedded HTTP servers > --- > > Key: HBASE-28138 > URL: https://issues.apache.org/jira/browse/HBASE-28138 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-4 >Reporter: qiuwei >Assignee: qiuwei >Priority: Minor > Fix For: 4.0.0-alpha-1 > > Attachments: > 0001-HBASE-28138-Slow-HTTP-Denial-of-Service-Attack-of-HB.patch > > > The Jetty Server may be slow HTTP denial of service attacks in hbase master > web ui due to the default value of jetty connector idle timeout is 3. > This may cause such an attack. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28138) Make the connection idle timeout configurable for the embedded HTTP servers
[ https://issues.apache.org/jira/browse/HBASE-28138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wellington Chevreuil updated HBASE-28138: - Fix Version/s: 4.0.0-alpha-1 (was: 3.0.0-beta-1) > Make the connection idle timeout configurable for the embedded HTTP servers > --- > > Key: HBASE-28138 > URL: https://issues.apache.org/jira/browse/HBASE-28138 > Project: HBase > Issue Type: Improvement >Affects Versions: 3.0.0-alpha-4 >Reporter: qiuwei >Assignee: qiuwei >Priority: Minor > Fix For: 4.0.0-alpha-1 > > Attachments: > 0001-HBASE-28138-Slow-HTTP-Denial-of-Service-Attack-of-HB.patch > > > The Jetty Server may be slow HTTP denial of service attacks in hbase master > web ui due to the default value of jetty connector idle timeout is 3. > This may cause such an attack. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28138 Make the connection idle timeout configurable for the embedded HTTP servers [hbase]
wchevreuil merged PR #5457: URL: https://github.com/apache/hbase/pull/5457 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Description: create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then will try again and again, will write too many proc record to master:store, we find num of the master WAL in oldWALs more than 13000.. Q: should add a suspend time logic for create table proc retry? i see TransitRegionStateProcedure has the logic.. --- sorry, i upload screenshot failed, just copy to here {code:java} // 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900) 2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created at hdfs://hacluster/hbase/.tmp for table themis:a | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346) 2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159) java.io.IOException: Unable to move table from temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase root=hdfs://hacluster/hbase/data/themis/a at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) 2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102) 2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating {ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME => 'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp | org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting for flushes and compactions to finish for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total wait time for flushes and compaction for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-the
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Description: create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then will try again and again, will write too many proc record to master:store, we find num of the master WAL in oldWALs more than 13000.. Q: should add a suspend time logic for create table proc retry? i see TransitRegionStateProcedure has the logic.. --- sorry, i upload screenshot failed, just copy to here {code:java} // 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900) 2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created at hdfs://hacluster/hbase/.tmp for table themis:a | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346) 2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159) java.io.IOException: Unable to move table from temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase root=hdfs://hacluster/hbase/data/themis/a at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) 2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102) 2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating {ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME => 'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp | org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting for flushes and compactions to finish for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total wait time for flushes and compaction for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-the
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Description: create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then will try again and again, will write too many proc record to master:store, we find num of the master WAL in oldWALs more than 13000.. Q: should add a suspend time logic for create table proc retry? i see TransitRegionStateProcedure has the logic.. --- sorry, i upload screenshot failed, just copy to here {code:java} // 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900) 2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created at hdfs://hacluster/hbase/.tmp for table themis:a | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346) 2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159) java.io.IOException: Unable to move table from temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase root=hdfs://hacluster/hbase/data/themis/a at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) 2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102) 2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating {ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME => 'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp | org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting for flushes and compactions to finish for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total wait time for flushes and compaction for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-the
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Description: create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then will try again and again, will write too many proc record to master:store, we find num of the master WAL in oldWALs more than 13000.. Q: should add a suspend time logic for create table proc retry? i see TransitRegionStateProcedure has the logic.. --- sorry, i upload screenshot failed, just copy to here {code:java} // 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900) 2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created at hdfs://hacluster/hbase/.tmp for table themis:a | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346) 2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159) java.io.IOException: Unable to move table from temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase root=hdfs://hacluster/hbase/data/themis/a at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) 2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102) 2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating {ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME => 'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp | org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting for flushes and compactions to finish for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total wait time for flushes and compaction for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-the
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Description: create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then will try again and again, will write too many proc record to master:store, we find num of the master WAL in oldWALs more than 1300.. --- sorry, i upload screenshot failed, just copy to here {code:java} // 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900) 2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created at hdfs://hacluster/hbase/.tmp for table themis:a | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346) 2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159) java.io.IOException: Unable to move table from temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase root=hdfs://hacluster/hbase/data/themis/a at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) 2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102) 2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating {ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME => 'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp | org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting for flushes and compactions to finish for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total wait time for flushes and compaction for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Description: create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then will try again and again, will write too many proc record to master:store, we find num of the master WAL in oldWALs more than 13000.. --- sorry, i upload screenshot failed, just copy to here {code:java} // 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900) 2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created at hdfs://hacluster/hbase/.tmp for table themis:a | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346) 2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159) java.io.IOException: Unable to move table from temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase root=hdfs://hacluster/hbase/data/themis/a at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) 2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102) 2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating {ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME => 'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp | org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting for flushes and compactions to finish for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total wait time for flushes and compaction for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Description: create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then will try again and again, will write too many proc record to master:store, we find num of the master WAL in oldWALs more than 1300.. --- sorry, i upload screenshot failed, just copy to here {code:java} // 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,360 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1900) 2023-10-12 12:34:35,360 | INFO | PEWorker-1 | Region directories are created at hdfs://hacluster/hbase/.tmp for table themis:a | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:346) 2023-10-12 12:34:35,362 | WARN | PEWorker-1 | Retriable error trying to create table=themis:a state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:159) java.io.IOException: Unable to move table from temp=hdfs://hacluster/hbase/.tmp/data/themis/a to hbase root=hdfs://hacluster/hbase/data/themis/a at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.moveTempDirectoryToHBaseRoot(CreateTableProcedure.java:391) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:350) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.createFsLayout(CreateTableProcedure.java:318) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:121) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:75) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1650) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1396) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1000(ProcedureExecutor.java:75) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.runProcedure(ProcedureExecutor.java:1962) at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:221) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1988) 2023-10-12 12:34:35,387 | INFO | PEWorker-1 | pid=917, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=themis:a execute state=CREATE_TABLE_WRITE_FS_LAYOUT | org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:102) 2023-10-12 12:34:35,414 | INFO | RegionOpenAndInit-themis:a-pool-0 | creating {ENCODED => 513d3d5b4d3ad5c8f13bacea4a888d69, NAME => 'themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='themis:a', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://hacluster/hbase/.tmp | org.apache.hadoop.hbase.regionserver.HRegion.createHRegion(HRegion.java:7906) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Waiting for flushes and compactions to finish for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1911) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Total wait time for flushes and compaction for the region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. is: 0ms | org.apache.hadoop.hbase.regionserver.HRegion.waitForFlushesAndCompactions(HRegion.java:1946) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closing region themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:1688) 2023-10-12 12:34:35,432 | INFO | RegionOpenAndInit-themis:a-pool-0 | Closed themis:a,,1697025107991.513d3d5b4d3ad5c8f13bacea4a888d69. | org.apache.hadoop.hbase.regionserver.
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Description: create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then will try again and again, will write too many proc record to master:store, we find num of the master WAL in oldWALs more than 1300.. > Too many master region WAL cause HDFS disk space full > - > > Key: HBASE-28150 > URL: https://issues.apache.org/jira/browse/HBASE-28150 > Project: HBase > Issue Type: Bug >Affects Versions: 2.4.14 >Reporter: chaijunjie >Priority: Critical > > create a table, but it failed when execute CREATE_TABLE_WRITE_FS_LAYOUT, then > will try again and again, will write too many proc record to master:store, we > find num of the master WAL in oldWALs more than 1300.. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28150) Too many master region WAL cause HDFS disk space full
[ https://issues.apache.org/jira/browse/HBASE-28150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chaijunjie updated HBASE-28150: --- Issue Type: Bug (was: Improvement) > Too many master region WAL cause HDFS disk space full > - > > Key: HBASE-28150 > URL: https://issues.apache.org/jira/browse/HBASE-28150 > Project: HBase > Issue Type: Bug >Affects Versions: 2.4.14 >Reporter: chaijunjie >Priority: Critical > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28150) Too many master region WAL cause HDFS disk space full
chaijunjie created HBASE-28150: -- Summary: Too many master region WAL cause HDFS disk space full Key: HBASE-28150 URL: https://issues.apache.org/jira/browse/HBASE-28150 Project: HBase Issue Type: Improvement Affects Versions: 2.4.14 Reporter: chaijunjie -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-28149) move_servers_namespaces_rsgroup is not changing the new rs group in namespace description
Rajeshbabu Chintaguntla created HBASE-28149: --- Summary: move_servers_namespaces_rsgroup is not changing the new rs group in namespace description Key: HBASE-28149 URL: https://issues.apache.org/jira/browse/HBASE-28149 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Rajeshbabu Chintaguntla {noformat} hbase:024:0> list_rsgroups NAME SERVER / TABLE rs1 server hostname2:16020 table hbase:meta table hbase:acl table hbase:namespace table hbase:rsgroup default server hostname3:16020 table ns_R:ta tenant_group1 tenant_group server hostname1:16020 table ns:tb1 table ns:tab table ns:t2 4 row(s) Took 0.0129 seconds hbase:025:0> move_servers_namespaces_rsgroup 'rs1',['hostname1:16020'], ['ns'] Took 0.0302 seconds hbase:026:0> list_rsgroups NAME SERVER / TABLE rs1 server hostname1:16020 server hostname2:16020 table ns:tb1 table ns:tab table hbase:meta table hbase:acl table ns:t2 table hbase:namespace table hbase:rsgroup default server hostname2:16020 table ns_R:ta tenant_group1 tenant_group 4 row(s) Took 0.0140 seconds hbase:027:0> describe_namespace 'ns' DESCRIPTION {NAME => 'ns', hbase.rsgroup.name => 'tenant_group'} Quota is disabled Took 0.0093 seconds hbase:028:0> {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28149) move_servers_namespaces_rsgroup is not changing the new rs group in namespace description
[ https://issues.apache.org/jira/browse/HBASE-28149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-28149: Issue Type: Bug (was: Improvement) > move_servers_namespaces_rsgroup is not changing the new rs group in namespace > description > - > > Key: HBASE-28149 > URL: https://issues.apache.org/jira/browse/HBASE-28149 > Project: HBase > Issue Type: Bug >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > > {noformat} > hbase:024:0> list_rsgroups > NAME SERVER / TABLE > rs1 server > hostname2:16020 > table hbase:meta > table hbase:acl > table > hbase:namespace > table > hbase:rsgroup > default server > hostname3:16020 > table ns_R:ta > tenant_group1 > tenant_group server > hostname1:16020 > table ns:tb1 > table ns:tab > table ns:t2 > 4 row(s) > Took 0.0129 seconds > hbase:025:0> move_servers_namespaces_rsgroup 'rs1',['hostname1:16020'], ['ns'] > Took 0.0302 seconds > hbase:026:0> list_rsgroups > NAME SERVER / TABLE > rs1 server > hostname1:16020 > server > hostname2:16020 > table ns:tb1 > table ns:tab > table hbase:meta > table hbase:acl > table ns:t2 > table > hbase:namespace > table > hbase:rsgroup > default server > hostname2:16020 > table ns_R:ta > tenant_group1 > tenant_group > 4 row(s) > Took 0.0140 seconds > hbase:027:0> describe_namespace 'ns' > DESCRIPTION > {NAME => 'ns', hbase.rsgroup.name => 'tenant_group'} > Quota is disabled > Took 0.0093 seconds > hbase:028:0> > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28140 AbstractWALProvider may miss the WAL which is under creat… [hbase]
Apache-HBase commented on PR #5455: URL: https://github.com/apache/hbase/pull/5455#issuecomment-1759078120 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 24s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 55s | master passed | | +1 :green_heart: | compile | 2m 27s | master passed | | +1 :green_heart: | checkstyle | 0m 34s | master passed | | +1 :green_heart: | spotless | 0m 41s | branch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 24s | master passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 2m 34s | the patch passed | | +1 :green_heart: | compile | 2m 22s | the patch passed | | +1 :green_heart: | javac | 2m 22s | the patch passed | | +1 :green_heart: | checkstyle | 0m 33s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 9m 26s | Patch does not cause any errors with Hadoop 3.2.4 3.3.6. | | +1 :green_heart: | spotless | 0m 38s | patch has no errors when running spotless:check. | | +1 :green_heart: | spotbugs | 1m 30s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. | | | | 31m 25s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5455 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile | | uname | Linux 60731907958b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / 991d4aab51 | | Default Java | Eclipse Adoptium-11.0.17+8 | | Max. process+thread count | 79 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5455/5/console | | versions | git=2.34.1 maven=3.8.6 spotbugs=4.7.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-28135) Specify -Xms for tests
[ https://issues.apache.org/jira/browse/HBASE-28135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Somogyi resolved HBASE-28135. --- Fix Version/s: 2.6.0 2.4.18 2.5.6 3.0.0-beta-1 Resolution: Fixed Pushed to branch-2.4+. Thanks [~stoty]! > Specify -Xms for tests > -- > > Key: HBASE-28135 > URL: https://issues.apache.org/jira/browse/HBASE-28135 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1 > > > The default -Xms value is JVM dependent, but the host memory size is usually > included in the calculation. > -Xms in turn is used to calculate some GC parameters, for example NewSize and > OldSize, which affect the behaviour of tests. > As the memory consumption on the tests is not dependent on the host VM size, > we could set -Xms for the tests explictly, and enjoy more consistent test > results. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28135 Specify -Xms for tests [hbase]
petersomogyi merged PR #5451: URL: https://github.com/apache/hbase/pull/5451 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-28148) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit
[ https://issues.apache.org/jira/browse/HBASE-28148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28148. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to master. > Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit > --- > > Key: HBASE-28148 > URL: https://issues.apache.org/jira/browse/HBASE-28148 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 4.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28148 Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit [hbase]
Apache-HBase commented on PR #5459: URL: https://github.com/apache/hbase/pull/5459#issuecomment-1759047295 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 33s | Docker mode activated. | | -1 :x: | patch | 0m 4s | https://github.com/apache/hbase/pull/5459 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5459/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5459 | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5459/2/console | | versions | git=2.34.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28148 Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit [hbase]
Apache-HBase commented on PR #5459: URL: https://github.com/apache/hbase/pull/5459#issuecomment-1759047078 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 12s | Docker mode activated. | | -1 :x: | patch | 0m 3s | https://github.com/apache/hbase/pull/5459 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5459/2/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/5459 | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5459/2/console | | versions | git=2.34.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (HBASE-28147) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests
[ https://issues.apache.org/jira/browse/HBASE-28147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-28147. --- Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.4+. > Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/flaky-tests > > > Key: HBASE-28147 > URL: https://issues.apache.org/jira/browse/HBASE-28147 > Project: HBase > Issue Type: Task > Components: dependabot, scripts, security >Reporter: Duo Zhang >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [PR] HBASE-28148 Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit [hbase]
Apache-HBase commented on PR #5459: URL: https://github.com/apache/hbase/pull/5459#issuecomment-1759046875 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | patch | 0m 3s | https://github.com/apache/hbase/pull/5459 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hbase/pull/5459 | | Console output | https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5459/2/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] HBASE-28148 Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit [hbase]
Apache9 merged PR #5459: URL: https://github.com/apache/hbase/pull/5459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-28148) Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit
Duo Zhang created HBASE-28148: - Summary: Bump gitpython from 3.1.35 to 3.1.37 in /dev-support/git-jira-release-audit Key: HBASE-28148 URL: https://issues.apache.org/jira/browse/HBASE-28148 Project: HBase Issue Type: Task Components: dependabot, scripts, security Reporter: Duo Zhang Fix For: 4.0.0-alpha-1 -- This message was sent by Atlassian Jira (v8.20.10#820010)