Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on PR #6951: URL: https://github.com/apache/hbase/pull/6951#issuecomment-2978738504 I see, thanks. I guess it'll get easier once I get used to the process. I can imagine writing up a script to automate as much of it as possible. Could it be a good candidate for a `dev-support` script? I'll let you know of the progress. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
NihalJain commented on PR #6951: URL: https://github.com/apache/hbase/pull/6951#issuecomment-2977276978 > Thank you @NihalJain for helping me out, and apologies for missing that part! Could you share your process of adding the "Signed-off-by" and "Reviewed-by" lines to the commit message? Do I have to manually collect the email addresses of the reviewers and write the lines, or is there an easier way to do it? I think its all manual, I usually refer hbase teams https://hbase.apache.org/team.html page, or old commits to get email id for a user; if still cant find it, I check their github homepage. If neither of these help, I skip adding it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on PR #6951: URL: https://github.com/apache/hbase/pull/6951#issuecomment-2977098299 Thank you @NihalJain for helping me out, and apologies for missing that part! Could you share your process of adding the "Signed-off-by" and "Reviewed-by" lines to the commit message? Do I have to manually collect the email addresses of the reviewers and write the lines, or is there an easier way to do it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
NihalJain commented on PR #6951: URL: https://github.com/apache/hbase/pull/6951#issuecomment-2977031874 Hi @junegunn welcome to hbase committers' group. I noticed you have merged a few PRs and missed to mention signed-off to those. Not a problem, even I did same when I became a new committer. But [this doc](https://docs.google.com/document/d/1_Op2OYLGcHaxJk6-YAQD6R3enTz9IXIf8_00Qk7gnYk/edit?tab=t.0) helped me understand how-to's specific to Apache HBase, might be useful to you as well. I wish this (HBASE-21701) was part of our official book! CC: @chandrasekhar-188k @guluo2016 @charlesconnell -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on PR #6951: URL: https://github.com/apache/hbase/pull/6951#issuecomment-2969465774 @Apache9 Thanks for the approval. I've opened [a PR](https://github.com/apache/hbase/pull/7100) to backport this to `branch-2`. But do you think it's okay to backport this to `branch-2.6` as well? I personally think it's a great enhancement of the feature, and it's safe to backport because it's completely backward compatible, but I'm curious how you see it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn merged PR #6951: URL: https://github.com/apache/hbase/pull/6951 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
Apache9 commented on code in PR #6951:
URL: https://github.com/apache/hbase/pull/6951#discussion_r2119177309
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ReopenTableRegionsProcedure.java:
##
@@ -83,6 +86,23 @@ public class ReopenTableRegionsProcedure
private long regionsReopened = 0;
private long batchesProcessed = 0;
+ /**
+ * Create a new ReopenTableRegionsProcedure respecting the throttling
configuration for the table.
+ * First check the table descriptor, then fall back to the global
configuration. Only used in
+ * ModifyTableProcedure.
+ */
+ public static ReopenTableRegionsProcedure throttled(final Configuration conf,
+final TableDescriptor desc) {
+long backoffMillis =
Optional.ofNullable(desc.getValue(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY))
Review Comment:
Pity we do not return Optional for desc.getValue...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
Apache-HBase commented on PR #6951:
URL: https://github.com/apache/hbase/pull/6951#issuecomment-2854236753
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 30s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 16s | | master passed |
| +1 :green_heart: | compile | 0m 58s | | master passed |
| +1 :green_heart: | javadoc | 0m 28s | | master passed |
| +1 :green_heart: | shadedjars | 5m 53s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 7s | | the patch passed |
| +1 :green_heart: | compile | 0m 57s | | the patch passed |
| +1 :green_heart: | javac | 0m 57s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed |
| +1 :green_heart: | shadedjars | 5m 51s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 211m 33s | | hbase-server in the patch
passed. |
| | | 237m 35s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/6951 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux c047c7f27940 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 3f520a40e3288b37a25d2c96481e5f1fd5c54b4c |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/4/testReport/
|
| Max. process+thread count | 5445 (vs. ulimit of 3) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/4/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
Apache-HBase commented on PR #6951:
URL: https://github.com/apache/hbase/pull/6951#issuecomment-2853659682
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 26s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 14s | | master passed |
| +1 :green_heart: | compile | 3m 16s | | master passed |
| +1 :green_heart: | checkstyle | 0m 35s | | master passed |
| +1 :green_heart: | spotbugs | 1m 36s | | master passed |
| +1 :green_heart: | spotless | 0m 47s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 5s | | the patch passed |
| +1 :green_heart: | compile | 3m 17s | | the patch passed |
| +1 :green_heart: | javac | 3m 17s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 37s | | the patch passed |
| +1 :green_heart: | spotbugs | 1m 40s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 54s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 44s | | patch has no errors when
running spotless:check. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not
generate ASF License warnings. |
| | | 38m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/4/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/6951 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell
detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux e55aefbad18d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 3f520a40e3288b37a25d2c96481e5f1fd5c54b4c |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 86 (vs. ulimit of 3) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/4/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on PR #6951: URL: https://github.com/apache/hbase/pull/6951#issuecomment-2853552423 Build failure seems unrelated. Force-pushed to trigger a rebuild. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
Apache-HBase commented on PR #6951:
URL: https://github.com/apache/hbase/pull/6951#issuecomment-2853119325
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 25s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 2s | | master passed |
| +1 :green_heart: | compile | 0m 58s | | master passed |
| +1 :green_heart: | javadoc | 0m 29s | | master passed |
| +1 :green_heart: | shadedjars | 5m 53s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 0s | | the patch passed |
| +1 :green_heart: | compile | 0m 55s | | the patch passed |
| +1 :green_heart: | javac | 0m 55s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed |
| +1 :green_heart: | shadedjars | 5m 49s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 210m 30s | | hbase-server in the patch
passed. |
| | | 236m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/6951 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux da1fa9070042 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5529cb47bd14a5e6eae57edc3d96fef5b0fd44c9 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/3/testReport/
|
| Max. process+thread count | 5116 (vs. ulimit of 3) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/3/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
Apache-HBase commented on PR #6951:
URL: https://github.com/apache/hbase/pull/6951#issuecomment-2852533227
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 15s | | master passed |
| +1 :green_heart: | compile | 3m 11s | | master passed |
| +1 :green_heart: | checkstyle | 0m 37s | | master passed |
| +1 :green_heart: | spotbugs | 1m 33s | | master passed |
| +1 :green_heart: | spotless | 0m 46s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 6s | | the patch passed |
| -1 :x: | compile | 1m 23s |
[/patch-compile-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/3/artifact/yetus-general-check/output/patch-compile-hbase-server.txt)
| hbase-server in the patch failed. |
| -0 :warning: | javac | 1m 23s |
[/patch-compile-hbase-server.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/3/artifact/yetus-general-check/output/patch-compile-hbase-server.txt)
| hbase-server in the patch failed. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 36s | | the patch passed |
| +1 :green_heart: | spotbugs | 2m 4s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 53s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 44s | | patch has no errors when
running spotless:check. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not
generate ASF License warnings. |
| | | 37m 2s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/3/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/6951 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell
detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 0628e539 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 5529cb47bd14a5e6eae57edc3d96fef5b0fd44c9 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 89 (vs. ulimit of 3) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/3/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on code in PR #6951:
URL: https://github.com/apache/hbase/pull/6951#discussion_r2073270852
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java:
##
@@ -231,10 +232,19 @@ protected Flow executeFromState(final MasterProcedureEnv
env, final ModifyTableS
case MODIFY_TABLE_REOPEN_ALL_REGIONS:
if (isTableEnabled(env)) {
Configuration conf = env.getMasterConfiguration();
-long backoffMillis =
conf.getLong(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY,
- PROGRESSIVE_BATCH_BACKOFF_MILLIS_DEFAULT);
+// Use table configuration if defined
+final TableDescriptor descriptor =
+
env.getMasterServices().getTableDescriptors().get(getTableName());
+long backoffMillis =
+
Optional.ofNullable(descriptor.getValue(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY))
+.map(Long::parseLong)
+.orElseGet(() ->
conf.getLong(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY,
+ PROGRESSIVE_BATCH_BACKOFF_MILLIS_DEFAULT));
int batchSizeMax =
- conf.getInt(PROGRESSIVE_BATCH_SIZE_MAX_KEY,
PROGRESSIVE_BATCH_SIZE_MAX_DISABLED);
+
Optional.ofNullable(descriptor.getValue(PROGRESSIVE_BATCH_SIZE_MAX_KEY))
+.map(Integer::parseInt).orElseGet(() ->
conf.getInt(PROGRESSIVE_BATCH_SIZE_MAX_KEY,
+ PROGRESSIVE_BATCH_SIZE_MAX_DISABLED));
+
Review Comment:
Refactored the code and added a simple test case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on code in PR #6951:
URL: https://github.com/apache/hbase/pull/6951#discussion_r2071003300
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java:
##
@@ -231,10 +232,19 @@ protected Flow executeFromState(final MasterProcedureEnv
env, final ModifyTableS
case MODIFY_TABLE_REOPEN_ALL_REGIONS:
if (isTableEnabled(env)) {
Configuration conf = env.getMasterConfiguration();
-long backoffMillis =
conf.getLong(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY,
- PROGRESSIVE_BATCH_BACKOFF_MILLIS_DEFAULT);
+// Use table configuration if defined
+final TableDescriptor descriptor =
+
env.getMasterServices().getTableDescriptors().get(getTableName());
+long backoffMillis =
+
Optional.ofNullable(descriptor.getValue(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY))
+.map(Long::parseLong)
+.orElseGet(() ->
conf.getLong(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY,
+ PROGRESSIVE_BATCH_BACKOFF_MILLIS_DEFAULT));
int batchSizeMax =
- conf.getInt(PROGRESSIVE_BATCH_SIZE_MAX_KEY,
PROGRESSIVE_BATCH_SIZE_MAX_DISABLED);
+
Optional.ofNullable(descriptor.getValue(PROGRESSIVE_BATCH_SIZE_MAX_KEY))
+.map(Integer::parseInt).orElseGet(() ->
conf.getInt(PROGRESSIVE_BATCH_SIZE_MAX_KEY,
+ PROGRESSIVE_BATCH_SIZE_MAX_DISABLED));
+
Review Comment:
I didn't add a test case for this change because it's minimal and quite
self-evident.
Also, it's a little tricky to add a test case, because the values are used
for a parameterized constructor call, which is not easily verifiable with
Mockito. We could extract the code to a helper method or change the constructor
to take `env` instead, but both approaches feel like overkill – the latter
would require many changes to the existing test cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on code in PR #6951:
URL: https://github.com/apache/hbase/pull/6951#discussion_r2071003300
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java:
##
@@ -231,10 +232,19 @@ protected Flow executeFromState(final MasterProcedureEnv
env, final ModifyTableS
case MODIFY_TABLE_REOPEN_ALL_REGIONS:
if (isTableEnabled(env)) {
Configuration conf = env.getMasterConfiguration();
-long backoffMillis =
conf.getLong(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY,
- PROGRESSIVE_BATCH_BACKOFF_MILLIS_DEFAULT);
+// Use table configuration if defined
+final TableDescriptor descriptor =
+
env.getMasterServices().getTableDescriptors().get(getTableName());
+long backoffMillis =
+
Optional.ofNullable(descriptor.getValue(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY))
+.map(Long::parseLong)
+.orElseGet(() ->
conf.getLong(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY,
+ PROGRESSIVE_BATCH_BACKOFF_MILLIS_DEFAULT));
int batchSizeMax =
- conf.getInt(PROGRESSIVE_BATCH_SIZE_MAX_KEY,
PROGRESSIVE_BATCH_SIZE_MAX_DISABLED);
+
Optional.ofNullable(descriptor.getValue(PROGRESSIVE_BATCH_SIZE_MAX_KEY))
+.map(Integer::parseInt).orElseGet(() ->
conf.getInt(PROGRESSIVE_BATCH_SIZE_MAX_KEY,
+ PROGRESSIVE_BATCH_SIZE_MAX_DISABLED));
+
Review Comment:
I didn't add a test case for this change because it's minimal and quite
self-evident.
Also, it's a little tricky to add a test case, because the values are used
for a parameterized constructor call, which is not easily verifiable with
Mockito. We could extract the code to a helper method or add another
constructor that takes `env`, but both approaches feel like overkill – the
latter would require many changes to the existing test cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on code in PR #6951:
URL: https://github.com/apache/hbase/pull/6951#discussion_r2071004086
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ReopenTableRegionsProcedure.java:
##
@@ -55,12 +56,11 @@ public class ReopenTableRegionsProcedure
private static final Logger LOG =
LoggerFactory.getLogger(ReopenTableRegionsProcedure.class);
public static final String PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY =
-"hbase.reopen.table.regions.progressive.batch.backoff.ms";
+ConfigKey.LONG("hbase.reopen.table.regions.progressive.batch.backoff.ms");
public static final long PROGRESSIVE_BATCH_BACKOFF_MILLIS_DEFAULT = 0L;
public static final String PROGRESSIVE_BATCH_SIZE_MAX_KEY =
-"hbase.reopen.table.regions.progressive.batch.size.max";
+ConfigKey.INT("hbase.reopen.table.regions.progressive.batch.size.max");
Review Comment:
Type checks are now necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on code in PR #6951:
URL: https://github.com/apache/hbase/pull/6951#discussion_r2071003300
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ModifyTableProcedure.java:
##
@@ -231,10 +232,19 @@ protected Flow executeFromState(final MasterProcedureEnv
env, final ModifyTableS
case MODIFY_TABLE_REOPEN_ALL_REGIONS:
if (isTableEnabled(env)) {
Configuration conf = env.getMasterConfiguration();
-long backoffMillis =
conf.getLong(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY,
- PROGRESSIVE_BATCH_BACKOFF_MILLIS_DEFAULT);
+// Use table configuration if defined
+final TableDescriptor descriptor =
+
env.getMasterServices().getTableDescriptors().get(getTableName());
+long backoffMillis =
+
Optional.ofNullable(descriptor.getValue(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY))
+.map(Long::parseLong)
+.orElseGet(() ->
conf.getLong(PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY,
+ PROGRESSIVE_BATCH_BACKOFF_MILLIS_DEFAULT));
int batchSizeMax =
- conf.getInt(PROGRESSIVE_BATCH_SIZE_MAX_KEY,
PROGRESSIVE_BATCH_SIZE_MAX_DISABLED);
+
Optional.ofNullable(descriptor.getValue(PROGRESSIVE_BATCH_SIZE_MAX_KEY))
+.map(Integer::parseInt).orElseGet(() ->
conf.getInt(PROGRESSIVE_BATCH_SIZE_MAX_KEY,
+ PROGRESSIVE_BATCH_SIZE_MAX_DISABLED));
+
Review Comment:
I didn't add a test case for this change because it's minimal and quite
self-evident.
Also, it's a little tricky to add a test case, because the values are used
for a parameterized constructor call, which is not easily verifiable with
Mockito. We could extract the code to a helper method or add another
constructor that takes `env`, but both approaches feel like overkill.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on PR #6951: URL: https://github.com/apache/hbase/pull/6951#issuecomment-2844888135 Hi @rmdmattingly, when you have a moment, could you take a look at this? It's a simple change but I believe it makes the throttling feature much more useful in production. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
Apache-HBase commented on PR #6951:
URL: https://github.com/apache/hbase/pull/6951#issuecomment-2844012505
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 2s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 6s | | master passed |
| +1 :green_heart: | compile | 1m 21s | | master passed |
| +1 :green_heart: | javadoc | 0m 33s | | master passed |
| +1 :green_heart: | shadedjars | 7m 2s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 57s | | the patch passed |
| +1 :green_heart: | compile | 1m 12s | | the patch passed |
| +1 :green_heart: | javac | 1m 12s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 41s | | the patch passed |
| +1 :green_heart: | shadedjars | 7m 45s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 246m 0s | | hbase-server in the patch
passed. |
| | | 277m 52s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/6951 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux dace460498a8 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 2e9b0dfa436cd00526cbb0666a0a002e7250fbcf |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/2/testReport/
|
| Max. process+thread count | 4447 (vs. ulimit of 3) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/2/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
Apache-HBase commented on PR #6951:
URL: https://github.com/apache/hbase/pull/6951#issuecomment-2843742528
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 30s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 6s | | master passed |
| +1 :green_heart: | compile | 3m 10s | | master passed |
| +1 :green_heart: | checkstyle | 0m 39s | | master passed |
| +1 :green_heart: | spotbugs | 1m 36s | | master passed |
| +1 :green_heart: | spotless | 0m 45s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 7s | | the patch passed |
| +1 :green_heart: | compile | 3m 10s | | the patch passed |
| +1 :green_heart: | javac | 3m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 36s | | the patch passed |
| +1 :green_heart: | spotbugs | 1m 42s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 59s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 45s | | patch has no errors when
running spotless:check. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not
generate ASF License warnings. |
| | | 38m 33s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/2/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/6951 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell
detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 2f8346ba3061 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 2e9b0dfa436cd00526cbb0666a0a002e7250fbcf |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 84 (vs. ulimit of 3) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/2/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
Apache-HBase commented on PR #6951:
URL: https://github.com/apache/hbase/pull/6951#issuecomment-2843222492
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 27s | | master passed |
| +1 :green_heart: | compile | 1m 16s | | master passed |
| +1 :green_heart: | javadoc | 0m 39s | | master passed |
| +1 :green_heart: | shadedjars | 7m 11s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 16s | | the patch passed |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed |
| +1 :green_heart: | shadedjars | 7m 6s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 269m 19s | | hbase-server in the patch
passed. |
| | | 301m 22s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/6951 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 0ccd1948e500 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 3d0590decb13ce7eabdd811c79be14c66578bf12 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/1/testReport/
|
| Max. process+thread count | 4494 (vs. ulimit of 3) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/1/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
Apache-HBase commented on PR #6951:
URL: https://github.com/apache/hbase/pull/6951#issuecomment-2842574332
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 32s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ master Compile Tests _ |
| +1 :green_heart: | mvninstall | 5m 1s | | master passed |
| +1 :green_heart: | compile | 4m 15s | | master passed |
| +1 :green_heart: | checkstyle | 0m 54s | | master passed |
| +1 :green_heart: | spotbugs | 2m 25s | | master passed |
| +1 :green_heart: | spotless | 1m 15s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 54s | | the patch passed |
| +1 :green_heart: | compile | 4m 16s | | the patch passed |
| +1 :green_heart: | javac | 4m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 52s | | the patch passed |
| +1 :green_heart: | spotbugs | 2m 31s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 15m 1s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| -1 :x: | spotless | 0m 57s | | patch has 23 errors when running
spotless:check, run spotless:apply to fix. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | | The patch does not
generate ASF License warnings. |
| | | 52m 31s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/1/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/6951 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell
detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 1bfc58ca66a0 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | master / 3d0590decb13ce7eabdd811c79be14c66578bf12 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| spotless |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/1/artifact/yetus-general-check/output/patch-spotless.txt
|
| Max. process+thread count | 84 (vs. ulimit of 3) |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6951/1/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29279 Allow throttling alter operation via table configuration [hbase]
junegunn commented on code in PR #6951:
URL: https://github.com/apache/hbase/pull/6951#discussion_r2068943632
##
hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/ReopenTableRegionsProcedure.java:
##
@@ -55,12 +56,11 @@ public class ReopenTableRegionsProcedure
private static final Logger LOG =
LoggerFactory.getLogger(ReopenTableRegionsProcedure.class);
public static final String PROGRESSIVE_BATCH_BACKOFF_MILLIS_KEY =
-"hbase.reopen.table.regions.progressive.batch.backoff.ms";
+ConfigKey.LONG("hbase.reopen.table.regions.progressive.batch.backoff.ms");
public static final long PROGRESSIVE_BATCH_BACKOFF_MILLIS_DEFAULT = 0L;
public static final String PROGRESSIVE_BATCH_SIZE_MAX_KEY =
-"hbase.reopen.table.regions.progressive.batch.size.max";
+ConfigKey.INT("hbase.reopen.table.regions.progressive.batch.size.max");
public static final int PROGRESSIVE_BATCH_SIZE_MAX_DISABLED = -1;
- private static final int PROGRESSIVE_BATCH_SIZE_MAX_DEFAULT_VALUE =
Integer.MAX_VALUE;
Review Comment:
This one was unused.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
