[GitHub] [hbase] huaxiangsun commented on a change in pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
huaxiangsun commented on a change in pull request #1922:
URL: https://github.com/apache/hbase/pull/1922#discussion_r446344983
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -369,7 +369,8 @@ private boolean skipForMerge(final RegionStates
regionStates, final RegionInfo r
}
final long currentSizeMb = getRegionSizeMB(current);
final long nextSizeMb = getRegionSizeMB(next);
- if (currentSizeMb + nextSizeMb < avgRegionSizeMb) {
+ // always merge away empty regions when they present themselves.
+ if (currentSizeMb == 0 || nextSizeMb == 0 || currentSizeMb + nextSizeMb
< avgRegionSizeMb) {
Review comment:
Yeah, this is a good idea!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
huaxiangsun commented on a change in pull request #1922:
URL: https://github.com/apache/hbase/pull/1922#discussion_r15410
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -369,7 +369,8 @@ private boolean skipForMerge(final RegionStates
regionStates, final RegionInfo r
}
final long currentSizeMb = getRegionSizeMB(current);
final long nextSizeMb = getRegionSizeMB(next);
- if (currentSizeMb + nextSizeMb < avgRegionSizeMb) {
+ // always merge away empty regions when they present themselves.
+ if (currentSizeMb == 0 || nextSizeMb == 0 || currentSizeMb + nextSizeMb
< avgRegionSizeMb) {
Review comment:
> ```
> if (currentSizeMb + nextSizeMb < avgRegionSizeMb * 0.4) {...}
> ```
The above statement does not address the case that 100 1 80 (average 60), in
this case 100 is not qualified for split, and none is qualified for merge as
well.
Do not mean to block your commit, and for cases like above, need to spend
more time to find a right solution, we can create an improvement jira to
address it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
huaxiangsun commented on a change in pull request #1922:
URL: https://github.com/apache/hbase/pull/1922#discussion_r442372173
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -369,7 +369,8 @@ private boolean skipForMerge(final RegionStates
regionStates, final RegionInfo r
}
final long currentSizeMb = getRegionSizeMB(current);
final long nextSizeMb = getRegionSizeMB(next);
- if (currentSizeMb + nextSizeMb < avgRegionSizeMb) {
+ // always merge away empty regions when they present themselves.
+ if (currentSizeMb == 0 || nextSizeMb == 0 || currentSizeMb + nextSizeMb
< avgRegionSizeMb) {
Review comment:
0.1 is a small random number, can be 0.05.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
huaxiangsun commented on a change in pull request #1922:
URL: https://github.com/apache/hbase/pull/1922#discussion_r442371151
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -369,7 +369,8 @@ private boolean skipForMerge(final RegionStates
regionStates, final RegionInfo r
}
final long currentSizeMb = getRegionSizeMB(current);
final long nextSizeMb = getRegionSizeMB(next);
- if (currentSizeMb + nextSizeMb < avgRegionSizeMb) {
+ // always merge away empty regions when they present themselves.
+ if (currentSizeMb == 0 || nextSizeMb == 0 || currentSizeMb + nextSizeMb
< avgRegionSizeMb) {
Review comment:
There is a special case for size to be 0, wondering if the region size
is very small but nonzero, and there is a large size neighbor region, can we do
something about it?
100 1 100 (avg size is 20), in this case, 1 wont be merged as well. What
if some logic like
```currentSizeMb < 0.1 * avgRegionSizeMb and currentSizeMb/nextSizeMb < 0.1
(switch currentSizeMb and nextSizeMb as well), then merge```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
huaxiangsun commented on a change in pull request #1922:
URL: https://github.com/apache/hbase/pull/1922#discussion_r442371151
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -369,7 +369,8 @@ private boolean skipForMerge(final RegionStates
regionStates, final RegionInfo r
}
final long currentSizeMb = getRegionSizeMB(current);
final long nextSizeMb = getRegionSizeMB(next);
- if (currentSizeMb + nextSizeMb < avgRegionSizeMb) {
+ // always merge away empty regions when they present themselves.
+ if (currentSizeMb == 0 || nextSizeMb == 0 || currentSizeMb + nextSizeMb
< avgRegionSizeMb) {
Review comment:
There is a special case for size to be 0, wondering if the region size
is very small but nonzero, and there is a large size neighbor region, can we do
something about it?
100 1 100 (avg size is 20), in this case, 1 wont be merged as well. What
if some logic like
currentSizeMb < 0.1 * avgRegionSizeMb and currentSizeMb/nextSizeMb < 0.1
(switch currentSizeMb and nextSizeMb as well), then merge
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] huaxiangsun commented on a change in pull request #1922: HBASE-24583 Normalizer can't actually merge empty regions when neighbor is larger than average size
huaxiangsun commented on a change in pull request #1922:
URL: https://github.com/apache/hbase/pull/1922#discussion_r442371151
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/SimpleRegionNormalizer.java
##
@@ -369,7 +369,8 @@ private boolean skipForMerge(final RegionStates
regionStates, final RegionInfo r
}
final long currentSizeMb = getRegionSizeMB(current);
final long nextSizeMb = getRegionSizeMB(next);
- if (currentSizeMb + nextSizeMb < avgRegionSizeMb) {
+ // always merge away empty regions when they present themselves.
+ if (currentSizeMb == 0 || nextSizeMb == 0 || currentSizeMb + nextSizeMb
< avgRegionSizeMb) {
Review comment:
There is a special case for size to be 0, wondering if the region size
is very small but nonzero, and there is a large size neighbor region, can we do
something about it?
100 1 100 (avg size is 20), in this case, 1 wont be merged as well. What
if some logic like
'''currentSizeMb < 0.1 * avgRegionSizeMb and currentSizeMb/nextSizeMb < 0.1
(switch currentSizeMb and nextSizeMb as well), then merge'''
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
