ragarkar commented on code in PR #4799:
URL: https://github.com/apache/hbase/pull/4799#discussion_r1121272883
##########
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/BalancerClusterState.java:
##########
@@ -553,6 +569,108 @@ enum LocalityType {
RACK
}
+ public float getOrComputeWeightedPrefetchRatio(int region, int server) {
+ return getRegionSizeMB(region) * getOrComputeRegionPrefetchRatio(region,
server);
+ }
+
+ protected float getRegionServerPrefetchRatio(int region, int
regionServerIndex) {
+ // Cost this server has from RegionLoad
+ float prefetchRatio = 0.0f;
+
+ // Get the prefetch ratio if the region is currently hosted on this server
+ for (int regionIndex : regionsPerServer[regionServerIndex]) {
+ if (region != regionIndex) {
+ continue;
+ }
+ Deque<BalancerRegionLoad> regionLoadList = regionLoads[regionIndex];
+
+ // The region is currently hosted on this region server. Now, get the
prefetch cache ratio
+ // for this region on this region server
+ prefetchRatio =
+ regionLoadList == null ? 0.0f :
regionLoadList.getLast().getPrefetchCacheRatio();
+
+ return prefetchRatio;
+ }
+
+ // Region is not currently hosted on this server. Check if the region was
prefetched on this
+ // server earlier. This can happen when the server was shutdown and the
cache was persisted.
+ // Seartch using the index name and server name and not the index id and
server id as these ids
+ // may change when a server is marked as dead or a new server is added.
+ String regionNameAsString = regions[region].getRegionNameAsString();
+ String serverNameAsString = servers[regionServerIndex].getServerName();
+ if (
+ historicalRegionServerPrefetchRatio != null
+ && historicalRegionServerPrefetchRatio.containsKey(regionNameAsString)
+ ) {
+ Map<String, Float> serverPrefetchRatio =
+ historicalRegionServerPrefetchRatio.get(regionNameAsString);
+ if (serverPrefetchRatio.containsKey(serverNameAsString)) {
+ prefetchRatio = serverPrefetchRatio.get(serverNameAsString);
+
+ // The old prefetch cache ratio has been accounted for and hence,
clear up this information
+ historicalRegionServerPrefetchRatio.remove(regionNameAsString,
serverPrefetchRatio);
+ }
+ }
+ return prefetchRatio;
Review Comment:
This historical prefetch ratio is just an indicator of how much a region was
prefetched on a region server before it was moved (because of the region server
going down or some other reason) to another region server. This information is
mainly used to calculate the difference between the prefetch ratio of this
region on a new region server and how much this region was prefetched on the
old region server (if any). If the region's prefetch ratio is higher on the
currently hosting region server, the region will not be moved back to the old
region server anyways. However, this historical prefetch ratio is useful if the
region is only partially prefetched on the new region server while the
historical prefetch ratio is higher to the extent that it still makes sense to
move the region back to the old region server assuming that the region's blocks
were persisted in the cache while shutting down the region server. In absence
of this metric, it will not be possible to find out how much a re
gion was historically prefetched on a region server. Without this metric, this
cost function will never move the regions based on the prefetch ratio as it
will not have any historical information to compare with.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
