Rajan Dhabalia created HDFS-17807:
-------------------------------------
Summary: Support balanced read distribution ordering in
getBlockLocations()
Key: HDFS-17807
URL: https://issues.apache.org/jira/browse/HDFS-17807
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Affects Versions: 3.4.1, 2.10.0
Reporter: Rajan Dhabalia
The current getBlockLocations method returns replica locations without
considering the real-time load on individual DataNodes. This can lead to
clients repeatedly selecting the same DataNode, particularly when network
distance or rack locality remains unchanged. Since data locality isn’t always
reliable due to various operational factors, the existing network
topology–based ordering has not fully optimized performance and may
inadvertently create load hotspots. so, replica ordering based on dynamic
weight metric and specifically current read rate or read load will create more
balanced read across the dataodes in a cluster.
The policy is disabled by default and can be enabled to replace the current
ordering mechanism, offering a more adaptive and performance-aware read path
strategy.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
