Wei Song created HDFS-17743:
-------------------------------
Summary: Support random datanode ordering in getBlockLocations()
Key: HDFS-17743
URL: https://issues.apache.org/jira/browse/HDFS-17743
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Affects Versions: 3.4.1, 2.10.0
Reporter: Wei Song
At LinkedIn, we aren't able to rely on data locality due to various reasons,
overall the current implementation of sorting by network topology didn't
improve overall performance, it also caused unnecessary concentration of load
on specific datanodes. We would like to request for a random policy that
randomly order datanodes, we expect this policy to improve load distribution.
The new policy should be configurable, and it is disabled. When enabled, it
replaces the current network topology based datanode ordering.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
