BELUGA BEHR created HDFS-13157: ---------------------------------- Summary: Do Not Remove Blocks Sequentially During Dec omission Key: HDFS-13157 URL: https://issues.apache.org/jira/browse/HDFS-13157 Project: Hadoop HDFS Issue Type: Improvement Components: datanode, namenode Affects Versions: 3.0.0 Reporter: BELUGA BEHR
>From what I understand of [DataNode >decommissioning|https://github.com/apache/hadoop/blob/42a1c98597e6dba2e371510a6b2b6b1fb94e4090/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminManager.java] > it appears that all the blocks are scheduled for removal _in order._. I'm >not 100% sure what the ordering is exactly, but I think it's loops through >each data volume and schedules each block to be replicated elsewhere. The net >affect is that during a decommission, all of the DataNode transfer threads >slam on a single volume until it is cleaned out. At which point, they all >slam on the next volume, etc. Please randomize the block list so that there is a more even distribution across all volumes when decommissioning a node. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org