[ https://issues.apache.org/jira/browse/HDFS-7461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoy Antony updated HDFS-7461: ------------------------------- Issue Type: Sub-task (was: Improvement) Parent: HDFS-7197 > Reduce impact of laggards on Mover > ---------------------------------- > > Key: HDFS-7461 > URL: https://issues.apache.org/jira/browse/HDFS-7461 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: balancer & mover > Affects Versions: 2.6.0 > Reporter: Benoy Antony > Assignee: Benoy Antony > > The current Mover logic is as follows : > {code} > for (Path target : targetPaths) { > hasRemaining |= processPath(target.toUri().getPath()); > } > // wait for pending move to finish and retry the failed migration > hasRemaining |= Dispatcher.waitForMoveCompletion(storages.targets.values()); > {code} > The _processPath_ will schedule moves, but it is bounded by the number of > concurrent moves (default is 5 per node} . Once block moves are scheduled, > it will wait for ALL scheduled moves to finish in _waitForMoveCompletion_. > One slow move could keep the Mover idle for a long time. > It will be a performance improvement to schedule the next moves as soon as > any (source , target) slot is available instead of waiting for all the > scheduled moves to finish. -- This message was sent by Atlassian JIRA (v6.3.4#6332)