[ https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260643#comment-13260643 ]
Ashish Singhi commented on HDFS-3295: ------------------------------------- My analysis : In each iteration we create a new Balancer object {code} b = new Balancer(nnc, p, conf); {code} The bytesMoved calculated from dispatchBlockMoves() in iteration n will get set to 0 again in iteration n+1 as per the above code. Hence when it reaches, {code} formatter.format("%-24s %10d %19s %18s %17s\n", DateFormat.getDateTimeInstance().format(new Date()), iteration, StringUtils.byteDesc(bytesMoved.get()), StringUtils.byteDesc(bytesLeftToMove), StringUtils.byteDesc(bytesToMove) ); {code} bytesMoved will be equal to 0. Also I have seen some cases where there is no block move for 5 continuous iterations Balancer is not exiting. This is also due to the above reason where notChangedIterations is getting set 0 for each iteration. Hence it never reaches the value >1. > Mismatch in number of bytes already moved and number of bytes being moved in > balancer report > -------------------------------------------------------------------------------------------- > > Key: HDFS-3295 > URL: https://issues.apache.org/jira/browse/HDFS-3295 > Project: Hadoop HDFS > Issue Type: Bug > Components: balancer > Affects Versions: 0.23.0 > Reporter: J.Andreina > Assignee: Ashish Singhi > Fix For: 0.24.0 > > > Scenario: > Replication factor = > 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port. > step 1: started DN1.Pumped 4.67GB of data > step 2: started DN2. > step 3: issued the balancer cmd(./start-balancer.sh -threshold 1) > Totally 848.41 MB has been moved to 2DN and took 4 iterations to move the > blocks to 2DN. > But in balancer output in all the iterations the number of bytes already > moved is alway 0KB and there is mismatch in the Bytes being moved with the > bytes left to move > Balancer output for 2nd Iteration > ================================= > {noformat} > Time Stamp Iteration# Bytes Already Moved Bytes Left To Move > Bytes Being Moved > Apr 18, 2012 12:41:20 PM 0 0 KB 848.41 MB > 151.18 MB > Apr 18, 2012 1:06:46 PM 1 0 KB 646.02 MB > 151.18 MB > Apr 18, 2012 1:10:28 PM 2 0 KB 417.43 MB > 151.18 MB > Apr 18, 2012 1:14:04 PM 3 0 KB 223.84 MB > 151.18 MB > Apr 18, 2012 1:18:10 PM 4 0 KB 30.34 MB > 151.18 MB{noformat} > In the above balancer output > •Bytes already moved is 0KB > • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is > 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - > 151.18 MB -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira