[ https://issues.apache.org/jira/browse/HDFS-9918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215796#comment-15215796 ]
Walter Su commented on HDFS-9918: --------------------------------- We also need to sort locations by distance. It's unlikely but from 2 duplicated in-service blocks we want to choose the nearer one. The logic is like {{sortLocatedBlocks(..)}}, how about reuse it? We move {{BlockIndex}} & {{BlockToken}} according to {{Location}}. Like {code} //public void sortLocatedBlocks(final String targethost, for (LocatedBlock b : locatedblocks) { DatanodeInfo[] di = b.getLocations(); + HashMap<DatanodeInfo, Byte> locToIndex = null; + HashMap<DatanodeInfo, Token<BlockTokenIdentifier>> locToToken = null; + if(b instanceof LocatedStripedBlock){ + locToIndex = new HashMap<>(); + locToToken = new HashMap<>(); + LocatedStripedBlock lb = (LocatedStripedBlock) b; + for(int i=0; i<di.length; i++){ + locToIndex.put(di[i], lb.getBlockIndices()[i]); + locToToken.put(di[i], lb.getBlockTokens()[i]); + } + } // Move decommissioned/stale datanodes to the bottom ... networktopology.sortByDistance(client, b.getLocations(), activeLen); // must update cache since we modified locations array b.updateCachedStorageInfo(); + + if(b instanceof LocatedStripedBlock){ + LocatedStripedBlock lb = (LocatedStripedBlock) b; + for(int i =0 ;i<di.length;i++){ + lb.getBlockIndices()[i] = locToIndex.get(di[i]); + lb.getBlockTokens()[i] = locToToken.get(di[i]); + } + } {code} It passes your tests. > Erasure Coding: Sort located striped blocks based on decommissioned states > -------------------------------------------------------------------------- > > Key: HDFS-9918 > URL: https://issues.apache.org/jira/browse/HDFS-9918 > Project: Hadoop HDFS > Issue Type: Sub-task > Reporter: Rakesh R > Assignee: Rakesh R > Attachments: HDFS-9918-001.patch, HDFS-9918-002.patch, > HDFS-9918-003.patch, HDFS-9918-004.patch, HDFS-9918-005.patch, > HDFS-9918-006.patch > > > This jira is a follow-on work of HDFS-8786, where we do decommissioning of > datanodes having striped blocks. > Now, after decommissioning it requires to change the ordering of the storage > list so that the decommissioned datanodes should only be last node in list. > For example, assume we have a block group with storage list:- > d0, d1, d2, d3, d4, d5, d6, d7, d8, d9 > mapping to indices > 0, 1, 2, 3, 4, 5, 6, 7, 8, 2 > Here the internal block b2 is duplicated, locating in d2 and d9. If d2 is a > decommissioning node then should switch d2 and d9 in the storage list. > Thanks [~jingzhao] for the > [discussions|https://issues.apache.org/jira/browse/HDFS-8786?focusedCommentId=15180415&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15180415] -- This message was sent by Atlassian JIRA (v6.3.4#6332)