[ https://issues.apache.org/jira/browse/HADOOP-16161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791292#comment-16791292 ]
He Xiaoqiao commented on HADOOP-16161: -------------------------------------- [~elgoiri], Thanks for digging deep to find the truth. {quote}The only issue would be if some code had a dependency on particular values of the weight. However, it looks like the test are passing with no issues.{quote} Actually, #getWeightUsingNetworkLocation is only invoke by getBlockLocations which determines block locations' order. in one word, #getWeightUsingNetworkLocation is about Read Locality. However I do not find any unit test to verify result of #sortLocatedBlocks directly in #blockmanagement, So it is well understood that test are passed under #blockmanagement. > NetworkTopology#getWeightUsingNetworkLocation return unexpected result > ---------------------------------------------------------------------- > > Key: HADOOP-16161 > URL: https://issues.apache.org/jira/browse/HADOOP-16161 > Project: Hadoop Common > Issue Type: Bug > Components: net > Reporter: He Xiaoqiao > Assignee: He Xiaoqiao > Priority: Major > Attachments: HADOOP-16161.001.patch, HADOOP-16161.002.patch, > HADOOP-16161.003.patch, HADOOP-16161.004.patch, HADOOP-16161.005.patch, > HADOOP-16161.006.patch > > > Consider the following scenario: > 1. there are 4 slaves and topology like: > Rack: /IDC/RACK1 > hostname1 > hostname2 > Rack: /IDC/RACK2 > hostname3 > hostname4 > 2. Reader from hostname1, and calculate weight between reader and [hostname1, > hostname3, hostname4] by #getWeight, and their corresponding values are > [0,4,4] > 3. Reader from client which is not in the topology, and in the same IDC but > in none rack of the topology, and calculate weight between reader and > [hostname1, hostname3, hostname4] by #getWeightUsingNetworkLocation, and > their corresponding values are [2,2,2] > 4. Other different Reader can get the similar results. > The weight result for case #3 is obviously not the expected value, the truth > is [4,4,4]. this issue may cause reader not really following arrange: local > -> local rack -> remote rack. > After dig the detailed implement, the root cause is > #getWeightUsingNetworkLocation only calculate distance between Racks rather > than hosts. > I think we should add constant 2 to correct the weight of > #getWeightUsingNetworkLocation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org