Hi Jameson, Inline reply below for a specific point of yours ("scheduler deadlock"):
On Mon, Jan 28, 2013 at 8:59 PM, Jameson Li <hovlj...@gmail.com> wrote: > Hi, > > Is it right that hadoop Network Topology design treat an imbalanced topology > as an invalid topology? > > This is mentioned here > https://issues.apache.org/jira/browse/MAPREDUCE-4095(TestJobInProgress#testLocality > uses a bogus topology.) > said:"(https://issues.apache.org/jira/secure/attachment/12345251/Rack_aware_HDFS_proposal.pdf) > which does not support imbalanced topology. " > > But when I re-read the desigh pdf, there just draw a balanced topology but > really not mentioned it will not support imbalanced topology. > > Also I found this JIRA: > https://issues.apache.org/jira/browse/HADOOP-8159(NetworkTopology: getLeaf > should check for invalid topologies). > > Just no why for my question, because of the design(Right?): > Why below is a invalid topology(except imbalanced), for hadoop rack > awareness: > serverA /default-rack ##default rack > serverB /dc1/rack2 ##actually the node has a pre "dc1" > Should all of the hadoop datanode have the same level number? If not, there > will be some issues for namenode and jt(like scheduler deadlock)? This was a problem in earlier releases, where an imbalanced or improper config for topology levels could lead to a jarring halt in the JobTracker's schedulers. This has been fixed in 1.1.0 already via https://issues.apache.org/jira/browse/MAPREDUCE-1740. -- Harsh J