[
https://issues.apache.org/jira/browse/HADOOP-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242574#comment-13242574
]
Steve Loughran commented on HADOOP-8232:
----------------------------------------
test with a table that can be found; the table is
{code}
host1 /rack1
host2 /rack1
host3 /rack2/switch1
host4 /rack2/switch1
{code}
the output is
{code}
bash hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/bin/hadoop --config
/Users/slo/Java/Hadoop/github/hadoop-patches/misc/topology/table
org.apache.hadoop.net.TopologyTool testfile
/Users/slo/Java/Hadoop/github/hadoop-patches/misc/topology/hosts.txt
net.topology.node.switch.mapping.impl=org.apache.hadoop.net.TableMapping
Caching wrapper class = class org.apache.hadoop.net.TableMapping
Instance information: cached switch mapping relaying to TableMapping with table
"/Users/slo/Java/Hadoop/github/hadoop-patches/misc/topology/table/table.txt"
Path: /Users/slo/Java/Hadoop/github/hadoop-patches/misc/topology/table/table.txt
Map size: 0
Topology is not considered single-switch
Resolving host1
Hostname "host1" resolved to "/rack1" in 17.476 milliseconds
Resolving host2
Hostname "host2" resolved to "/rack1" in 0.878 milliseconds
Resolving host3
Hostname "host3" resolved to "/rack2/switch1" in 0.428 milliseconds
Resolving host4
Hostname "host4" resolved to "/rack2/switch1" in 0.34 milliseconds
Resolving host5
Hostname "host5" resolved to "/default-rack" in 0.578 milliseconds
Resolving host5
Hostname "host5" resolved to "/default-rack" in 0.095 milliseconds
Resolving host3
Hostname "host3" resolved to "/rack2/switch1" in 0.023 milliseconds
Resolving host1.hadoop.apache.org
Hostname "host1.hadoop.apache.org" resolved to "/default-rack" in 0.555
milliseconds
Final topology:
Mapping: cached switch mapping relaying to TableMapping with table
"/Users/slo/Java/Hadoop/github/hadoop-patches/misc/topology/table/table.txt"
Path: /Users/slo/Java/Hadoop/github/hadoop-patches/misc/topology/table/table.txt
Map size: 4
Known mappings:
host1.hadoop.apache.org -> /default-rack
host5 -> /default-rack
host4 -> /rack2/switch1
host3 -> /rack2/switch1
host2 -> /rack1
host1 -> /rack1
Nodes: 6
Switches: 3
{code}
The output at the end is less than ideal. The "Map Size" value is from the
{{TableMapping.toString()}} method, which, in the patch, lists it. The name
could be changed, but what it represents can't: it is the size of the
hostname->rack values provided by the table file.
The map that is then printed below it is the map known by the
CachedDNSSwitchMapping, which implements the method needed to get a copy of its
mapping for diagnostics purposes.
Ideally, both should be shown, which could be done by having the inner
{{RawTableMapping}} extend {{AbstractDNSToSwitchMapping}} and implement the
{{getSwitchMap()}} method by returning its static map. That still needs to be
made visible somehow through {{CachedDNSMapping}}
An alternative would just be for {{TableMapping.getSwitchMap()} to merge its
cached values with those provided by the original table, so displaying
everything that came in the table, and any cached values for unknown hosts now
mapped to /default-rack.
> Provide a command line entry point to view/test topology options
> ----------------------------------------------------------------
>
> Key: HADOOP-8232
> URL: https://issues.apache.org/jira/browse/HADOOP-8232
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: util
> Affects Versions: 0.23.1
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Attachments: HADOOP-8232.patch
>
>
> Add a new command line entry point "topo" with commands for preflight
> checking of a clusters topology setup.
> The initial operations would be to list the implementation class of the
> mapper, and attempt to load it, resolve a set of supplied hostnames, then
> dump the topology map after the resolution process.
> Target audience:
> # ops teams trying to get a new/changed script working before deploying it on
> a cluster.
> # someone trying to write their first script.
> Resolve and list the rack mappings of the given host
> {code}
> hadoop topo test [host1] [host2] ...
> {code}
> This would load the hostnames from a given file, resolve all of them and list
> the results:
> {code}
> hadoop topo testfile filename
> {code}
> This version is intended for the ops team who have a list of hostnames, IP
> addresses.
> * Rather than just list them, the ops team may want to mandate that there
> were no /default-rack mappings found, as that is invariably a sign that the
> script isn't handling a hostname properly.
> * No attempt to be clever and do IP address resolution, FQDN to hostname
> mapping, etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
