[
https://issues.apache.org/jira/browse/HADOOP-985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12471003
]
Raghu Angadi commented on HADOOP-985:
-------------------------------------
I prefer #2 as well. This could be the default behavior and if
dfs.datanode.dns.interface is specified, then we can use the ip of the specific
interface (this might be required for some special cases).
Instead of modifying RPC so that namenode sees remote ip for this case,
datanode can report the ip and hostname. Datanode can open a UDP socket to
namenode and check the local ip of the socket. I think it does not even need to
send any packets. Either case, it does not need namenode to be up or wait for
namenode response.
Datanode can resolve the ip for hostname. This won't always match 'hostname
-f'.. I will check how exactly we currently get the hostname.
> Namenode should identify DataNodes as ip:port instead of hostname:port
> ----------------------------------------------------------------------
>
> Key: HADOOP-985
> URL: https://issues.apache.org/jira/browse/HADOOP-985
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.11.0
> Reporter: Raghu Angadi
> Assigned To: Raghu Angadi
> Fix For: 0.12.0
>
>
> Right now NameNode keeps track of DataNodes with "hostname:port". One
> proposal is to keep track of datanodes with "ip:port". There are various
> concerns expressed regd hostnames and ip. Please add your experiences here so
> that we have better idea on what we should fix etc.
> How should be calculate datanode ip:
> 1) Just like how we calculate hostname currently with
> "dfs.datanode.dns.interface" and "dfs.datanode.dns.nameserver". So if
> interface specified wrong, it could report ip like 127.0.0.1 which might or
> might not be intended.
> 2) Namenode can use the remove socket address when the datanode
> registers. Not sure how easy it to get this address in RPC or if this is
> desirable.
> 3) Namenode could just resolve the hostname when a datanode
> registers. It could print of a warning if the resolved ip and reported ip
> don't match.
> One advantage of using IPs is that DFSClient does not need to resolve them
> when it connects to datanode. This could save few milliseconds for each
> block. Also, DFSClient should check all its ips to see if a given ip is local
> or not.
> As far I see namenode does not resolve any DNS in normal operations since it
> does not actively contact datanodes. In that sense not sure if this have any
> change in Namenode performance.
> Thoughts?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
