[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628230#comment-13628230
]
Colin Patrick McCabe commented on HDFS-3934:
--------------------------------------------
It's true that this does add additional DNS checks for {{dfs.hosts.exclude}}
and {{dfs.hosts}}.
It should be possible to drop the {{FSNamesystem}} lock while these DNS
resolutions are going on.
Alternately, we could move the DNS resolutions into
{{DatanodeManager#refreshHostsReader}}. This would mean that administrators
would have to re-run "{{dfsadmin -refreshNodes}}" if the DNS mapping changed
for a hostname in the include or excludes file.
Finally, we could simply tell people who are concerned about DNS resolution
time to use IP addresses in the include and exclude files. We'd also have to
change the code to avoid doing the reverse DNS lookup in this scenario.
I think I like solution #2 the best; what do you think?
> duplicative dfs_hosts entries handled wrong
> -------------------------------------------
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.0.1-alpha
> Reporter: Andy Isaacson
> Assignee: Andy Isaacson
> Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{<a
> href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010</a>}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
