[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.001.patch
This patch resolves the entries in the hosts files to their first IP addresses
before de-duplicating everything. when creating a new {{DatanodeInfo}} for
them, it uses {{getCanonicalAddress}}.
This fixes the "invisible node" problem where due to lacking a hostname, the
NameNode web UI would show an entry like {{:50010}} in its lists (note missing
hostname).
It also fixes the problem where we put two hostnames which refer to the same IP
address in a host file, or a hostname and an IP which both turn out to map to
the same hostname.
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Andy Isaacson
>Priority: Minor
> Attachments: HDFS-3934.001.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.002.patch
* fix test that relies on unresolvable hostname
* fix behavior for hosts files that have lines of the form :
rather than just
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Andy Isaacson
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Status: Patch Available (was: Open)
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Andy Isaacson
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.003.patch
* add a test for duplicate hosts entries.
* Add test timeouts for tests in {{TestDecommission}}.
* In {{DatanodeManager#getDatanodeListForReport}}, de-dupe by ip address +
port, rather than by ip address alone. (since multiple DNs can run on the same
node with different ports).
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Andy Isaacson
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.004.patch
when the port is not specified in the hosts file, use a default from the
configuration, rather than a compile-time default.
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Andy Isaacson
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.005.patch
here is a new version that is smarter about DNS resolution. it resolves the
DNS only during the refreshNodes operation.
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Andy Isaacson
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.006.patch
* use immutablemap
* reduce synchronization
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Andy Isaacson
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.007.patch
* add license header
* use a HashMap rather than ImmutableMap.Builder, to handle duplicates
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Andy Isaacson
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.008.patch
* If the port is missing, treat the include/exclude entry as covering any
Datanode on the host.
* grammar fix in comment.
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.010.patch
bq. It'd be nice to continue to use the HostsFileReader and post-process the
result. Otherwise it's a consistency/maintenance to copy-n-paste any new
parsing functionality.
OK, I'll use the {{HostsFileReader}} parsing code.
bq. Why does the reader need to instantiate dummy DatanodeID?
You're right. Re-using {{DatanodeID}} for this purpose doesn't reall ymake
sense. I created a new type called {{HostFileManager#Entry}} to represent host
file entries.
bq. It appears to be for repeatedly making the somewhat fragile assumption that
xferAddr is ipAddr+port? If that relationship changes, we've got a problem...
Fixed to use getIpAddr() + ":" + getXferPort() in all cases.
bq. Patch appears to have dropped supported for the node's registration name.
Eli Collins wanted me to maintain that feature in HDFS-3990. If we need to keep
it, doing a lookup and a canonical lookup (can trigger another dns lookup)
isn't compatible with supporting the reg name.
Thanks for pointing this out. I talked to Eli and he explained the distinction
between registration names and hostnames to me. I added back support for
"registration names" and added a unit test to ensure this works properly.
bq. Doing a lookup followed by getCanonicalName is a bad idea. It does 2 more
lookups: hostname -> PTR -> A so it can resolve CNAMES to IP to hostname. With
this change I think it will cause 3 lookups per host.
One key feature of this change is that all the lookups happen when the include
and exclude files are read. *No* lookups happen during
{{DatanodeManager#getDatanodeListForReport}}, or any of the other cases where
we check the host file entries.
On the advice of Eli, I removed the call to {{getCanonicalName}}. We can just
use the name the user specified in the hosts file; that should be fine.
bq. Question about "// If no transfer port was specified, we take a guess". Why
needed, and what are the ramifications for getting this wrong? Just a display
issue?
We just don't have the information. If the datanode is dead, we only know what
the entry says in the hosts file(s). If the entries don't have the port, we
have to guess. I don't see any way around this. It might be more elegant if
the web UI could understand the concept of "port is unknown," but adding that
seems out of scope.
In addition to the unit tests, I did some manual testing on this and verified
that it got rid of the double-counting of nodes in the web UI for me.
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.011.patch
rebase on trunk
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.012.patch
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.013.patch
fix javadoc warning, etc
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: (was: HDFS-3934.013.patch)
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.013.patch
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch,
> HDFS-3934.013.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.014.patch
removed some stray log messages
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch,
> HDFS-3934.013.patch, HDFS-3934.014.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.015.patch
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch,
> HDFS-3934.013.patch, HDFS-3934.014.patch, HDFS-3934.015.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.016.patch
* Fix a place in {{TestDatanodeRegistration}} where we created a mock datanode
that returned 0 for its xferPort. Since we're asserting on this now, we need
to return a legal port value here.
* Looks like {{DFS_DATANODE_IPC_ADDRESS_DEFAULT}} has been wrong all along (the
colon is missing), but we never noticed because we never tried to parse it as a
host:port until now. So let's fix {{DFS_DATANODE_IPC_ADDRESS_DEFAULT}}.
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch,
> HDFS-3934.013.patch, HDFS-3934.014.patch, HDFS-3934.015.patch,
> HDFS-3934.016.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Attachment: HDFS-3934.017.patch
rebased on trunk
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch,
> HDFS-3934.013.patch, HDFS-3934.014.patch, HDFS-3934.015.patch,
> HDFS-3934.016.patch, HDFS-3934.017.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Fix Version/s: 3.0.0
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Fix For: 3.0.0
>
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch,
> HDFS-3934.013.patch, HDFS-3934.014.patch, HDFS-3934.015.patch,
> HDFS-3934.016.patch, HDFS-3934.017.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Fix Version/s: 2.3.0
2.1.0-beta
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Fix For: 3.0.0, 2.1.0-beta, 2.3.0
>
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch,
> HDFS-3934.013.patch, HDFS-3934.014.patch, HDFS-3934.015.patch,
> HDFS-3934.016.patch, HDFS-3934.017.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3934) duplicative dfs_hosts entries handled wrong
[
https://issues.apache.org/jira/browse/HDFS-3934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-3934:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
committed to branch-2.1-beta and branch-2
> duplicative dfs_hosts entries handled wrong
> ---
>
> Key: HDFS-3934
> URL: https://issues.apache.org/jira/browse/HDFS-3934
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.1-alpha
>Reporter: Andy Isaacson
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Fix For: 3.0.0, 2.1.0-beta, 2.3.0
>
> Attachments: HDFS-3934.001.patch, HDFS-3934.002.patch,
> HDFS-3934.003.patch, HDFS-3934.004.patch, HDFS-3934.005.patch,
> HDFS-3934.006.patch, HDFS-3934.007.patch, HDFS-3934.008.patch,
> HDFS-3934.010.patch, HDFS-3934.011.patch, HDFS-3934.012.patch,
> HDFS-3934.013.patch, HDFS-3934.014.patch, HDFS-3934.015.patch,
> HDFS-3934.016.patch, HDFS-3934.017.patch
>
>
> A dead DN listed in dfs_hosts_allow.txt by IP and in dfs_hosts_exclude.txt by
> hostname ends up being displayed twice in {{dfsnodelist.jsp?whatNodes=DEAD}}
> after the NN restarts because {{getDatanodeListForReport}} does not handle
> such a "pseudo-duplicate" correctly:
> # the "Remove any nodes we know about from the map" loop no longer has the
> knowledge to remove the spurious entries
> # the "The remaining nodes are ones that are referenced by the hosts files"
> loop does not do hostname lookups, so does not know that the IP and hostname
> refer to the same host.
> Relatedly, such an IP-based dfs_hosts entry results in a cosmetic problem in
> the JSP output: The *Node* column shows ":50010" as the nodename, with HTML
> markup {{ href="http://:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F&nnaddr=172.29.97.196:8020";
> title="172.29.97.216:50010">:50010}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
