[
https://issues.apache.org/jira/browse/HDFS-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoqiao He updated HDFS-17891:
-------------------------------
Fix Version/s: 3.6.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
> HostSet.add() throws uncaught exception if DataNode hostname is not resolved
> ----------------------------------------------------------------------------
>
> Key: HDFS-17891
> URL: https://issues.apache.org/jira/browse/HDFS-17891
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.4.3
> Reporter: Jim Halfpenny
> Assignee: Jim Halfpenny
> Priority: Minor
> Labels: pull-request-available
> Fix For: 3.6.0
>
>
> The HostSet.add() method that adds DataNode addresses to a HostSet checks to
> see if the hostname resolves. If this test fails then
> Preconditions.checkArgument() throws an uncaught IllegalArgumentException:
> {{ void add(InetSocketAddress addr) {}}
> {{ Preconditions.checkArgument(!addr.isUnresolved());}}
> {{ addrs.put(addr.getAddress(), addr.getPort());}}
> It would make more sense log the failure and return from the add() method
> rather than cause the creation of the HostSet to fail entirely. An example of
> when this issue manifests is when running `hdfs dfsadmin -report` on a
> cluster where one of the DataNode addresses cannot be resolved. The DataNode
> report fails with an IllegalArgumentException:
> {{stackable@simple-hdfs-namenode-default-0 /stackable/hadoop-3.4.0 $ hdfs
> dfsadmin -report}}
> {{Configured Capacity: 2933688373248 (2.67 TB)}}
> {{Present Capacity: 1299099013120 (1.18 TB)}}
> {{DFS Remaining: 1299098914816 (1.18 TB)}}
> {{DFS Used: 98304 (96 KB)}}
> {{DFS Used%: 0.00%}}
> {{Replicated Blocks:}}
> {{ Under replicated blocks: 0}}
> {{ Blocks with corrupt replicas: 0}}
> {{ Missing blocks: 0}}
> {{ Missing blocks (with replication factor 1): 0}}
> {{ Low redundancy blocks with highest priority to recover: 0}}
> {{ Pending deletion blocks: 0}}
> {{Erasure Coded Block Groups:}}
> {{ Low redundancy block groups: 0}}
> {{ Block groups with corrupt internal blocks: 0}}
> {{ Missing block groups: 0}}
> {{ Low redundancy blocks with highest priority to recover: 0}}
> {{ Pending deletion blocks: 0}}
> {{-------------------------------------------------}}
> {{report: java.lang.IllegalArgumentException}}
> The chance of a DataNode address not resolving is much more likely when HDFS
> is being run in Kubernetes; if a DataNode pod is removed the corresponding
> DNS entry will also be removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
