[
https://issues.apache.org/jira/browse/HADOOP-20027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18129796#comment-18129796
]
ASF GitHub Bot commented on HADOOP-20027:
-----------------------------------------
smengcl opened a new pull request, #8850:
URL: https://github.com/apache/hadoop/pull/8850
Generated-by: Claude Code (Opus 5.5)
### Description of PR
`DatanodeID` builds its address strings with `ipAddr + ":" + port`. For a
DataNode with an IPv6 address this gives `2001:db8:0:0:0:0:0:1:9866`, which
`NetUtils.createSocketAddr` rejects (`Does not contain a valid host:port
authority`). `getXferAddr` is what clients use to connect to DataNodes for
reads and writes, so on an IPv6 cluster that uses IP addresses
(`dfs.client.use.datanode.hostname=false`, the default) clients cannot reach
any DataNode. This is a likely cause of HADOOP-17543.
This PR builds the xfer, ipc, info and info secure addresses (both the IP
and the host name variants) with `NetUtils.getHostPortString(host, port)`,
which brackets IPv6 literals. IPv4 and host name output is unchanged. The
NameNode and Router web UIs already split `infoAddr` on `]:`, so they handle
the bracketed form.
Sub-task of HADOOP-11890.
### How was this patch tested?
Added `TestDatanodeID#testAddrStrings`. It checks the exact strings for
IPv4, IPv6 and host names, checks that `setIpAddr` rebuilds the cached xfer
address, and parses every address back with `NetUtils.createSocketAddr`. It
fails on trunk.
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
🤖 Generated with [Claude Code](https://claude.com/claude-code)
> DatanodeID builds unbracketed IPv6 host:port strings
> ----------------------------------------------------
>
> Key: HADOOP-20027
> URL: https://issues.apache.org/jira/browse/HADOOP-20027
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: ipv6
>
> {{DatanodeID}} builds its address strings by plain concatenation:
> {code:java}
> this.xferAddr = ipAddr + ":" + xferPort; // setIpAndXferPort
> return ipAddr + ":" + ipcPort; // getIpcAddr
> return ipAddr + ":" + infoPort; // getInfoAddr
> return ipAddr + ":" + infoSecurePort; // getInfoSecureAddr
> {code}
> [DatanodeID.java:138, 204, 211,
> 218|https://github.com/apache/hadoop/blob/9ede2e95c6fb577d7ae8664833d5e151cbfc47c9/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeID.java#L138]
> For a DataNode with an IPv6 address this gives an ambiguous string such as
> {{2001:db8:0:0:0:0:0:1:9866}} instead of {{[2001:db8:0:0:0:0:0:1]:9866}}.
> {{NetUtils.createSocketAddr}} parses it as the authority of a URI.
> {{java.net.URI}} returns a null host for the unbracketed form, so
> {{createSocketAddr}} throws {{IllegalArgumentException: Does not contain a
> valid host:port authority}}.
> {{getXferAddr}} is what the client uses to reach DataNodes: {{DataStreamer}}
> for writes, {{DFSInputStream}} for reads, {{DFSUtilClient}} and
> {{ClientContext}}. On an IPv6-only cluster that uses IP addresses
> ({{dfs.client.use.datanode.hostname=false}}, the default), clients cannot
> connect to any DataNode. The xfer address is cached in the {{DatanodeID}} and
> sent to clients in every block location, so one bad string affects every read
> and write. The info addresses have the same problem in the web UI and in
> WebHDFS redirects.
> {{getXferAddrWithHostname}} and {{getIpcAddrWithHostname}} (lines 225 and
> 232) use the same pattern. They are only wrong if the host name is itself an
> IPv6 literal, but they should use the same helper.
> *Proposed fix:* build all of these strings with
> {{NetUtils.getHostPortString(host, port)}}, which brackets IPv6 literals
> since HADOOP-19958. IPv4 and host name output is unchanged. Add a unit test
> that round-trips each address through {{NetUtils.createSocketAddr}} for IPv4,
> IPv6 and host names.
> HADOOP-17543 reports an HDFS put failure on an IPv6 cluster with no stack
> trace; this is a likely cause.
> POC:
> https://github.com/smengcl/hadoop/commit/8aeae47d67401180a954f7859ae423acf641477c
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
