[
https://issues.apache.org/jira/browse/HADOOP-20025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HADOOP-20025:
-------------------------------------
Parent: (was: HADOOP-11890)
Issue Type: Bug (was: Sub-task)
> Block pool ID generated on an IPv6 NameNode contains colons
> -----------------------------------------------------------
>
> Key: HADOOP-20025
> URL: https://issues.apache.org/jira/browse/HADOOP-20025
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: ipv6
>
> {{NNStorage.newBlockPoolID}} puts the NameNode's IP address into the block
> pool ID:
> {code:java}
> return "BP-" + rand + "-"+ ip + "-" + Time.now();
> {code}
> [NNStorage.java:1042|https://github.com/apache/hadoop/blob/9ede2e95c6fb577d7ae8664833d5e151cbfc47c9/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java#L1042]
> When {{DNS.getDefaultIP}} returns an IPv6 address, the ID contains colons,
> for example {{BP-1234-fd00:dead:beef:0:0:0:0:10-1760000000000}}. The ID is
> used as a directory name in every DataNode volume and inside local file
> names. On the IPv6 docker-compose test cluster, DataNode storage
> initialization failed with {{URISyntaxException: Relative path in absolute
> URI}} when {{ChecksumFileSystem}} built a {{.crc}} sidecar path from such a
> name. Colons in directory names also cannot exist on Windows.
> *Proposed fix:* when the IP is IPv6, replace {{:}} with {{_}} and drop any
> scope ID before building the ID. IPv4 block pool IDs are unchanged, and
> existing block pool IDs are not touched, because the ID is generated only
> when a namespace is formatted.
> The general problem of colons in local file names under
> {{ChecksumFileSystem}} is HADOOP-17845.
> POC:
> https://github.com/smengcl/hadoop/commit/a2fbda753b05be8f20ba263451a6ecd68e966605
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
