[
https://issues.apache.org/jira/browse/HDFS-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang resolved HDFS-17181.
------------------------------------
Fix Version/s: 3.5.0
Resolution: Fixed
> WebHDFS not considering whether a DN is good when called from outside the
> cluster
> ---------------------------------------------------------------------------------
>
> Key: HDFS-17181
> URL: https://issues.apache.org/jira/browse/HDFS-17181
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode, webhdfs
> Affects Versions: 3.3.6
> Reporter: Lars Francke
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.5.0
>
> Attachments: Test_fix_for_HDFS-171811.patch
>
>
> When calling WebHDFS to create a file (I'm sure the same problem occurs for
> other actions e.g. OPEN but I haven't checked all of them yet) it will
> happily redirect to nodes that are in maintenance.
> The reason is in the
> [{{chooseDatanode}}|https://github.com/apache/hadoop/blob/rel/release-3.3.6/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java#L307C9-L315]
> method in {{NamenodeWebHdfsMethods}} where it will only call the
> {{BlockPlacementPolicy}} (which considers all these edge cases) in case the
> {{remoteAddr}} (i.e. the address making the request to WebHDFS) is also
> running a DataNode.
>
> In all other cases it just refers to
> [{{NetworkTopology#chooseRandom}}|https://github.com/apache/hadoop/blob/rel/release-3.3.6/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java#L342-L343]
> which does not consider any of these circumstances (e.g. load, maintenance).
> I don't understand the reason to not just always refer to the placement
> policy and we're currently testing a patch to do just that.
> I have attached a draft patch for now.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
