[ https://issues.apache.org/jira/browse/HDFS-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17814207#comment-17814207 ]
Kanaka Kumar Avvaru commented on HDFS-17181: -------------------------------------------- Can someone take it forward with PR review and assert if the test failures related? > WebHDFS not considering whether a DN is good when called from outside the > cluster > --------------------------------------------------------------------------------- > > Key: HDFS-17181 > URL: https://issues.apache.org/jira/browse/HDFS-17181 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode, webhdfs > Affects Versions: 3.3.6 > Reporter: Lars Francke > Priority: Major > Labels: pull-request-available > Attachments: Test_fix_for_HDFS-171811.patch > > > When calling WebHDFS to create a file (I'm sure the same problem occurs for > other actions e.g. OPEN but I haven't checked all of them yet) it will > happily redirect to nodes that are in maintenance. > The reason is in the > [{{chooseDatanode}}|https://github.com/apache/hadoop/blob/rel/release-3.3.6/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java#L307C9-L315] > method in {{NamenodeWebHdfsMethods}} where it will only call the > {{BlockPlacementPolicy}} (which considers all these edge cases) in case the > {{remoteAddr}} (i.e. the address making the request to WebHDFS) is also > running a DataNode. > > In all other cases it just refers to > [{{NetworkTopology#chooseRandom}}|https://github.com/apache/hadoop/blob/rel/release-3.3.6/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java#L342-L343] > which does not consider any of these circumstances (e.g. load, maintenance). > I don't understand the reason to not just always refer to the placement > policy and we're currently testing a patch to do just that. > I have attached a draft patch for now. > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org