[ https://issues.apache.org/jira/browse/HDFS-16287?focusedWorklogId=675615&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-675615 ]
ASF GitHub Bot logged work on HDFS-16287: ----------------------------------------- Author: ASF GitHub Bot Created on: 04/Nov/21 01:08 Start Date: 04/Nov/21 01:08 Worklog Time Spent: 10m Work Description: haiyang1987 commented on a change in pull request #3596: URL: https://github.com/apache/hadoop/pull/3596#discussion_r741696525 ########## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java ########## @@ -260,17 +257,14 @@ final Timer timer = new Timer(); this.slowPeerTracker = dataNodePeerStatsEnabled ? new SlowPeerTracker(conf, timer) : null; - this.excludeSlowNodesEnabled = conf.getBoolean( - DFS_NAMENODE_BLOCKPLACEMENTPOLICY_EXCLUDE_SLOW_NODES_ENABLED_KEY, - DFS_NAMENODE_BLOCKPLACEMENTPOLICY_EXCLUDE_SLOW_NODES_ENABLED_DEFAULT); this.maxSlowPeerReportNodes = conf.getInt( DFSConfigKeys.DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_KEY, DFSConfigKeys.DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_DEFAULT); this.slowPeerCollectionInterval = conf.getTimeDuration( DFSConfigKeys.DFS_NAMENODE_SLOWPEER_COLLECT_INTERVAL_KEY, DFSConfigKeys.DFS_NAMENODE_SLOWPEER_COLLECT_INTERVAL_DEFAULT, TimeUnit.MILLISECONDS); - if (slowPeerTracker != null && excludeSlowNodesEnabled) { Review comment: @tomscut Thank you for your review. 1.Current parameter 'dataNodePeerStatsEnabled' and 'excludeSlowNodesEnabled' decision SlowPeerCollector thread whether to start ,But it didn't take into account avoid SlowDataNodesForRead logic 2.So think about two phases: a.The first is to start SlowPeerCollector thread b.Second, you can control whether to enable read/write avoid slow datanode according to dynamic parameters ########## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java ########## @@ -511,7 +505,16 @@ private boolean isInactive(DatanodeInfo datanode) { private boolean isSlowNode(String dnUuid) { return avoidSlowDataNodesForRead && slowNodesUuidSet.contains(dnUuid); } - + + public void setAvoidSlowDataNodesForReadEnabled(boolean enable) { Review comment: Consider slowNodesUuidSet is generated when the SlowPeerCollector thread is started,therefore it is logical to judge Therefore, it is logical to judge whether the dnUuid exists in the slowNodesUuidSet? ########## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java ########## @@ -511,7 +505,16 @@ private boolean isInactive(DatanodeInfo datanode) { private boolean isSlowNode(String dnUuid) { return avoidSlowDataNodesForRead && slowNodesUuidSet.contains(dnUuid); } - + + public void setAvoidSlowDataNodesForReadEnabled(boolean enable) { Review comment: Consider slowNodesUuidSet is generated when the SlowPeerCollector thread is started,therefore it is logical to judge whether the dnUuid exists in the slowNodesUuidSet? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 675615) Time Spent: 7h 10m (was: 7h) > Support to make dfs.namenode.avoid.read.slow.datanode reconfigurable > --------------------------------------------------------------------- > > Key: HDFS-16287 > URL: https://issues.apache.org/jira/browse/HDFS-16287 > Project: Hadoop HDFS > Issue Type: Improvement > Reporter: Haiyang Hu > Assignee: Haiyang Hu > Priority: Major > Labels: pull-request-available > Time Spent: 7h 10m > Remaining Estimate: 0h > > 1. Consider that make dfs.namenode.avoid.read.slow.datanode reconfigurable > and rapid rollback in case this feature > [HDFS-16076|https://issues.apache.org/jira/browse/HDFS-16076] unexpected > things happen in production environment > 2. DatanodeManager#startSlowPeerCollector by parameter > 'dfs.datanode.peer.stats.enabled' to control -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org