[
https://issues.apache.org/jira/browse/HDFS-17260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17792331#comment-17792331
]
ASF GitHub Bot commented on HDFS-17260:
---------------------------------------
huangzhaobo99 commented on code in PR #6279:
URL: https://github.com/apache/hadoop/pull/6279#discussion_r1412732623
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/SlowPeerDisabledTracker.java:
##########
@@ -46,11 +46,6 @@ public class SlowPeerDisabledTracker extends SlowPeerTracker
{
public SlowPeerDisabledTracker(Configuration conf, Timer timer) {
super(conf, timer);
- final boolean dataNodePeerStatsEnabledVal =
- conf.getBoolean(DFSConfigKeys.DFS_DATANODE_PEER_STATS_ENABLED_KEY,
- DFSConfigKeys.DFS_DATANODE_PEER_STATS_ENABLED_DEFAULT);
- Preconditions.checkArgument(!dataNodePeerStatsEnabledVal,
- "SlowPeerDisabledTracker should only be used for disabled slow peer
stats.");
Review Comment:
1. The logic here may not be correct because the system is now changing the
value of the key from true to false, while the key of getConf is the old key,
throw IllegalArgumentException in Preconditions.checkArgument.
<img width="1100" alt="image"
src="https://github.com/apache/hadoop/assets/63718681/613e610f-3999-4960-9adb-0b9fed18dcdf";>
---<img width="944" alt="image"
src="https://github.com/apache/hadoop/assets/63718681/7495c5e2-c8d4-479f-9f3b-4f4eda5d7787";>
2. It does not affect reconfig DFS_ DATANODE_ PEER_ STATS_ ENABLED_ KEY's
logic
3. TestNameNodeReconfigure#testSlowPeerTrackerEnabled UT skipped the logic
of refreshing the configuration, so when true is changed to false, the original
value in the system remains false. Therefore, the following code passed the UT
<img width="1181" alt="image"
src="https://github.com/apache/hadoop/assets/63718681/8772ae17-6da0-49e9-8b83-de7da84b63e2";>
4. When to refresh the configuration, refer to the following
code(ReconfigurableBase#reconfigureProperty)
<img width="941" alt="image"
src="https://github.com/apache/hadoop/assets/63718681/e3dd7d4e-1649-472f-8d68-2f0fc436a8e3";>
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyExcludeSlowNodes.java:
##########
@@ -136,4 +138,41 @@ public void testChooseTargetExcludeSlowNodes() throws
Exception {
NameNode.LOG.info("Done working on it");
}
+ @Test
+ public void testSlowPeerTrackerEnabledClearSlowNodes() throws Exception {
+ namenode.getNamesystem().writeLock();
+ try {
+ // add nodes
+ for (DatanodeDescriptor dataNode : dataNodes) {
+ dnManager.addDatanode(dataNode);
+ }
+
+ // mock slow nodes
+ SlowPeerTracker tracker = dnManager.getSlowPeerTracker();
+ assert tracker != null;
+ OutlierMetrics outlierMetrics = new OutlierMetrics(0.0, 0.0, 0.0, 5.0);
+ tracker.addReport(dataNodes[0].getInfoAddr(), dataNodes[3].getInfoAddr(),
+ outlierMetrics);
+ tracker.addReport(dataNodes[1].getInfoAddr(), dataNodes[3].getInfoAddr(),
+ outlierMetrics);
+ tracker.addReport(dataNodes[2].getInfoAddr(), dataNodes[3].getInfoAddr(),
+ outlierMetrics);
+
+ // waiting for slow nodes collector run
+ Thread.sleep(3000);
Review Comment:
fix
> Fix the logic for reconfigure slow peer enable for Namenode.
> ------------------------------------------------------------
>
> Key: HDFS-17260
> URL: https://issues.apache.org/jira/browse/HDFS-17260
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: huangzhaobo99
> Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
