[ https://issues.apache.org/jira/browse/HDFS-17446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17832782#comment-17832782 ]
ASF GitHub Bot commented on HDFS-17446: --------------------------------------- hiwangzhihui commented on code in PR #6685: URL: https://github.com/apache/hadoop/pull/6685#discussion_r1546135795 ########## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java: ########## @@ -749,6 +749,8 @@ void addToReplicasMap(ReplicaMap volumeMap, File dir, lazyWriteReplicaMap, isFinalized, exceptions, subTaskQueue); subTask.fork(); subTaskQueue.add(subTask); + } else if (isFinalized && ++scanBlockCnt % 10000 == 0) { + LOG.info((scanBlockCnt - 1) + " blocks (finalize state) have been scanned."); Review Comment: Please take care of concurrency issues that lead to inaccurate counting. ########## hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java: ########## @@ -749,6 +749,8 @@ void addToReplicasMap(ReplicaMap volumeMap, File dir, lazyWriteReplicaMap, isFinalized, exceptions, subTaskQueue); subTask.fork(); subTaskQueue.add(subTask); + } else if (isFinalized && ++scanBlockCnt % 10000 == 0) { + LOG.info((scanBlockCnt - 1) + " blocks (finalize state) have been scanned."); Review Comment: The log style is not easy to view. > The DataNode adds a log to report its progress during addToReplicasMap > execution. > --------------------------------------------------------------------------------- > > Key: HDFS-17446 > URL: https://issues.apache.org/jira/browse/HDFS-17446 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs > Reporter: xiaojunxiang > Priority: Critical > Labels: pull-request-available > Attachments: image-2024-03-28-23-53-29-057.png > > > Datanodes do not print logs when addToReplicasMap is just started, so we will > not be aware of its progress, or even wait up to 4 hours for a large cluster. > This is very confusing. > A cluster is as follows: > !image-2024-03-28-23-53-29-057.png! -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org