[ https://issues.apache.org/jira/browse/HDFS-10941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637538#comment-15637538 ]
Chen Liang commented on HDFS-10941: ----------------------------------- The failed tests seem unrelated. Local tests never had {{TestEncryptionZones.testStartFileRetry}} failed. And the other three tests randomly fail either with or without the patch so it appears the tests themselves are flaky. > Improve BlockManager#processMisReplicatesAsync log > -------------------------------------------------- > > Key: HDFS-10941 > URL: https://issues.apache.org/jira/browse/HDFS-10941 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode > Reporter: Xiaoyu Yao > Assignee: Chen Liang > Attachments: HDFS-10941.001.patch, HDFS-10941.002.patch, > HDFS-10941.002.patch > > > BlockManager#processMisReplicatesAsync is the daemon thread running inside > namenode to handle miserplicated blocks. As shown below, it has a trace log > for each of the block in the cluster being processed (10000 blocks per > iteration after sleep 10s). > {code} > MisReplicationResult res = processMisReplicatedBlock(block); > if (LOG.isTraceEnabled()) { > LOG.trace("block " + block + ": " + res); > } > {code} > However, it is not very useful as dumping every block in the cluster will > overwhelm the namenode log without much useful information assuming the > majority of the blocks are not over/under replicated. This ticket is opened > to improve the log for easy troubleshooting of block replication related > issues by: > > 1) add debug log for blocks that get under/over replicated result during > {{processMisReplicatedBlock()}} > 2) or change to trace log for only blocks that get non-OK result during > {{processMisReplicatedBlock()}} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org