[ https://issues.apache.org/jira/browse/HDFS-12182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16128545#comment-16128545 ]
Wellington Chevreuil edited comment on HDFS-12182 at 8/16/17 9:28 AM: ---------------------------------------------------------------------- Fixed issue on TestMetaSave. Other tests failed on previous build are passing locally, so maybe it's a timeout issue only? was (Author: wchevreuil): Fixed issue on TestMetaSave. Other tests failed on previous jira are passing locally. > BlockManager.metaSave does not distinguish between "under replicated" and > "missing" blocks > ------------------------------------------------------------------------------------------ > > Key: HDFS-12182 > URL: https://issues.apache.org/jira/browse/HDFS-12182 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs > Reporter: Wellington Chevreuil > Assignee: Wellington Chevreuil > Priority: Trivial > Labels: newbie > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12182.001.patch, HDFS-12182.002.patch, > HDFS-12182.003.patch, HDFS-12182.004.patch, HDFS-12182-branch-2.001.patch, > HDFS-12182-branch-2.002.patch > > > Currently, *BlockManager.metaSave* method (which is called by "-metasave" dfs > CLI command) reports both "under replicated" and "missing" blocks under same > metric *Metasave: Blocks waiting for reconstruction:* as shown on below code > snippet: > {noformat} > synchronized (neededReconstruction) { > out.println("Metasave: Blocks waiting for reconstruction: " > + neededReconstruction.size()); > for (Block block : neededReconstruction) { > dumpBlockMeta(block, out); > } > } > {noformat} > *neededReconstruction* is an instance of *LowRedundancyBlocks*, which > actually wraps 5 priority queues currently. 4 of these queues store different > under replicated scenarios, but the 5th one is dedicated for corrupt/missing > blocks. > Thus, metasave report may suggest some corrupt blocks are just under > replicated. This can be misleading for admins and operators trying to track > block missing/corruption issues, and/or other issues related to > *BlockManager* metrics. > I would like to propose a patch with trivial changes that would report > corrupt blocks separately. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org