[ https://issues.apache.org/jira/browse/HDFS-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15578574#comment-15578574 ]
Allen Wittenauer commented on HDFS-10999: ----------------------------------------- bq. That's what I was getting at with the pendingReconstructionBlocksCount. If we fix it as I talked about above, it'd actually tell you how much work is remaining, and how fast that work is progressing. That might work, but I just had a thought. Are we exposing how many blocks are EC blocks and how many blocks are normally replicated blocks? (If not, I really hope the explanation is a good one...) It seems that we should have symmetry here. If we have N types of blocks, I'm going to want to know NxM counts of information. It's pretty much the only way that advanced users will know if certain types of blocks are actually working to their benefit. Like compression, space savings isn't the only consideration. bq. I really, really hope that manually copying blocks around is not a normal part of operating an HDFS cluster. ... bq. I recall seeing some customer issues where we temporarily bumped up these values to more quickly recover from failures. You've sort of answered your own question. ;) Most of the advanced admins I know do it several times a year, either because the NN was too stupid to fix it's own replication problems and/or because it was simply faster for us to do it rather than wait for the normal block replication process. For example, as an admin, I might know that there is no YARN running on a source node or the destination node, so it's totally OK to do a brute copy from one DN to another other without busting the network. HDFS block deletes are significantly faster than replication, so just do the copy, run the balancer, and let the NN remove the duplicates at it's leisure. All without fumbling with the continually ever growing and poorly documented HDFS settings. > Use more generic "low redundancy" blocks instead of "under replicated" blocks > ----------------------------------------------------------------------------- > > Key: HDFS-10999 > URL: https://issues.apache.org/jira/browse/HDFS-10999 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: erasure-coding > Affects Versions: 3.0.0-alpha1 > Reporter: Wei-Chiu Chuang > Assignee: Yuanbo Liu > Labels: supportability > > Per HDFS-9857, it seems in the Hadoop 3 world, people prefer the more generic > term "low redundancy" to the old-fashioned "under replicated". But this term > is still being used in messages in several places, such as web ui, dfsadmin > and fsck. We should probably change them to avoid confusion. > File this jira to discuss it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org