[ https://issues.apache.org/jira/browse/HDFS-16985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chengwei Wang updated HDFS-16985: --------------------------------- Description: We encounterd several missing-block problem in our production cluster which hdfs running on AWS EC2 + EBS. The root cause: # the block remains only 1 replication left and hasn't been reconstruction # DN checks block file existing when BlockSender construction # the EBS checking failed and throw FileNotFoundException (EBS may be in fault condition) # DN invalidateBlock and schedule block async deletion # EBS already back to normal when DN do delete block # the block file be delete permanently and can't be recovered was: We encounterd several missing-block problem in our production cluster which hdfs running on AWS EC2 + EBS. The root cause: # the block remains only 1 replication and hasn't do # check block file existing when BlockSender construction # the EBS checking failed and throw FileNotFoundException # DN invalidateBlock and schedule delete block async > delete local block file when FileNotFoundException occurred may lead to > missing block. > -------------------------------------------------------------------------------------- > > Key: HDFS-16985 > URL: https://issues.apache.org/jira/browse/HDFS-16985 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode > Reporter: Chengwei Wang > Assignee: Chengwei Wang > Priority: Major > > We encounterd several missing-block problem in our production cluster which > hdfs running on AWS EC2 + EBS. > The root cause: > # the block remains only 1 replication left and hasn't been reconstruction > # DN checks block file existing when BlockSender construction > # the EBS checking failed and throw FileNotFoundException (EBS may be in > fault condition) > # DN invalidateBlock and schedule block async deletion > # EBS already back to normal when DN do delete block > # the block file be delete permanently and can't be recovered -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org