[jira] [Updated] (HDFS-11032) [SPS]: Handling of block movement failure at the coordinator datanode
[ https://issues.apache.org/jira/browse/HDFS-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-11032: Fix Version/s: 3.2.0 > [SPS]: Handling of block movement failure at the coordinator datanode > - > > Key: HDFS-11032 > URL: https://issues.apache.org/jira/browse/HDFS-11032 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Rakesh R >Assignee: Rakesh R >Priority: Major > Fix For: HDFS-10285, 3.2.0 > > Attachments: HDFS-11032-HDFS-10285-00.patch, > HDFS-11032-HDFS-10285-01.patch > > > The idea of this jira is to discuss and implement an efficient failure(block > movement failure) handling logic at the datanode cooridnator. [Code > reference|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StoragePolicySatisfyWorker.java#L243]. > Following are the possible errors during block movement: > # Block pinned - no retries marked as success/no-retry to NN. It is not > possible to relocate this block to another datanode. > # Network errors(IOException) - no retries maked as failure/retry to NN. > # No disk space(IOException) - no retries maked as failure/retry to NN. > # Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a > case that the file might have re-opened. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11032) [SPS]: Handling of block movement failure at the coordinator datanode
[ https://issues.apache.org/jira/browse/HDFS-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-11032: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-10285 Status: Resolved (was: Patch Available) I have just pushed this to branch > [SPS]: Handling of block movement failure at the coordinator datanode > - > > Key: HDFS-11032 > URL: https://issues.apache.org/jira/browse/HDFS-11032 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Rakesh R >Assignee: Rakesh R > Fix For: HDFS-10285 > > Attachments: HDFS-11032-HDFS-10285-00.patch, > HDFS-11032-HDFS-10285-01.patch > > > The idea of this jira is to discuss and implement an efficient failure(block > movement failure) handling logic at the datanode cooridnator. [Code > reference|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StoragePolicySatisfyWorker.java#L243]. > Following are the possible errors during block movement: > # Block pinned - no retries marked as success/no-retry to NN. It is not > possible to relocate this block to another datanode. > # Network errors(IOException) - no retries maked as failure/retry to NN. > # No disk space(IOException) - no retries maked as failure/retry to NN. > # Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a > case that the file might have re-opened. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11032) [SPS]: Handling of block movement failure at the coordinator datanode
[ https://issues.apache.org/jira/browse/HDFS-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-11032: Attachment: HDFS-11032-HDFS-10285-01.patch Attached new patch fixing checkstyle warnings. > [SPS]: Handling of block movement failure at the coordinator datanode > - > > Key: HDFS-11032 > URL: https://issues.apache.org/jira/browse/HDFS-11032 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Rakesh R >Assignee: Rakesh R > Attachments: HDFS-11032-HDFS-10285-00.patch, > HDFS-11032-HDFS-10285-01.patch > > > The idea of this jira is to discuss and implement an efficient failure(block > movement failure) handling logic at the datanode cooridnator. [Code > reference|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StoragePolicySatisfyWorker.java#L243]. > Following are the possible errors during block movement: > # Block pinned - no retries marked as success/no-retry to NN. It is not > possible to relocate this block to another datanode. > # Network errors(IOException) - no retries maked as failure/retry to NN. > # No disk space(IOException) - no retries maked as failure/retry to NN. > # Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a > case that the file might have re-opened. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11032) [SPS]: Handling of block movement failure at the coordinator datanode
[ https://issues.apache.org/jira/browse/HDFS-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-11032: Description: The idea of this jira is to discuss and implement an efficient failure(block movement failure) handling logic at the datanode cooridnator. [Code reference|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StoragePolicySatisfyWorker.java#L243]. Following are the possible errors during block movement: # Block pinned - no retries marked as success/no-retry to NN. It is not possible to relocate this block to another datanode. # Network errors(IOException) - no retries maked as failure/retry to NN. # No disk space(IOException) - no retries maked as failure/retry to NN. # Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a case that the file might have re-opened. was: The idea of this jira is to discuss and implement an efficient failure(block movement failure) handling logic at the datanode cooridnator. [Code reference|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StoragePolicySatisfyWorker.java#L243]. Following are the possible errors during block movement: # Network errors(IOException) - provide retries(may be a hard coded 2 time retries) if the block storage movement is failed due to network errors. If its still end up with errors after 2 retries then marked as failure/retry to NN. # No disk space(IOException) - no retries maked as failure/retry to NN. # Block pinned - no retries marked as success/no-retry to NN. It is not possible to relocate this block to another datanode. # Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a case that the file might have re-opened. > [SPS]: Handling of block movement failure at the coordinator datanode > - > > Key: HDFS-11032 > URL: https://issues.apache.org/jira/browse/HDFS-11032 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Rakesh R >Assignee: Rakesh R > Attachments: HDFS-11032-HDFS-10285-00.patch > > > The idea of this jira is to discuss and implement an efficient failure(block > movement failure) handling logic at the datanode cooridnator. [Code > reference|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StoragePolicySatisfyWorker.java#L243]. > Following are the possible errors during block movement: > # Block pinned - no retries marked as success/no-retry to NN. It is not > possible to relocate this block to another datanode. > # Network errors(IOException) - no retries maked as failure/retry to NN. > # No disk space(IOException) - no retries maked as failure/retry to NN. > # Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a > case that the file might have re-opened. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11032) [SPS]: Handling of block movement failure at the coordinator datanode
[ https://issues.apache.org/jira/browse/HDFS-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-11032: Status: Patch Available (was: Open) > [SPS]: Handling of block movement failure at the coordinator datanode > - > > Key: HDFS-11032 > URL: https://issues.apache.org/jira/browse/HDFS-11032 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Rakesh R >Assignee: Rakesh R > Attachments: HDFS-11032-HDFS-10285-00.patch > > > The idea of this jira is to discuss and implement an efficient failure(block > movement failure) handling logic at the datanode cooridnator. [Code > reference|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StoragePolicySatisfyWorker.java#L243]. > Following are the possible errors during block movement: > # Network errors(IOException) - provide retries(may be a hard coded 2 time > retries) if the block storage movement is failed due to network errors. If > its still end up with errors after 2 retries then marked as failure/retry to > NN. > # No disk space(IOException) - no retries maked as failure/retry to NN. > # Block pinned - no retries marked as success/no-retry to NN. It is not > possible to relocate this block to another datanode. > # Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a > case that the file might have re-opened. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11032) [SPS]: Handling of block movement failure at the coordinator datanode
[ https://issues.apache.org/jira/browse/HDFS-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-11032: Attachment: HDFS-11032-HDFS-10285-00.patch > [SPS]: Handling of block movement failure at the coordinator datanode > - > > Key: HDFS-11032 > URL: https://issues.apache.org/jira/browse/HDFS-11032 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Rakesh R >Assignee: Rakesh R > Attachments: HDFS-11032-HDFS-10285-00.patch > > > The idea of this jira is to discuss and implement an efficient failure(block > movement failure) handling logic at the datanode cooridnator. [Code > reference|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StoragePolicySatisfyWorker.java#L243]. > Following are the possible errors during block movement: > # Network errors(IOException) - provide retries(may be a hard coded 2 time > retries) if the block storage movement is failed due to network errors. If > its still end up with errors after 2 retries then marked as failure/retry to > NN. > # No disk space(IOException) - no retries maked as failure/retry to NN. > # Block pinned - no retries marked as success/no-retry to NN. It is not > possible to relocate this block to another datanode. > # Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a > case that the file might have re-opened. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11032) [SPS]: Handling of block movement failure at the coordinator datanode
[ https://issues.apache.org/jira/browse/HDFS-11032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-11032: Component/s: (was: namenode) > [SPS]: Handling of block movement failure at the coordinator datanode > - > > Key: HDFS-11032 > URL: https://issues.apache.org/jira/browse/HDFS-11032 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode >Reporter: Rakesh R >Assignee: Rakesh R > > The idea of this jira is to discuss and implement an efficient failure(block > movement failure) handling logic at the datanode cooridnator. [Code > reference|https://github.com/apache/hadoop/blob/HDFS-10285/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/StoragePolicySatisfyWorker.java#L243]. > Following are the possible errors during block movement: > # Network errors(IOException) - provide retries(may be a hard coded 2 time > retries) if the block storage movement is failed due to network errors. If > its still end up with errors after 2 retries then marked as failure/retry to > NN. > # No disk space(IOException) - no retries maked as failure/retry to NN. > # Block pinned - no retries marked as success/no-retry to NN. It is not > possible to relocate this block to another datanode. > # Gen_Stamp mismatches - no retries marked as failure/retry to NN. Could be a > case that the file might have re-opened. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
