[jira] [Created] (HADOOP-12334) Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries
Gaurav Kanade created HADOOP-12334: -- Summary: Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries Key: HADOOP-12334 URL: https://issues.apache.org/jira/browse/HADOOP-12334 Project: Hadoop Common Issue Type: Improvement Components: tools Reporter: Gaurav Kanade Assignee: Gaurav Kanade HADOOP-11693 mitigated the problem of HMaster aborting regionserver due to Azure Storage Throttling event during HBase WAL archival. The way this was achieved was by applying an intensive exponential retry when throttling occurred. As a second level of mitigation we will change the mode of copy operation if the operation fails even after all retries -i.e. we will do a client side copy of the blob and then copy it back to destination. This operation will not be subject to throttling and hence should provide a stronger mitigation. However it is more expensive, hence we do it only in the case we fail after all retries -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12484) Single File Rename Throws Incorrectly In Potential Race Condition Scenarios
Gaurav Kanade created HADOOP-12484: -- Summary: Single File Rename Throws Incorrectly In Potential Race Condition Scenarios Key: HADOOP-12484 URL: https://issues.apache.org/jira/browse/HADOOP-12484 Project: Hadoop Common Issue Type: Bug Components: tools Reporter: Gaurav Kanade -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12508) delete fails with exception when lease is held on blob
Gaurav Kanade created HADOOP-12508: -- Summary: delete fails with exception when lease is held on blob Key: HADOOP-12508 URL: https://issues.apache.org/jira/browse/HADOOP-12508 Project: Hadoop Common Issue Type: Bug Reporter: Gaurav Kanade -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-12634) Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy
Gaurav Kanade created HADOOP-12634: -- Summary: Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy Key: HADOOP-12634 URL: https://issues.apache.org/jira/browse/HADOOP-12634 Project: Hadoop Common Issue Type: Bug Environment: HADOOP-12334 changed mode of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries. This was via client side copy. However a process crash when the copy is partially done would result in a scenario where the source and destination blobs will have different contents and lazy rename pending operation will not handle this thus causing data loss. We need to fix the lazy rename pending operation to address this issue Reporter: Gaurav Kanade Priority: Critical -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HADOOP-13831) Correct check for error code to detect Azure Storage Throttling and provide retries
Gaurav Kanade created HADOOP-13831: -- Summary: Correct check for error code to detect Azure Storage Throttling and provide retries Key: HADOOP-13831 URL: https://issues.apache.org/jira/browse/HADOOP-13831 Project: Hadoop Common Issue Type: Bug Reporter: Gaurav Kanade Azure Storage throttling affects HBase operations such as archiving old WALS and others. In such cases the storage driver needs to detect and handle the exception. We put in this logic to do the retries however the condition to check for the exception is not always met due to inconsistency in which the manner the error code is passed back. Instead the retry logic should check for http status code (503) which is more reliable and consistent check -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
