[ https://issues.apache.org/jira/browse/HADOOP-12334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14961070#comment-14961070 ]
Gaurav Kanade commented on HADOOP-12334: ---------------------------------------- Thanks [~cnauroth]. Quick follow up question: were you suggesting that these tests be over actual Azure Storage - because I do make some API calls. I am new to the test framework and Trying to understand what would be possible in this scenario. [~onpduo] for reference > Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage > Throttling after retries > ----------------------------------------------------------------------------------------------------- > > Key: HADOOP-12334 > URL: https://issues.apache.org/jira/browse/HADOOP-12334 > Project: Hadoop Common > Issue Type: Improvement > Components: tools > Reporter: Gaurav Kanade > Assignee: Gaurav Kanade > Attachments: HADOOP-12334.01.patch, HADOOP-12334.02.patch, > HADOOP-12334.03.patch, HADOOP-12334.04.patch, HADOOP-12334.05.patch, > HADOOP-12334.06.patch > > > HADOOP-11693 mitigated the problem of HMaster aborting regionserver due to > Azure Storage Throttling event during HBase WAL archival. The way this was > achieved was by applying an intensive exponential retry when throttling > occurred. > As a second level of mitigation we will change the mode of copy operation if > the operation fails even after all retries -i.e. we will do a client side > copy of the blob and then copy it back to destination. This operation will > not be subject to throttling and hence should provide a stronger mitigation. > However it is more expensive, hence we do it only in the case we fail after > all retries -- This message was sent by Atlassian JIRA (v6.3.4#6332)