[ https://issues.apache.org/jira/browse/HADOOP-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sunil Govindan updated HADOOP-15703: ------------------------------------ Fix Version/s: 3.2.0 > ABFS - Implement client-side throttling > ---------------------------------------- > > Key: HADOOP-15703 > URL: https://issues.apache.org/jira/browse/HADOOP-15703 > Project: Hadoop Common > Issue Type: Sub-task > Reporter: Sneha Varma > Assignee: Thomas Marquardt > Priority: Major > Fix For: 3.2.0 > > Attachments: HADOOP-15703-HADOOP-15407-001.patch, > HADOOP-15703-HADOOP-15407-002.patch, HADOOP-15703-HADOOP-15407-003.patch, > HADOOP-15703-HADOOP-15407-004.patch > > > Big data workloads frequently exceed the AzureBlobFS max ingress and egress > limits > (https://docs.microsoft.com/en-us/azure/storage/common/storage-scalability-targets). > For example, the max ingress limit for a GRS account in the United States is > currently 10 Gbps. When the limit is exceeded, the AzureBlobFS service fails > a percentage of incoming requests, and this causes the client to initiate the > retry policy. The retry policy delays requests by sleeping, but the sleep > duration is independent of the client throughput and account limit. This > results in low throughput, due to the high number of failed requests and > thrashing causes by the retry policy. > To fix this, we introduce a client-side throttle which minimizes failed > requests and maximizes throughput. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org