anmolanmol1234 commented on code in PR #4039:
URL: https://github.com/apache/hadoop/pull/4039#discussion_r1040936094


##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsHttpOperation.java:
##########
@@ -314,18 +314,21 @@ public void sendRequest(byte[] buffer, int offset, int 
length) throws IOExceptio
     if (this.isTraceEnabled) {
       startTime = System.nanoTime();
     }
-    try (OutputStream outputStream = this.connection.getOutputStream()) {
-      // update bytes sent before they are sent so we may observe
-      // attempted sends as well as successful sends via the
-      // accompanying statusCode
-      this.bytesSent = length;
+    OutputStream outputStream;
+    try {
+      try {
+        outputStream = this.connection.getOutputStream();
+      } catch (IOException e) {
+        // If getOutputStream fails with an exception due to 100-continue

Review Comment:
   1. The first point is valid, I have made the change where getOutputStream 
throws exception for the cases where 100 continue is not enabled and returns 
back to the caller when it catches an IOException due to 100 continue being 
enabled which leads to processResponse getting the correct status code and then 
eventually the retry logic coming into play. 
   
   2. We need to update the bytes sent for failed as well as passed cases. The 
current change will not swallow any exceptions.
   The handling for various status code with 100 continue enabled is as follows 
   
       1. Case 1 :- getOutputStream doesn't throw any exception, response is 
processed and it gives status code of 200, no retry is needed and hence the 
request succeeds.
       2. Case 2:- getOutputSteam throws exception, we return to the caller and 
in processResponse in this.connection.getResponseCode() it gives status code of 
404 (user error), exponential retry is not needed. We retry without 100 
continue enabled.
       3. Case 3:- getOutputSteam throws exception, we return to the caller and 
in processResponse  it gives status code of 503,
       which shows throttling so we backoff accordingly with exponential retry. 
Since each append request waits for 100 continue response, the stress on the 
server gets reduced.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

Reply via email to