[james-project] 06/07: JAMES-2806 ObjectStorage Swift retry one more time when fails to save

[btellier]

This is an automated email from the ASF dual-hosted git repository.

btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git

commit 2232634235d72a8c4ed0a84a85422118fe662bbd
Author: Tran Tien Duc <dt...@linagora.com>
AuthorDate: Fri Jul 5 17:37:39 2019 +0700

    JAMES-2806 ObjectStorage Swift retry one more time when fails to save
    
    Swift cause a big time wait for the first put blob request to response.
    
    Assuming there are 2 thread trying to save a blob in the same time,
    and both are expected to fail and when a thread fall back to retry and
    create the bucket, the other thread comes to the step of checking
    bucket existing, now the bucket is already created then retry mechanism
    is rejected.
    
    I want to have the last retry regardless bucket existence to increase
    the chance to save a blob. This is just a conner case and rarely we never
    come up with retrying saving blob. Except the only one use case of Vault
    switching new Bucket to save deleted message.
    
    Then I have to take care about this issue rather than relax
    test concurrency level or disable it.
---
 .../objectstorage/StreamCompatibleBlobPutter.java  | 24 ++++++++++++++--------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/StreamCompatibleBlobPutter.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/StreamCompatibleBlobPutter.java
index 9a3a21f..2375864 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/StreamCompatibleBlobPutter.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/StreamCompatibleBlobPutter.java
@@ -42,6 +42,7 @@ public class StreamCompatibleBlobPutter implements BlobPutter 
{
     private static final Duration FIRST_BACK_OFF = Duration.ofMillis(100);
     private static final Duration FOREVER = Duration.ofMillis(Long.MAX_VALUE);
     private static final Location DEFAULT_LOCATION = null;
+    private static final long RETRY_ONE_LAST_TIME_ON_CONCURRENT_SAVING = 1;
 
     private final BlobStore blobStore;
 
@@ -58,6 +59,10 @@ public class StreamCompatibleBlobPutter implements 
BlobPutter {
                 .withBackoffScheduler(Schedulers.elastic())
                 .retryMax(MAX_RETRIES)
                 .doOnRetry(retryContext -> 
blobStore.createContainerInLocation(DEFAULT_LOCATION, bucketName.asString())))
+            .retryWhen(Retry.onlyIf(RetryContext -> 
isPutMethod(RetryContext.exception()))
+                .withBackoffScheduler(Schedulers.elastic())
+                .exponentialBackoff(FIRST_BACK_OFF, FOREVER)
+                .retryMax(RETRY_ONE_LAST_TIME_ON_CONCURRENT_SAVING))
             .block();
     }
 
@@ -76,15 +81,16 @@ public class StreamCompatibleBlobPutter implements 
BlobPutter {
     }
 
     private boolean needToCreateBucket(Throwable throwable, BucketName 
bucketName) {
-        if (throwable instanceof HttpResponseException
-            || throwable instanceof KeyNotFoundException) {
-
-            return extractHttpException(throwable)
-                    .map(ex -> isPutMethod(ex) && !bucketExisted(bucketName))
-                    .orElse(false);
-        }
+        return Optional.of(throwable)
+            .filter(t -> t instanceof HttpResponseException || t instanceof 
KeyNotFoundException)
+            .flatMap(this::extractHttpException)
+            .map(ex -> isPutMethod(ex) && !bucketExists(bucketName))
+            .orElse(false);
+    }
 
-        return false;
+    private boolean isPutMethod(Throwable throwable) {
+        return throwable instanceof HttpResponseException
+            && isPutMethod((HttpResponseException) throwable);
     }
 
     private boolean isPutMethod(HttpResponseException ex) {
@@ -94,7 +100,7 @@ public class StreamCompatibleBlobPutter implements 
BlobPutter {
             .equals("PUT");
     }
 
-    private boolean bucketExisted(BucketName bucketName) {
+    private boolean bucketExists(BucketName bucketName) {
         return blobStore.containerExists(bucketName.asString());
     }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org