[incubator-druid] branch master updated: Fix for getSingleObjectSummary, replacing keyCount with objectSummaries().size (#7000)

[cwylie]

This is an automated email from the ASF dual-hosted git repository.

cwylie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-druid.git


The following commit(s) were added to refs/heads/master by this push:
     new 315ccb7  Fix for getSingleObjectSummary, replacing keyCount with 
objectSummaries().size (#7000)
315ccb7 is described below

commit 315ccb76b819790a4861bf16e493dba47605291b
Author: anantmf <anan...@outlook.com>
AuthorDate: Tue Feb 5 15:45:45 2019 -0800

    Fix for getSingleObjectSummary, replacing keyCount with 
objectSummaries().size (#7000)
    
    * Instead of using keyCount, changing it to check the size of 
objectSummaries.
    
    For issue:
    https://github.com/apache/incubator-druid/issues/6980
    https://github.com/apache/incubator-druid/issues/6980#issuecomment-460006580
    
    * Changing another usage of keyCount with size of objectSummaries.
    
    * Adding some comments to explain why using keyCount is not working as 
expected.
---
 .../main/java/org/apache/druid/storage/s3/S3DataSegmentMover.java    | 5 ++++-
 .../src/main/java/org/apache/druid/storage/s3/S3Utils.java           | 5 ++++-
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git 
a/extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentMover.java
 
b/extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentMover.java
index 4fd54b4..84a76d1 100644
--- 
a/extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentMover.java
+++ 
b/extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentMover.java
@@ -174,7 +174,10 @@ public class S3DataSegmentMover implements DataSegmentMover
               .withPrefix(s3Path)
               .withMaxKeys(1)
       );
-      if (listResult.getKeyCount() == 0) {
+      // Using getObjectSummaries().size() instead of getKeyCount as, in some 
cases
+      // it is observed that even though the getObjectSummaries returns some 
data
+      // keyCount is still zero.
+      if (listResult.getObjectSummaries().size() == 0) {
         // should never happen
         throw new ISE("Unable to list object [s3://%s/%s]", s3Bucket, s3Path);
       }
diff --git 
a/extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3Utils.java
 
b/extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3Utils.java
index e0a3dac..9785886 100644
--- 
a/extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3Utils.java
+++ 
b/extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3Utils.java
@@ -251,7 +251,10 @@ public class S3Utils
         .withMaxKeys(1);
     final ListObjectsV2Result result = s3Client.listObjectsV2(request);
 
-    if (result.getKeyCount() == 0) {
+    // Using getObjectSummaries().size() instead of getKeyCount as, in some 
cases
+    // it is observed that even though the getObjectSummaries returns some data
+    // keyCount is still zero.
+    if (result.getObjectSummaries().size() == 0) {
       throw new ISE("Cannot find object for bucket[%s] and key[%s]", bucket, 
key);
     }
     final S3ObjectSummary objectSummary = result.getObjectSummaries().get(0);


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org