[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=315221=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-315221 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 19/Sep/19 18:06 Start Date: 19/Sep/19 18:06 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 315221) Time Spent: 10h 50m (was: 10h 40m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Fix For: 0.5.0 > > Time Spent: 10h 50m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=315220=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-315220 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 19/Sep/19 18:05 Start Date: 19/Sep/19 18:05 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-533244921 +1 LGTM. Thank You @elek for the contribution. I will commit this to the trunk. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 315220) Time Spent: 10h 40m (was: 10.5h) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 10h 40m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=314931=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314931 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 19/Sep/19 09:57 Start Date: 19/Sep/19 09:57 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-533058852 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 3434 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 2 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 4 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 44 | Maven dependency ordering for branch | | -1 | mvninstall | 31 | hadoop-ozone in trunk failed. | | -1 | compile | 22 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 125 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 897 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 47 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 164 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 27 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 29 | Maven dependency ordering for patch | | -1 | mvninstall | 33 | hadoop-ozone in the patch failed. | | -1 | compile | 25 | hadoop-ozone in the patch failed. | | -1 | cc | 25 | hadoop-ozone in the patch failed. | | -1 | javac | 25 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 29 | hadoop-hdds: The patch generated 6 new + 8 unchanged - 2 fixed = 14 total (was 10) | | -0 | checkstyle | 96 | hadoop-ozone: The patch generated 370 new + 2478 unchanged - 15 fixed = 2848 total (was 2493) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 660 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 51 | hadoop-ozone in the patch failed. | | -1 | findbugs | 27 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | +1 | unit | 249 | hadoop-hdds in the patch passed. | | -1 | unit | 29 | hadoop-ozone in the patch failed. | | +1 | asflicense | 32 | The patch does not generate ASF License warnings. | | | | 6687 | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 68f3b34ec47c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 1029060 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/patch-compile-hadoop-ozone.txt | | cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/diff-checkstyle-hadoop-hdds.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/diff-checkstyle-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/patch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/13/artifact/out/patch-findbugs-hadoop-ozone.txt | | unit
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=314048=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314048 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 17/Sep/19 23:02 Start Date: 17/Sep/19 23:02 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#discussion_r325421790 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/protocolPB/OzoneManagerRequestHandler.java ## @@ -822,6 +822,7 @@ private MultipartInfoInitiateResponse initiateMultiPartUpload( .setBucketName(keyArgs.getBucketName()) .setKeyName(keyArgs.getKeyName()) .setType(keyArgs.getType()) +.setFactor(keyArgs.getFactor()) Review comment: Minor: We don't need this, as in line 827 it is already being done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314048) Time Spent: 10h 20m (was: 10h 10m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 10h 20m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=313792=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313792 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 17/Sep/19 16:50 Start Date: 17/Sep/19 16:50 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-532305637 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 43 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 4 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 48 | Maven dependency ordering for branch | | -1 | mvninstall | 33 | hadoop-ozone in trunk failed. | | -1 | compile | 23 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 124 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 1003 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 164 | trunk passed | | 0 | spotbugs | 184 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 26 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 32 | Maven dependency ordering for patch | | -1 | mvninstall | 35 | hadoop-ozone in the patch failed. | | -1 | compile | 27 | hadoop-ozone in the patch failed. | | -1 | cc | 27 | hadoop-ozone in the patch failed. | | -1 | javac | 27 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 31 | hadoop-hdds: The patch generated 5 new + 9 unchanged - 1 fixed = 14 total (was 10) | | -0 | checkstyle | 97 | hadoop-ozone: The patch generated 448 new + 2401 unchanged - 92 fixed = 2849 total (was 2493) | | +1 | mvnsite | 0 | the patch passed | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | shadedclient | 765 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 91 | hadoop-ozone generated 9 new + 249 unchanged - 7 fixed = 258 total (was 256) | | -1 | findbugs | 28 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 247 | hadoop-hdds in the patch failed. | | -1 | unit | 31 | hadoop-ozone in the patch failed. | | +1 | asflicense | 35 | The patch does not generate ASF License warnings. | | | | 3706 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdds.scm.container.placement.algorithms.TestSCMContainerPlacementRackAware | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/12/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 4392a4c17837 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / c474e24 | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/12/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/12/artifact/out/branch-compile-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/12/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/12/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/12/artifact/out/patch-compile-hadoop-ozone.txt | | cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/12/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/12/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/12/artifact/out/diff-checkstyle-hadoop-hdds.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/12/artifact/out/diff-checkstyle-hadoop-ozone.txt | | whitespace |
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=313791=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313791
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 17/Sep/19 16:50
Start Date: 17/Sep/19 16:50
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r325276989
##
File path:
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadListParts.java
##
@@ -39,11 +45,15 @@
// A list can be truncated if the number of parts exceeds the limit
// returned in the MaxParts element.
private boolean truncated;
+
private final List partInfoList = new ArrayList<>();
public OmMultipartUploadListParts(HddsProtos.ReplicationType type,
+ HddsProtos.ReplicationFactor factor,
int nextMarker, boolean truncate) {
this.replicationType = type;
+this.replicationFactor = factor;
+
Review comment:
whitespace:end of line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 313791)
Time Spent: 10h (was: 9h 50m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 10h
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=313710=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313710
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 17/Sep/19 14:08
Start Date: 17/Sep/19 14:08
Worklog Time Spent: 10m
Work Description: elek commented on pull request #1277: HDDS-1054. List
Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r325188832
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3StorageType.java
##
@@ -52,4 +52,12 @@ public static S3StorageType getDefault() {
return STANDARD;
}
+ public static S3StorageType fromReplicationType(
+ ReplicationType replicationType) {
+if (replicationType == ReplicationType.STAND_ALONE) {
+ return S3StorageType.REDUCED_REDUNDANCY;
Review comment:
Sounds reasonable but it requires the change of the protobuf (factor is not
available from the message). Seems to be a bigger change but I added it to this
patch (sorry, now it's an other commit to review :-P )
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 313710)
Time Spent: 9h 50m (was: 9h 40m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=313651=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313651
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 17/Sep/19 12:02
Start Date: 17/Sep/19 12:02
Worklog Time Spent: 10m
Work Description: elek commented on pull request #1277: HDDS-1054. List
Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r325127443
##
File path:
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadList.java
##
@@ -18,15 +18,21 @@
package org.apache.hadoop.ozone.om.helpers;
-import java.util.ArrayList;
import java.util.List;
+import org.apache.hadoop.hdds.client.ReplicationType;
+import org.apache.hadoop.hdds.protocol.proto.HddsProtos.ReplicationFactor;
+
/**
- * List of in-flight MU upoads.
+ * List of in-flight MPU uploads.
*/
public class OmMultipartUploadList {
- private List uploads = new ArrayList<>();
+ private ReplicationType replicationType;
+
+ private ReplicationFactor replicationFactor;
Review comment:
Yes, it was added during one of the experiments. I removed it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 313651)
Time Spent: 9h 40m (was: 9.5h)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=313270=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313270 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 16/Sep/19 20:04 Start Date: 16/Sep/19 20:04 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-531935761 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 313270) Time Spent: 9.5h (was: 9h 20m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 9.5h > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=313268=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313268
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 16/Sep/19 20:02
Start Date: 16/Sep/19 20:02
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r324859826
##
File path:
hadoop-ozone/s3gateway/src/main/java/org/apache/hadoop/ozone/s3/util/S3StorageType.java
##
@@ -52,4 +52,12 @@ public static S3StorageType getDefault() {
return STANDARD;
}
+ public static S3StorageType fromReplicationType(
+ ReplicationType replicationType) {
+if (replicationType == ReplicationType.STAND_ALONE) {
+ return S3StorageType.REDUCED_REDUNDANCY;
Review comment:
just a question, I think this is before even this patch.
Previously we use STAND_ALONE for replication factor one, now we use RATIS
with one and three. So, I think we need to change the code for this right?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 313268)
Time Spent: 9h 20m (was: 9h 10m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=313259=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313259
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 16/Sep/19 19:46
Start Date: 16/Sep/19 19:46
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r324853402
##
File path:
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadList.java
##
@@ -18,15 +18,21 @@
package org.apache.hadoop.ozone.om.helpers;
-import java.util.ArrayList;
import java.util.List;
+import org.apache.hadoop.hdds.client.ReplicationType;
+import org.apache.hadoop.hdds.protocol.proto.HddsProtos.ReplicationFactor;
+
/**
- * List of in-flight MU upoads.
+ * List of in-flight MPU uploads.
*/
public class OmMultipartUploadList {
- private List uploads = new ArrayList<>();
+ private ReplicationType replicationType;
+
+ private ReplicationFactor replicationFactor;
Review comment:
Why do we need this in the multipart list result?
These are specific to each in-progress MPU.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 313259)
Time Spent: 9h 10m (was: 9h)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 9h 10m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=313027=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313027 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 16/Sep/19 14:53 Start Date: 16/Sep/19 14:53 Worklog Time Spent: 10m Work Description: elek commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-531813760 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 313027) Time Spent: 9h (was: 8h 50m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 9h > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=313025=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-313025 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 16/Sep/19 14:49 Start Date: 16/Sep/19 14:49 Worklog Time Spent: 10m Work Description: elek commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-531811701 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 313025) Time Spent: 8h 50m (was: 8h 40m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 8h 50m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=312991=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312991 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 16/Sep/19 13:54 Start Date: 16/Sep/19 13:54 Worklog Time Spent: 10m Work Description: elek commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-531787206 /retest This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 312991) Time Spent: 8h 40m (was: 8.5h) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 8h 40m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=312492=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312492 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 14/Sep/19 04:10 Start Date: 14/Sep/19 04:10 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-531446769 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 1208 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 77 | Maven dependency ordering for branch | | -1 | mvninstall | 32 | hadoop-ozone in trunk failed. | | -1 | compile | 19 | hadoop-ozone in trunk failed. | | +1 | checkstyle | 100 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 937 | branch has no errors when building and testing our client artifacts. | | -1 | javadoc | 13 | hadoop-hdds in trunk failed. | | -1 | javadoc | 15 | hadoop-ozone in trunk failed. | | 0 | spotbugs | 159 | Used deprecated FindBugs config; considering switching to SpotBugs. | | -1 | findbugs | 23 | hadoop-ozone in trunk failed. | ||| _ Patch Compile Tests _ | | 0 | mvndep | 23 | Maven dependency ordering for patch | | -1 | mvninstall | 31 | hadoop-ozone in the patch failed. | | -1 | compile | 22 | hadoop-ozone in the patch failed. | | -1 | cc | 22 | hadoop-ozone in the patch failed. | | -1 | javac | 22 | hadoop-ozone in the patch failed. | | -0 | checkstyle | 24 | hadoop-hdds: The patch generated 5 new + 9 unchanged - 1 fixed = 14 total (was 10) | | -0 | checkstyle | 71 | hadoop-ozone: The patch generated 373 new + 2410 unchanged - 15 fixed = 2783 total (was 2425) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 1 | The patch has no whitespace issues. | | +1 | shadedclient | 726 | patch has no errors when building and testing our client artifacts. | | -1 | javadoc | 14 | hadoop-hdds in the patch failed. | | -1 | javadoc | 15 | hadoop-ozone in the patch failed. | | -1 | findbugs | 23 | hadoop-ozone in the patch failed. | ||| _ Other Tests _ | | -1 | unit | 163 | hadoop-hdds in the patch failed. | | -1 | unit | 25 | hadoop-ozone in the patch failed. | | +1 | asflicense | 29 | The patch does not generate ASF License warnings. | | | | 4264 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.container.ozoneimpl.TestOzoneContainer | | | hadoop.ozone.container.keyvalue.TestKeyValueContainer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/11/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 86494a8d4639 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 6a9f7ca | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/11/artifact/out/branch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/11/artifact/out/branch-compile-hadoop-ozone.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/11/artifact/out/branch-javadoc-hadoop-hdds.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/11/artifact/out/branch-javadoc-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/11/artifact/out/branch-findbugs-hadoop-ozone.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/11/artifact/out/patch-mvninstall-hadoop-ozone.txt | | compile | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/11/artifact/out/patch-compile-hadoop-ozone.txt | | cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/11/artifact/out/patch-compile-hadoop-ozone.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/11/artifact/out/patch-compile-hadoop-ozone.txt | | checkstyle |
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=312482=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312482
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 14/Sep/19 02:52
Start Date: 14/Sep/19 02:52
Worklog Time Spent: 10m
Work Description: elek commented on pull request #1277: HDDS-1054. List
Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r324409675
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##
@@ -1270,6 +1271,58 @@ public void abortMultipartUpload(OmKeyArgs omKeyArgs)
throws IOException {
}
+ @Override
+ public OmMultipartUploadList listMultipartUploads(String volumeName,
+ String bucketName, String prefix) throws OMException {
+Preconditions.checkNotNull(volumeName);
+Preconditions.checkNotNull(bucketName);
+
+metadataManager.getLock().acquireLock(BUCKET_LOCK, volumeName, bucketName);
+try {
+
+ List multipartUploadKeys =
+ metadataManager
+ .getMultipartUploadKeys(volumeName, bucketName, prefix);
+
+ List collect = multipartUploadKeys.stream()
+ .map(OmMultipartUpload::from)
+ .map(upload -> {
+String dbKey = metadataManager
+.getOzoneKey(upload.getVolumeName(),
+upload.getBucketName(),
+upload.getKeyName());
+try {
+ Table openKeyTable =
+ metadataManager.getOpenKeyTable();
+
+ OmKeyInfo omKeyInfo =
+ openKeyTable.get(upload.getDbKey());
Review comment:
You are 100% right. But it seems to be a bigger change. Let's do it in
https://issues.apache.org/jira/browse/HDDS-2131
(On the other hand I added audit + metrics support because they were just a
few lines)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 312482)
Time Spent: 8h 20m (was: 8h 10m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=312481=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312481 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 14/Sep/19 02:48 Start Date: 14/Sep/19 02:48 Worklog Time Spent: 10m Work Description: elek commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-531442173 > Have few comments in place. > Can we open Jira's for all TODO's in this Jira for tracking purpose? > > 1. Audit support for the new method. > > 2. Pagination support and other query parameters support. > > 3. If replication type will not be handled in this Jira, can you open Jira for this one too. Sure. Audit + the replication type support is added to this patch. I created HDDS-2130 for the pagination support. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 312481) Time Spent: 8h 10m (was: 8h) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 8h 10m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=312478=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312478
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 14/Sep/19 02:39
Start Date: 14/Sep/19 02:39
Worklog Time Spent: 10m
Work Description: elek commented on pull request #1277: HDDS-1054. List
Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r324409356
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java
##
@@ -826,6 +828,28 @@ private VolumeList getVolumesByUser(String userNameKey)
return count;
}
+ @Override
+ public List getMultipartUploadKeys(
+ String volumeName, String bucketName, String prefix) throws IOException {
+List response = new ArrayList<>();
+TableIterator>
+iterator = getMultipartInfoTable().iterator();
+
+String prefixKey =
+OmMultipartUpload.getDbKey(volumeName, bucketName, prefix);
+iterator.seek(prefixKey);
+
+while (iterator.hasNext()) {
Review comment:
Can you please help me to understand how should it be done? What about if an
MPU is finished and deleted? How is it cached? I think I can't return with
(cached values + db values) because the deletions
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 312478)
Time Spent: 8h (was: 7h 50m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 8h
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=312476=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-312476 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 14/Sep/19 02:26 Start Date: 14/Sep/19 02:26 Worklog Time Spent: 10m Work Description: elek commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#discussion_r324409058 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMMetadataManager.java ## @@ -327,4 +327,11 @@ String getMultipartKey(String volume, String bucket, String key, String */ long countEstimatedRowsInTable(Table table) throws IOException; + + /** + * Return the existing upload keys which includes volumeName, bucketName, + * keyName and uploadId. Review comment: Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 312476) Time Spent: 7h 50m (was: 7h 40m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 7h 50m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=311676=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311676
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 12/Sep/19 19:59
Start Date: 12/Sep/19 19:59
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r315708896
##
File path: hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
##
@@ -200,3 +200,24 @@ Test Multipart Upload with the simplified aws s3 cp API
Execute AWSS3Clicp s3://${BUCKET}/mpyawscli
/tmp/part1.result
Execute AWSS3Clirm s3://${BUCKET}/mpyawscli
Compare files /tmp/part1
/tmp/part1.result
+
+Test Multipart Upload list
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key1
+${uploadID1} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key1
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key2
+${uploadID2} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key2
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli list-multipart-uploads
--bucket ${BUCKET} --prefix listtest
+Should contain ${result}${uploadID1}
+Should contain ${result}${uploadID2}
+
+${count} = Execute and checkrc echo '${result}' | jq -r
'.Uploads | length' 0
Review comment:
whitespace:end of line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311676)
Time Spent: 7h 40m (was: 7.5h)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=311675=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311675
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 12/Sep/19 19:59
Start Date: 12/Sep/19 19:59
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r314769222
##
File path: hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
##
@@ -200,3 +200,24 @@ Test Multipart Upload with the simplified aws s3 cp API
Execute AWSS3Clicp s3://${BUCKET}/mpyawscli
/tmp/part1.result
Execute AWSS3Clirm s3://${BUCKET}/mpyawscli
Compare files /tmp/part1
/tmp/part1.result
+
+Test Multipart Upload list
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key1
+${uploadID1} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key1
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key2
+${uploadID2} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key2
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli list-multipart-uploads
--bucket ${BUCKET} --prefix listtest
+Should contain ${result}${uploadID1}
+Should contain ${result}${uploadID2}
+
+${count} = Execute and checkrc echo '${result}' | jq -r
'.Uploads | length' 0
Review comment:
whitespace:end of line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311675)
Time Spent: 7.5h (was: 7h 20m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=311674=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311674
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 12/Sep/19 19:59
Start Date: 12/Sep/19 19:59
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r312740496
##
File path: hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
##
@@ -200,3 +200,24 @@ Test Multipart Upload with the simplified aws s3 cp API
Execute AWSS3Clicp s3://${BUCKET}/mpyawscli
/tmp/part1.result
Execute AWSS3Clirm s3://${BUCKET}/mpyawscli
Compare files /tmp/part1
/tmp/part1.result
+
+Test Multipart Upload list
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key1
+${uploadID1} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key1
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key2
+${uploadID2} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key2
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli list-multipart-uploads
--bucket ${BUCKET} --prefix listtest
+Should contain ${result}${uploadID1}
+Should contain ${result}${uploadID2}
+
+${count} = Execute and checkrc echo '${result}' | jq -r
'.Uploads | length' 0
Review comment:
whitespace:end of line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 311674)
Time Spent: 7h 20m (was: 7h 10m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=311671=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311671 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 12/Sep/19 19:58 Start Date: 12/Sep/19 19:58 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-526197265 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 34 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 12 | Maven dependency ordering for branch | | +1 | mvninstall | 597 | trunk passed | | +1 | compile | 383 | trunk passed | | +1 | checkstyle | 68 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 917 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 167 | trunk passed | | 0 | spotbugs | 470 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 701 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 22 | Maven dependency ordering for patch | | +1 | mvninstall | 593 | the patch passed | | +1 | compile | 394 | the patch passed | | +1 | cc | 394 | the patch passed | | +1 | javac | 394 | the patch passed | | +1 | checkstyle | 81 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 756 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 171 | the patch passed | | +1 | findbugs | 639 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 303 | hadoop-hdds in the patch passed. | | -1 | unit | 1591 | hadoop-ozone in the patch failed. | | +1 | asflicense | 45 | The patch does not generate ASF License warnings. | | | | 7646 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.TestOzoneConfigurationFields | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/9/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 376784018ab2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 8c0759d | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/9/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/9/testReport/ | | Max. process+thread count | 5019 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/9/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311671) Time Spent: 7h (was: 6h 50m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 7h > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] >
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=311669=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311669 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 12/Sep/19 19:58 Start Date: 12/Sep/19 19:58 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-525862802 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 0 | Docker mode activated. | | -1 | patch | 10 | https://github.com/apache/hadoop/pull/1277 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/6/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311669) Time Spent: 6h 40m (was: 6.5h) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 6h 40m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=311672=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311672 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 12/Sep/19 19:58 Start Date: 12/Sep/19 19:58 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-527276378 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 39 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for branch | | +1 | mvninstall | 614 | trunk passed | | +1 | compile | 388 | trunk passed | | +1 | checkstyle | 99 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 922 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 178 | trunk passed | | 0 | spotbugs | 431 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 639 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 26 | Maven dependency ordering for patch | | +1 | mvninstall | 561 | the patch passed | | +1 | compile | 423 | the patch passed | | +1 | cc | 423 | the patch passed | | +1 | javac | 423 | the patch passed | | +1 | checkstyle | 83 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 778 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 192 | the patch passed | | +1 | findbugs | 770 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 310 | hadoop-hdds in the patch passed. | | -1 | unit | 2359 | hadoop-ozone in the patch failed. | | +1 | asflicense | 56 | The patch does not generate ASF License warnings. | | | | 8614 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.container.common.statemachine.commandhandler.TestBlockDeletion | | | hadoop.ozone.client.rpc.TestContainerStateMachineFailures | | | hadoop.ozone.client.rpc.TestCommitWatcher | | | hadoop.ozone.om.TestSecureOzoneManager | | | hadoop.ozone.client.rpc.TestWatchForCommit | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/10/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 371835c48863 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 915cbc9 | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/10/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/10/testReport/ | | Max. process+thread count | 4430 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/10/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311672) Time Spent: 7h 10m (was: 7h) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker >
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=311670=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311670 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 12/Sep/19 19:58 Start Date: 12/Sep/19 19:58 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-52616 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 76 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for branch | | +1 | mvninstall | 616 | trunk passed | | +1 | compile | 386 | trunk passed | | +1 | checkstyle | 69 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 926 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 175 | trunk passed | | 0 | spotbugs | 471 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 709 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 21 | Maven dependency ordering for patch | | +1 | mvninstall | 610 | the patch passed | | +1 | compile | 409 | the patch passed | | +1 | cc | 409 | the patch passed | | +1 | javac | 409 | the patch passed | | +1 | checkstyle | 95 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 808 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 198 | the patch passed | | +1 | findbugs | 685 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 374 | hadoop-hdds in the patch passed. | | -1 | unit | 262 | hadoop-ozone in the patch failed. | | +1 | asflicense | 40 | The patch does not generate ASF License warnings. | | | | 6658 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.om.TestKeyManagerUnit | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux fc03450f9b59 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / c749f62 | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/8/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/8/testReport/ | | Max. process+thread count | 1237 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/8/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311670) Time Spent: 6h 50m (was: 6h 40m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 6h 50m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] >
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=311668=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311668 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 12/Sep/19 19:57 Start Date: 12/Sep/19 19:57 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-525885449 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 0 | Docker mode activated. | | -1 | patch | 10 | https://github.com/apache/hadoop/pull/1277 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/7/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311668) Time Spent: 6.5h (was: 6h 20m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 6.5h > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=311664=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-311664 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 12/Sep/19 19:52 Start Date: 12/Sep/19 19:52 Worklog Time Spent: 10m Work Description: elek commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#discussion_r323920792 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java ## @@ -555,6 +555,16 @@ public OzoneOutputStream createFile(String keyName, long size, .listStatus(volumeName, name, keyName, recursive, startKey, numEntries); } + /** + * Return with the list of the in-flight multipart uploads. + * + * @param prefix Optional string to filter for the selected keys. + */ + public OzoneMultipartUploadList listMultpartUploads(String prefix) Review comment: thanks, fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 311664) Time Spent: 6h 20m (was: 6h 10m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 6h 20m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305940=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305940
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 03/Sep/19 22:31
Start Date: 03/Sep/19 22:31
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r320505479
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java
##
@@ -826,6 +828,28 @@ private VolumeList getVolumesByUser(String userNameKey)
return count;
}
+ @Override
+ public List getMultipartUploadKeys(
+ String volumeName, String bucketName, String prefix) throws IOException {
+List response = new ArrayList<>();
+TableIterator>
+iterator = getMultipartInfoTable().iterator();
+
+String prefixKey =
+OmMultipartUpload.getDbKey(volumeName, bucketName, prefix);
+iterator.seek(prefixKey);
+
+while (iterator.hasNext()) {
Review comment:
Here, we need to consider table cache also now HA/Non-HA code path is merged
(HDDS-1909)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 305940)
Time Spent: 6h 10m (was: 6h)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305786=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305786
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 03/Sep/19 18:53
Start Date: 03/Sep/19 18:53
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r320425899
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##
@@ -1270,6 +1271,58 @@ public void abortMultipartUpload(OmKeyArgs omKeyArgs)
throws IOException {
}
+ @Override
+ public OmMultipartUploadList listMultipartUploads(String volumeName,
+ String bucketName, String prefix) throws OMException {
+Preconditions.checkNotNull(volumeName);
+Preconditions.checkNotNull(bucketName);
+
+metadataManager.getLock().acquireLock(BUCKET_LOCK, volumeName, bucketName);
+try {
+
+ List multipartUploadKeys =
+ metadataManager
+ .getMultipartUploadKeys(volumeName, bucketName, prefix);
+
+ List collect = multipartUploadKeys.stream()
+ .map(OmMultipartUpload::from)
+ .map(upload -> {
+String dbKey = metadataManager
+.getOzoneKey(upload.getVolumeName(),
+upload.getBucketName(),
+upload.getKeyName());
+try {
+ Table openKeyTable =
+ metadataManager.getOpenKeyTable();
+
+ OmKeyInfo omKeyInfo =
+ openKeyTable.get(upload.getDbKey());
+ upload.setCreationTime(
+ Instant.ofEpochMilli(omKeyInfo.getCreationTime()));
+} catch (IOException e) {
+ LOG.warn(
+ "Open key entry for multipart upload record can be read {}",
+ dbKey);
+}
+return upload;
+ })
+ .collect(Collectors.toList());
+
+ OmMultipartUploadList omMultipartUploadList =
+ new OmMultipartUploadList(collect);
+
+ return omMultipartUploadList;
+
+} catch (IOException ex) {
+ LOG.error("List Multipart Uploads Failed: volume: " + volumeName +
Review comment:
LOG.error("List Multipart Uploads Failed: volume: " + volumeName +
"bucket: " + bucketName + "prefix: " + prefix, ex);
to
LOG.error("List Multipart Uploads Failed: volume: {} bucket: {} prefix:
{}",volumeName, bucketName, prefix, ex);
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 305786)
Time Spent: 6h (was: 5h 50m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 6h
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305774=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305774
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 03/Sep/19 18:43
Start Date: 03/Sep/19 18:43
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on issue #1277: HDDS-1054.
List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#issuecomment-527587093
When i run the command I see below output, whereas it is not showing up
bucketName, keyMarker other fields.
bash-4.2$ aws s3api --endpoint http://s3g:9878 list-multipart-uploads
--bucket b1234 --prefix mpu
{
"Uploads": [
{
"Initiator": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
},
"Initiated": "2019-09-03T18:39:35.916Z",
"UploadId":
"24eea7f4-52db-4a0f-978a-f06cb7a57657-102730037717565440",
"StorageClass": "STANDARD",
"Key": "mpukey",
"Owner": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
}
},
{
"Initiator": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
},
"Initiated": "2019-09-03T18:39:37.816Z",
"UploadId":
"81c0e5c2-db11-4b11-a5f7-81f48bdbfb04-102730037842083841",
"StorageClass": "STANDARD",
"Key": "mpukey1",
"Owner": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
}
},
{
"Initiator": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
},
"Initiated": "2019-09-03T18:39:39.259Z",
"UploadId":
"4aab75b8-1954-4e8a-a658-0d403bcbc42f-102730037936717826",
"StorageClass": "STANDARD",
"Key": "mpukey2",
"Owner": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
}
}
]
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 305774)
Time Spent: 5.5h (was: 5h 20m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305777=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305777 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 03/Sep/19 18:44 Start Date: 03/Sep/19 18:44 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#discussion_r320422434 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadList.java ## @@ -6,9 +6,9 @@ * to you under the Apache License, Version 2.0 (the * "License"); you may not use this file except in compliance * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * + * + * http://www.apache.org/licenses/LICENSE-2.0 Review comment: Unintended change? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305777) Time Spent: 5h 50m (was: 5h 40m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 5h 50m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305775=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305775
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 03/Sep/19 18:43
Start Date: 03/Sep/19 18:43
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on issue #1277: HDDS-1054.
List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#issuecomment-527587093
When i run the command I see below output, whereas it is not showing up
bucketName, keyMarker other fields.
```
bash-4.2$ aws s3api --endpoint http://s3g:9878 list-multipart-uploads
--bucket b1234 --prefix mpu
{
"Uploads": [
{
"Initiator": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
},
"Initiated": "2019-09-03T18:39:35.916Z",
"UploadId":
"24eea7f4-52db-4a0f-978a-f06cb7a57657-102730037717565440",
"StorageClass": "STANDARD",
"Key": "mpukey",
"Owner": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
}
},
{
"Initiator": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
},
"Initiated": "2019-09-03T18:39:37.816Z",
"UploadId":
"81c0e5c2-db11-4b11-a5f7-81f48bdbfb04-102730037842083841",
"StorageClass": "STANDARD",
"Key": "mpukey1",
"Owner": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
}
},
{
"Initiator": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
},
"Initiated": "2019-09-03T18:39:39.259Z",
"UploadId":
"4aab75b8-1954-4e8a-a658-0d403bcbc42f-102730037936717826",
"StorageClass": "STANDARD",
"Key": "mpukey2",
"Owner": {
"DisplayName": "Not Supported",
"ID": "NOT-SUPPORTED"
}
}
]
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 305775)
Time Spent: 5h 40m (was: 5.5h)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305752=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305752
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 03/Sep/19 18:10
Start Date: 03/Sep/19 18:10
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r320407326
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##
@@ -1270,6 +1271,58 @@ public void abortMultipartUpload(OmKeyArgs omKeyArgs)
throws IOException {
}
+ @Override
+ public OmMultipartUploadList listMultipartUploads(String volumeName,
+ String bucketName, String prefix) throws OMException {
+Preconditions.checkNotNull(volumeName);
+Preconditions.checkNotNull(bucketName);
Review comment:
prefix also should not be null. As prefix is also required in
ListMultipartUploadRequest in proto.
And also here we using "+" for concatentation, so if we pass null for
prefix, then it will be /volume/bucket/null. The below method is called by
getMultipartUploadKeys.
public static String getDbKey(String volume, String bucket, String key) {
return OM_KEY_PREFIX + volume + OM_KEY_PREFIX + bucket +
OM_KEY_PREFIX + key;
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 305752)
Time Spent: 5h 20m (was: 5h 10m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305750=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305750
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 03/Sep/19 18:08
Start Date: 03/Sep/19 18:08
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r320407326
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##
@@ -1270,6 +1271,58 @@ public void abortMultipartUpload(OmKeyArgs omKeyArgs)
throws IOException {
}
+ @Override
+ public OmMultipartUploadList listMultipartUploads(String volumeName,
+ String bucketName, String prefix) throws OMException {
+Preconditions.checkNotNull(volumeName);
+Preconditions.checkNotNull(bucketName);
Review comment:
prefix also should not be null. As prefix is also required in
ListMultipartUploadRequest in proto.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 305750)
Time Spent: 5h 10m (was: 5h)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305749=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305749
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 03/Sep/19 18:04
Start Date: 03/Sep/19 18:04
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r320405582
##
File path:
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUpload.java
##
@@ -0,0 +1,110 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.helpers;
+
+import java.time.Instant;
+
+import static org.apache.hadoop.ozone.OzoneConsts.OM_KEY_PREFIX;
+
+/**
+ * Information about one initialized upload.
+ */
+public class OmMultipartUpload {
+
+ private String volumeName;
+
+ private String bucketName;
+
+ private String keyName;
+
+ private String uploadId;
+
+ private Instant creationTime;
+
+ public OmMultipartUpload(String volumeName, String bucketName,
+ String keyName, String uploadId) {
+this.volumeName = volumeName;
+this.bucketName = bucketName;
+this.keyName = keyName;
+this.uploadId = uploadId;
+ }
+
+ public OmMultipartUpload(String volumeName, String bucketName,
+ String keyName, String uploadId, Instant creationDate) {
+this.volumeName = volumeName;
+this.bucketName = bucketName;
+this.keyName = keyName;
+this.uploadId = uploadId;
+this.creationTime = creationDate;
+ }
+
+ public static OmMultipartUpload from(String key) {
+String[] split = key.split(OM_KEY_PREFIX);
+if (split.length < 5) {
+ throw new IllegalArgumentException("Key " + key
+ + " doesn't have enough segments to be a valid multpart upload key");
Review comment:
multpart -> multipart
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 305749)
Time Spent: 5h (was: 4h 50m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 5h
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305747=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305747 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 03/Sep/19 18:03 Start Date: 03/Sep/19 18:03 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#discussion_r320405424 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneBucket.java ## @@ -555,6 +555,16 @@ public OzoneOutputStream createFile(String keyName, long size, .listStatus(volumeName, name, keyName, recursive, startKey, numEntries); } + /** + * Return with the list of the in-flight multipart uploads. + * + * @param prefix Optional string to filter for the selected keys. + */ + public OzoneMultipartUploadList listMultpartUploads(String prefix) Review comment: listMultpartUploads -> listMultipartUploads This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305747) Time Spent: 4h 50m (was: 4h 40m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 4h 50m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305737=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305737
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 03/Sep/19 17:54
Start Date: 03/Sep/19 17:54
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r320401502
##
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##
@@ -1270,6 +1271,58 @@ public void abortMultipartUpload(OmKeyArgs omKeyArgs)
throws IOException {
}
+ @Override
+ public OmMultipartUploadList listMultipartUploads(String volumeName,
+ String bucketName, String prefix) throws OMException {
+Preconditions.checkNotNull(volumeName);
+Preconditions.checkNotNull(bucketName);
+
+metadataManager.getLock().acquireLock(BUCKET_LOCK, volumeName, bucketName);
+try {
+
+ List multipartUploadKeys =
+ metadataManager
+ .getMultipartUploadKeys(volumeName, bucketName, prefix);
+
+ List collect = multipartUploadKeys.stream()
+ .map(OmMultipartUpload::from)
+ .map(upload -> {
+String dbKey = metadataManager
+.getOzoneKey(upload.getVolumeName(),
+upload.getBucketName(),
+upload.getKeyName());
+try {
+ Table openKeyTable =
+ metadataManager.getOpenKeyTable();
+
+ OmKeyInfo omKeyInfo =
+ openKeyTable.get(upload.getDbKey());
Review comment:
Here we are reading openKeyTable only for getting creation time. If we can
have this information in omMultipartKeyInfo, we could avoid DB calls for
openKeyTable.
To do this, We can set creationTime in OmMultipartKeyInfo during
initiateMultipartUpload . In this way, we can get all the required information
from the MultipartKeyInfo table.
And also StorageClass is missing from the returned OmMultipartUpload, as
listMultipartUploads shows StorageClass information. For this, if we can return
replicationType and depending on this value, we can set StorageClass in the
listMultipartUploads Response.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 305737)
Time Spent: 4h 40m (was: 4.5h)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305713=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305713 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 03/Sep/19 17:23 Start Date: 03/Sep/19 17:23 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#discussion_r320388230 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMMetadataManager.java ## @@ -327,4 +327,11 @@ String getMultipartKey(String volume, String bucket, String key, String */ long countEstimatedRowsInTable(Table table) throws IOException; + + /** + * Return the existing upload keys which includes volumeName, bucketName, + * keyName and uploadId. Review comment: Upload is not passed. Need to update javadoc. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305713) Time Spent: 4.5h (was: 4h 20m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 4.5h > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305711=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305711 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 03/Sep/19 17:22 Start Date: 03/Sep/19 17:22 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#discussion_r320387656 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMMetadataManager.java ## @@ -327,4 +327,11 @@ String getMultipartKey(String volume, String bucket, String key, String */ long countEstimatedRowsInTable(Table table) throws IOException; + + /** + * Return the existing upload keys which includes volumeName, bucketName, Review comment: Return the in-flight Multipart uploads for a specified volumeName and bucketName which starts with prefix. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305711) Time Spent: 4h 20m (was: 4h 10m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305710=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305710 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 03/Sep/19 17:22 Start Date: 03/Sep/19 17:22 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#discussion_r320387656 ## File path: hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMMetadataManager.java ## @@ -327,4 +327,11 @@ String getMultipartKey(String volume, String bucket, String key, String */ long countEstimatedRowsInTable(Table table) throws IOException; + + /** + * Return the existing upload keys which includes volumeName, bucketName, Review comment: Return the in-flight Multipart uploads for a specified volumeName and bucketName which starts with prefix. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305710) Time Spent: 4h 10m (was: 4h) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305704=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305704
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 03/Sep/19 17:16
Start Date: 03/Sep/19 17:16
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r320385171
##
File path:
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneMultipartUploadList.java
##
@@ -0,0 +1,44 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.client;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * List of in-flight MU upoads.
+ */
+public class OzoneMultipartUploadList {
+
+ private List uploads = new ArrayList<>();
Review comment:
Unneeded initialization.
As, we are setting uploads with value passed to constructor.
Do we need to have Not null check here below?
```
public OzoneMultipartUploadList(
List uploads) {
this.uploads = uploads;
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 305704)
Time Spent: 4h (was: 3h 50m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305701=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305701 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 03/Sep/19 17:11 Start Date: 03/Sep/19 17:11 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#discussion_r320383068 ## File path: hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneMultipartUploadList.java ## @@ -0,0 +1,44 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.ozone.client; + +import java.util.ArrayList; +import java.util.List; + +/** + * List of in-flight MU upoads. Review comment: Typo: Here do you mean Multipart Uploads? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305701) Time Spent: 3h 50m (was: 3h 40m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=305369=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-305369 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 03/Sep/19 02:06 Start Date: 03/Sep/19 02:06 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-527276378 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 39 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for branch | | +1 | mvninstall | 614 | trunk passed | | +1 | compile | 388 | trunk passed | | +1 | checkstyle | 99 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 922 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 178 | trunk passed | | 0 | spotbugs | 431 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 639 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 26 | Maven dependency ordering for patch | | +1 | mvninstall | 561 | the patch passed | | +1 | compile | 423 | the patch passed | | +1 | cc | 423 | the patch passed | | +1 | javac | 423 | the patch passed | | +1 | checkstyle | 83 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 778 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 192 | the patch passed | | +1 | findbugs | 770 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 310 | hadoop-hdds in the patch passed. | | -1 | unit | 2359 | hadoop-ozone in the patch failed. | | +1 | asflicense | 56 | The patch does not generate ASF License warnings. | | | | 8614 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.container.common.statemachine.commandhandler.TestBlockDeletion | | | hadoop.ozone.client.rpc.TestContainerStateMachineFailures | | | hadoop.ozone.client.rpc.TestCommitWatcher | | | hadoop.ozone.om.TestSecureOzoneManager | | | hadoop.ozone.client.rpc.TestWatchForCommit | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/10/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 371835c48863 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 915cbc9 | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/10/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/10/testReport/ | | Max. process+thread count | 4430 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/10/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 305369) Time Spent: 3h 40m (was: 3.5h) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker >
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=303723=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303723 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 29/Aug/19 13:57 Start Date: 29/Aug/19 13:57 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-526197265 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 34 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 12 | Maven dependency ordering for branch | | +1 | mvninstall | 597 | trunk passed | | +1 | compile | 383 | trunk passed | | +1 | checkstyle | 68 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 917 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 167 | trunk passed | | 0 | spotbugs | 470 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 701 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 22 | Maven dependency ordering for patch | | +1 | mvninstall | 593 | the patch passed | | +1 | compile | 394 | the patch passed | | +1 | cc | 394 | the patch passed | | +1 | javac | 394 | the patch passed | | +1 | checkstyle | 81 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 756 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 171 | the patch passed | | +1 | findbugs | 639 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 303 | hadoop-hdds in the patch passed. | | -1 | unit | 1591 | hadoop-ozone in the patch failed. | | +1 | asflicense | 45 | The patch does not generate ASF License warnings. | | | | 7646 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.TestOzoneConfigurationFields | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/9/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 376784018ab2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 8c0759d | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/9/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/9/testReport/ | | Max. process+thread count | 5019 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/9/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303723) Time Spent: 3.5h (was: 3h 20m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=303567=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303567 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 29/Aug/19 10:17 Start Date: 29/Aug/19 10:17 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-52616 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 76 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 14 | Maven dependency ordering for branch | | +1 | mvninstall | 616 | trunk passed | | +1 | compile | 386 | trunk passed | | +1 | checkstyle | 69 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 926 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 175 | trunk passed | | 0 | spotbugs | 471 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 709 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 21 | Maven dependency ordering for patch | | +1 | mvninstall | 610 | the patch passed | | +1 | compile | 409 | the patch passed | | +1 | cc | 409 | the patch passed | | +1 | javac | 409 | the patch passed | | +1 | checkstyle | 95 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 808 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 198 | the patch passed | | +1 | findbugs | 685 | the patch passed | ||| _ Other Tests _ | | +1 | unit | 374 | hadoop-hdds in the patch passed. | | -1 | unit | 262 | hadoop-ozone in the patch failed. | | +1 | asflicense | 40 | The patch does not generate ASF License warnings. | | | | 6658 | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.ozone.om.TestKeyManagerUnit | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/8/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux fc03450f9b59 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / c749f62 | | Default Java | 1.8.0_222 | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/8/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/8/testReport/ | | Max. process+thread count | 1237 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/8/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303567) Time Spent: 3h 20m (was: 3h 10m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 3h 20m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] >
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=303183=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303183 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 28/Aug/19 19:19 Start Date: 28/Aug/19 19:19 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-525885449 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 0 | Docker mode activated. | | -1 | patch | 10 | https://github.com/apache/hadoop/pull/1277 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/7/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303183) Time Spent: 3h 10m (was: 3h) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=303180=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303180 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 28/Aug/19 19:16 Start Date: 28/Aug/19 19:16 Worklog Time Spent: 10m Work Description: anuengineer commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-525884481 We need to rebase this patch since we have removed the Rest Client. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303180) Time Spent: 3h (was: 2h 50m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=303075=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-303075 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 28/Aug/19 18:17 Start Date: 28/Aug/19 18:17 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-525862802 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 0 | Docker mode activated. | | -1 | patch | 10 | https://github.com/apache/hadoop/pull/1277 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/6/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 303075) Time Spent: 2h 50m (was: 2h 40m) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=302425=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-302425 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 27/Aug/19 21:49 Start Date: 27/Aug/19 21:49 Worklog Time Spent: 10m Work Description: elek commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-525496981 > In the list, we have different parameters like key-marker, max-uploads and few other parameters. Are we planning to do this in a new Jira? Yes. I pagination is not yet implemented. This is the minimal implementation to support docker registry. Will create an other jira to add all the pagination magic. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 302425) Time Spent: 2h 40m (was: 2.5h) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=300080=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-300080 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 23/Aug/19 06:28 Start Date: 23/Aug/19 06:28 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-523030761 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 115 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 32 | Maven dependency ordering for branch | | +1 | mvninstall | 767 | trunk passed | | +1 | compile | 445 | trunk passed | | +1 | checkstyle | 86 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 1129 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 201 | trunk passed | | 0 | spotbugs | 554 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 828 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 29 | Maven dependency ordering for patch | | +1 | mvninstall | 724 | the patch passed | | +1 | compile | 443 | the patch passed | | +1 | cc | 443 | the patch passed | | +1 | javac | 443 | the patch passed | | +1 | checkstyle | 76 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | shadedclient | 827 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 216 | the patch passed | | -1 | findbugs | 546 | hadoop-ozone generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 433 | hadoop-hdds in the patch passed. | | -1 | unit | 303 | hadoop-ozone in the patch failed. | | +1 | asflicense | 46 | The patch does not generate ASF License warnings. | | | | 7713 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone | | | Dead store to multipartKey in org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:[line 1313] | | Failed junit tests | hadoop.ozone.om.ratis.TestOzoneManagerRatisServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux eecbd90521e2 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 6244502 | | Default Java | 1.8.0_222 | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/artifact/out/whitespace-eol.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/artifact/out/new-findbugs-hadoop-ozone.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/testReport/ | | Max. process+thread count | 1295 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ---
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=300079=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-300079 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 23/Aug/19 06:28 Start Date: 23/Aug/19 06:28 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-522048099 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 41 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 11 | Maven dependency ordering for branch | | +1 | mvninstall | 591 | trunk passed | | +1 | compile | 353 | trunk passed | | +1 | checkstyle | 59 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 785 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 149 | trunk passed | | 0 | spotbugs | 427 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 622 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 547 | the patch passed | | +1 | compile | 355 | the patch passed | | +1 | cc | 355 | the patch passed | | +1 | javac | 355 | the patch passed | | +1 | checkstyle | 64 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | shadedclient | 626 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 149 | the patch passed | | -1 | findbugs | 444 | hadoop-ozone generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 309 | hadoop-hdds in the patch passed. | | -1 | unit | 170 | hadoop-ozone in the patch failed. | | +1 | asflicense | 34 | The patch does not generate ASF License warnings. | | | | 5670 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone | | | Dead store to multipartKey in org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:[line 1313] | | Failed junit tests | hadoop.ozone.om.ratis.TestOzoneManagerRatisServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 08d3de8a4e7f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 9b8359b | | Default Java | 1.8.0_222 | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/artifact/out/whitespace-eol.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/artifact/out/new-findbugs-hadoop-ozone.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/testReport/ | | Max. process+thread count | 1371 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ---
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=300081=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-300081 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 23/Aug/19 06:28 Start Date: 23/Aug/19 06:28 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-523397374 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 63 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 28 | Maven dependency ordering for branch | | +1 | mvninstall | 600 | trunk passed | | +1 | compile | 346 | trunk passed | | +1 | checkstyle | 64 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 848 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 158 | trunk passed | | 0 | spotbugs | 433 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 636 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 19 | Maven dependency ordering for patch | | +1 | mvninstall | 581 | the patch passed | | +1 | compile | 374 | the patch passed | | +1 | cc | 374 | the patch passed | | +1 | javac | 374 | the patch passed | | -0 | checkstyle | 37 | hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 661 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 153 | the patch passed | | -1 | findbugs | 441 | hadoop-ozone generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 383 | hadoop-hdds in the patch passed. | | -1 | unit | 329 | hadoop-ozone in the patch failed. | | +1 | asflicense | 34 | The patch does not generate ASF License warnings. | | | | 6160 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone | | | Dead store to multipartKey in org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:[line 1313] | | Failed junit tests | hadoop.ozone.om.ratis.TestOzoneManagerRatisServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux a8ec28791adb 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 10ec31d | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/artifact/out/diff-checkstyle-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/artifact/out/new-findbugs-hadoop-ozone.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/testReport/ | | Max. process+thread count | 1354 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id:
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=300078=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-300078 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 23/Aug/19 06:28 Start Date: 23/Aug/19 06:28 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-520231221 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 73 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 30 | Maven dependency ordering for branch | | +1 | mvninstall | 599 | trunk passed | | +1 | compile | 371 | trunk passed | | +1 | checkstyle | 73 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 997 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 164 | trunk passed | | 0 | spotbugs | 432 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 629 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 21 | Maven dependency ordering for patch | | +1 | mvninstall | 558 | the patch passed | | +1 | compile | 371 | the patch passed | | +1 | cc | 371 | the patch passed | | +1 | javac | 371 | the patch passed | | +1 | checkstyle | 77 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | shadedclient | 740 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 166 | the patch passed | | -1 | findbugs | 443 | hadoop-ozone generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 347 | hadoop-hdds in the patch passed. | | -1 | unit | 225 | hadoop-ozone in the patch failed. | | +1 | asflicense | 39 | The patch does not generate ASF License warnings. | | | | 6300 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone | | | Dead store to multipartKey in org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:[line 1312] | | Failed junit tests | hadoop.ozone.om.ratis.TestOzoneManagerRatisServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 6ec7387069c5 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / cf5d895 | | Default Java | 1.8.0_222 | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/artifact/out/whitespace-eol.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/artifact/out/new-findbugs-hadoop-ozone.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/testReport/ | | Max. process+thread count | 1272 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ---
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=298575=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-298575 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 21/Aug/19 10:26 Start Date: 21/Aug/19 10:26 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-523397374 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 63 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 28 | Maven dependency ordering for branch | | +1 | mvninstall | 600 | trunk passed | | +1 | compile | 346 | trunk passed | | +1 | checkstyle | 64 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 848 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 158 | trunk passed | | 0 | spotbugs | 433 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 636 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 19 | Maven dependency ordering for patch | | +1 | mvninstall | 581 | the patch passed | | +1 | compile | 374 | the patch passed | | +1 | cc | 374 | the patch passed | | +1 | javac | 374 | the patch passed | | -0 | checkstyle | 37 | hadoop-ozone: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 | mvnsite | 0 | the patch passed | | +1 | whitespace | 0 | The patch has no whitespace issues. | | +1 | shadedclient | 661 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 153 | the patch passed | | -1 | findbugs | 441 | hadoop-ozone generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 383 | hadoop-hdds in the patch passed. | | -1 | unit | 329 | hadoop-ozone in the patch failed. | | +1 | asflicense | 34 | The patch does not generate ASF License warnings. | | | | 6160 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone | | | Dead store to multipartKey in org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:[line 1313] | | Failed junit tests | hadoop.ozone.om.ratis.TestOzoneManagerRatisServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux a8ec28791adb 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 10ec31d | | Default Java | 1.8.0_222 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/artifact/out/diff-checkstyle-hadoop-ozone.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/artifact/out/new-findbugs-hadoop-ozone.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/testReport/ | | Max. process+thread count | 1354 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/4/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id:
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=298505=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-298505
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 21/Aug/19 08:04
Start Date: 21/Aug/19 08:04
Worklog Time Spent: 10m
Work Description: elek commented on pull request #1277: HDDS-1054. List
Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r316047094
##
File path:
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadCompleteList.java
##
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.Part;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.TreeMap;
+
+/**
+ * This class represents multipart list, which is required for
+ * CompleteMultipart upload request.
+ */
+public class OmMultipartUploadCompleteList {
+
+ private final TreeMap multipartMap;
+
+ /**
+ * Construct OmMultipartUploadList which holds multipart map which contains
Review comment:
Thanks a lot, fixed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 298505)
Time Spent: 1h 40m (was: 1.5h)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=297901=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-297901
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 20/Aug/19 14:05
Start Date: 20/Aug/19 14:05
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r315708896
##
File path: hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
##
@@ -200,3 +200,24 @@ Test Multipart Upload with the simplified aws s3 cp API
Execute AWSS3Clicp s3://${BUCKET}/mpyawscli
/tmp/part1.result
Execute AWSS3Clirm s3://${BUCKET}/mpyawscli
Compare files /tmp/part1
/tmp/part1.result
+
+Test Multipart Upload list
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key1
+${uploadID1} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key1
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key2
+${uploadID2} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key2
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli list-multipart-uploads
--bucket ${BUCKET} --prefix listtest
+Should contain ${result}${uploadID1}
+Should contain ${result}${uploadID2}
+
+${count} = Execute and checkrc echo '${result}' | jq -r
'.Uploads | length' 0
Review comment:
whitespace:end of line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 297901)
Time Spent: 1h 20m (was: 1h 10m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=297902=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-297902 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 20/Aug/19 14:05 Start Date: 20/Aug/19 14:05 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-523030761 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 115 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 32 | Maven dependency ordering for branch | | +1 | mvninstall | 767 | trunk passed | | +1 | compile | 445 | trunk passed | | +1 | checkstyle | 86 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 1129 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 201 | trunk passed | | 0 | spotbugs | 554 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 828 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 29 | Maven dependency ordering for patch | | +1 | mvninstall | 724 | the patch passed | | +1 | compile | 443 | the patch passed | | +1 | cc | 443 | the patch passed | | +1 | javac | 443 | the patch passed | | +1 | checkstyle | 76 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | shadedclient | 827 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 216 | the patch passed | | -1 | findbugs | 546 | hadoop-ozone generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 433 | hadoop-hdds in the patch passed. | | -1 | unit | 303 | hadoop-ozone in the patch failed. | | +1 | asflicense | 46 | The patch does not generate ASF License warnings. | | | | 7713 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone | | | Dead store to multipartKey in org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:[line 1313] | | Failed junit tests | hadoop.ozone.om.ratis.TestOzoneManagerRatisServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux eecbd90521e2 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 6244502 | | Default Java | 1.8.0_222 | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/artifact/out/whitespace-eol.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/artifact/out/new-findbugs-hadoop-ozone.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/testReport/ | | Max. process+thread count | 1295 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/3/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ---
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=296888=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-296888 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 18/Aug/19 05:34 Start Date: 18/Aug/19 05:34 Worklog Time Spent: 10m Work Description: bharatviswa504 commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-522292705 In the list, we have different parameters like key-marker, max-uploads and few other parameters. Are we planning to do this in a new Jira? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 296888) Time Spent: 1h 10m (was: 1h) > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=296887=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-296887
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 18/Aug/19 05:31
Start Date: 18/Aug/19 05:31
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r314968679
##
File path:
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/helpers/OmMultipartUploadCompleteList.java
##
@@ -0,0 +1,63 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.ozone.om.helpers;
+
+import org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos.Part;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.TreeMap;
+
+/**
+ * This class represents multipart list, which is required for
+ * CompleteMultipart upload request.
+ */
+public class OmMultipartUploadCompleteList {
+
+ private final TreeMap multipartMap;
+
+ /**
+ * Construct OmMultipartUploadList which holds multipart map which contains
Review comment:
OmMultipartUploadList -> OmMultipartUploadCompleteList
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 296887)
Time Spent: 1h (was: 50m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=296335=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-296335 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 16/Aug/19 15:22 Start Date: 16/Aug/19 15:22 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-522048099 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 41 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 11 | Maven dependency ordering for branch | | +1 | mvninstall | 591 | trunk passed | | +1 | compile | 353 | trunk passed | | +1 | checkstyle | 59 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 785 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 149 | trunk passed | | 0 | spotbugs | 427 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 622 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 16 | Maven dependency ordering for patch | | +1 | mvninstall | 547 | the patch passed | | +1 | compile | 355 | the patch passed | | +1 | cc | 355 | the patch passed | | +1 | javac | 355 | the patch passed | | +1 | checkstyle | 64 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | shadedclient | 626 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 149 | the patch passed | | -1 | findbugs | 444 | hadoop-ozone generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 309 | hadoop-hdds in the patch passed. | | -1 | unit | 170 | hadoop-ozone in the patch failed. | | +1 | asflicense | 34 | The patch does not generate ASF License warnings. | | | | 5670 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone | | | Dead store to multipartKey in org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:[line 1313] | | Failed junit tests | hadoop.ozone.om.ratis.TestOzoneManagerRatisServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 08d3de8a4e7f 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 9b8359b | | Default Java | 1.8.0_222 | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/artifact/out/whitespace-eol.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/artifact/out/new-findbugs-hadoop-ozone.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/testReport/ | | Max. process+thread count | 1371 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ---
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=296334=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-296334
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 16/Aug/19 15:22
Start Date: 16/Aug/19 15:22
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r314769222
##
File path: hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
##
@@ -200,3 +200,24 @@ Test Multipart Upload with the simplified aws s3 cp API
Execute AWSS3Clicp s3://${BUCKET}/mpyawscli
/tmp/part1.result
Execute AWSS3Clirm s3://${BUCKET}/mpyawscli
Compare files /tmp/part1
/tmp/part1.result
+
+Test Multipart Upload list
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key1
+${uploadID1} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key1
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key2
+${uploadID2} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key2
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli list-multipart-uploads
--bucket ${BUCKET} --prefix listtest
+Should contain ${result}${uploadID1}
+Should contain ${result}${uploadID2}
+
+${count} = Execute and checkrc echo '${result}' | jq -r
'.Uploads | length' 0
Review comment:
whitespace:end of line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 296334)
Time Spent: 40m (was: 0.5h)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[
https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=292707=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-292707
]
ASF GitHub Bot logged work on HDDS-1054:
Author: ASF GitHub Bot
Created on: 11/Aug/19 14:07
Start Date: 11/Aug/19 14:07
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #1277:
HDDS-1054. List Multipart uploads in a bucket
URL: https://github.com/apache/hadoop/pull/1277#discussion_r312740496
##
File path: hadoop-ozone/dist/src/main/smoketest/s3/MultipartUpload.robot
##
@@ -200,3 +200,24 @@ Test Multipart Upload with the simplified aws s3 cp API
Execute AWSS3Clicp s3://${BUCKET}/mpyawscli
/tmp/part1.result
Execute AWSS3Clirm s3://${BUCKET}/mpyawscli
Compare files /tmp/part1
/tmp/part1.result
+
+Test Multipart Upload list
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key1
+${uploadID1} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key1
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli create-multipart-upload
--bucket ${BUCKET} --key listtest/key2
+${uploadID2} = Execute and checkrc echo '${result}' | jq -r
'.UploadId'0
+Should contain ${result}${BUCKET}
+Should contain ${result}listtest/key2
+Should contain ${result}UploadId
+
+${result} = Execute AWSS3APICli list-multipart-uploads
--bucket ${BUCKET} --prefix listtest
+Should contain ${result}${uploadID1}
+Should contain ${result}${uploadID2}
+
+${count} = Execute and checkrc echo '${result}' | jq -r
'.Uploads | length' 0
Review comment:
whitespace:end of line
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 292707)
Time Spent: 20m (was: 10m)
> List Multipart uploads in a bucket
> --
>
> Key: HDDS-1054
> URL: https://issues.apache.org/jira/browse/HDDS-1054
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Elek, Marton
>Priority: Blocker
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> This Jira is to implement in ozone to list of in-progress multipart uploads
> in a bucket.
> [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html]
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=292708=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-292708 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 11/Aug/19 14:07 Start Date: 11/Aug/19 14:07 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on issue #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277#issuecomment-520231221 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 73 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 1 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | 0 | mvndep | 30 | Maven dependency ordering for branch | | +1 | mvninstall | 599 | trunk passed | | +1 | compile | 371 | trunk passed | | +1 | checkstyle | 73 | trunk passed | | +1 | mvnsite | 0 | trunk passed | | +1 | shadedclient | 997 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 164 | trunk passed | | 0 | spotbugs | 432 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 629 | trunk passed | ||| _ Patch Compile Tests _ | | 0 | mvndep | 21 | Maven dependency ordering for patch | | +1 | mvninstall | 558 | the patch passed | | +1 | compile | 371 | the patch passed | | +1 | cc | 371 | the patch passed | | +1 | javac | 371 | the patch passed | | +1 | checkstyle | 77 | the patch passed | | +1 | mvnsite | 0 | the patch passed | | -1 | whitespace | 0 | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 | shadedclient | 740 | patch has no errors when building and testing our client artifacts. | | +1 | javadoc | 166 | the patch passed | | -1 | findbugs | 443 | hadoop-ozone generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | ||| _ Other Tests _ | | +1 | unit | 347 | hadoop-hdds in the patch passed. | | -1 | unit | 225 | hadoop-ozone in the patch failed. | | +1 | asflicense | 39 | The patch does not generate ASF License warnings. | | | | 6300 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-ozone | | | Dead store to multipartKey in org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:org.apache.hadoop.ozone.om.KeyManagerImpl.listMultipartUploads(String, String, String) At KeyManagerImpl.java:[line 1312] | | Failed junit tests | hadoop.ozone.om.ratis.TestOzoneManagerRatisServer | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1277 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc | | uname | Linux 6ec7387069c5 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / cf5d895 | | Default Java | 1.8.0_222 | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/artifact/out/whitespace-eol.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/artifact/out/new-findbugs-hadoop-ozone.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/artifact/out/patch-unit-hadoop-ozone.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/testReport/ | | Max. process+thread count | 1272 (vs. ulimit of 5500) | | modules | C: hadoop-ozone/common hadoop-ozone/client hadoop-ozone/ozone-manager hadoop-ozone/s3gateway hadoop-ozone/dist U: hadoop-ozone | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1277/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ---
[jira] [Work logged] (HDDS-1054) List Multipart uploads in a bucket
[ https://issues.apache.org/jira/browse/HDDS-1054?focusedWorklogId=292694=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-292694 ] ASF GitHub Bot logged work on HDDS-1054: Author: ASF GitHub Bot Created on: 11/Aug/19 12:21 Start Date: 11/Aug/19 12:21 Worklog Time Spent: 10m Work Description: elek commented on pull request #1277: HDDS-1054. List Multipart uploads in a bucket URL: https://github.com/apache/hadoop/pull/1277 This Jira is to implement in ozone to list of in-progress multipart uploads in a bucket. [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] See: https://issues.apache.org/jira/browse/HDDS-1054 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 292694) Time Spent: 10m Remaining Estimate: 0h > List Multipart uploads in a bucket > -- > > Key: HDDS-1054 > URL: https://issues.apache.org/jira/browse/HDDS-1054 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Elek, Marton >Priority: Blocker > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > This Jira is to implement in ozone to list of in-progress multipart uploads > in a bucket. > [https://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html] > -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
