[jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
This adds a stop-gap async blobstore SubmissionAsyncBlobStore, which was copy/pasted from openstack-labs. This simulates what the async client did, basically defer the call to a future. Using this, I unasynced s3 and aws-s3, ran the live tests, which had the same results as before: s3 all pass, aws-s3 fails one test. You can merge this Pull Request by running: git pull https://github.com/adriancole/jclouds adrian.no-s3-async Or you can view, comment on it, or merge it online at: https://github.com/jclouds/jclouds/pull/538 -- Commit Summary -- * JCLOUDS-150 add SubmissionAsyncBlobStore; nasync s3 and aws-s3 -- File Changes -- M apis/s3/src/main/java/org/jclouds/s3/S3ApiMetadata.java (57) D apis/s3/src/main/java/org/jclouds/s3/S3AsyncClient.java (363) M apis/s3/src/main/java/org/jclouds/s3/S3Client.java (302) D apis/s3/src/main/java/org/jclouds/s3/blobstore/S3AsyncBlobStore.java (273) M apis/s3/src/main/java/org/jclouds/s3/blobstore/S3BlobRequestSigner.java (4) M apis/s3/src/main/java/org/jclouds/s3/blobstore/S3BlobStoreContext.java (3) M apis/s3/src/main/java/org/jclouds/s3/blobstore/config/S3BlobStoreContextModule.java (11) M apis/s3/src/main/java/org/jclouds/s3/blobstore/internal/S3BlobStoreContextImpl.java (7) R apis/s3/src/main/java/org/jclouds/s3/config/S3HttpApiModule.java (44) M apis/s3/src/test/java/org/jclouds/s3/S3ClientLiveTest.java (8) R apis/s3/src/test/java/org/jclouds/s3/S3ClientTest.java (64) M apis/s3/src/test/java/org/jclouds/s3/binders/BindAsHostPrefixIfConfiguredNoPathTest.java (8) M apis/s3/src/test/java/org/jclouds/s3/binders/BindAsHostPrefixIfConfiguredTest.java (8) M apis/s3/src/test/java/org/jclouds/s3/binders/BindNoBucketLoggingToXmlPayloadTest.java (6) M apis/s3/src/test/java/org/jclouds/s3/binders/BindS3ObjectMetadataToRequestTest.java (6) M apis/s3/src/test/java/org/jclouds/s3/blobstore/S3BlobSignerExpectTest.java (7) M apis/s3/src/test/java/org/jclouds/s3/filters/RequestAuthorizeSignatureTest.java (16) M apis/s3/src/test/java/org/jclouds/s3/filters/RequestAuthorizeSignatureWithSessionCredentialsTest.java (9) M apis/s3/src/test/java/org/jclouds/s3/internal/BaseS3ClientExpectTest.java (14) R apis/s3/src/test/java/org/jclouds/s3/internal/BaseS3ClientTest.java (6) D apis/s3/src/test/java/org/jclouds/s3/internal/StubS3AsyncClient.java (343) M apis/s3/src/test/java/org/jclouds/s3/services/BucketsLiveTest.java (15) A blobstore/src/main/java/org/jclouds/blobstore/internal/SubmissionAsyncBlobStore.java (293) M providers/aws-s3/src/main/java/org/jclouds/aws/s3/AWSS3ApiMetadata.java (22) D providers/aws-s3/src/main/java/org/jclouds/aws/s3/AWSS3AsyncClient.java (135) M providers/aws-s3/src/main/java/org/jclouds/aws/s3/AWSS3Client.java (83) D providers/aws-s3/src/main/java/org/jclouds/aws/s3/blobstore/AWSS3AsyncBlobStore.java (139) M providers/aws-s3/src/main/java/org/jclouds/aws/s3/blobstore/AWSS3BlobRequestSigner.java (6) M providers/aws-s3/src/main/java/org/jclouds/aws/s3/blobstore/AWSS3BlobStore.java (4) M providers/aws-s3/src/main/java/org/jclouds/aws/s3/blobstore/AWSS3BlobStoreContext.java (3) M providers/aws-s3/src/main/java/org/jclouds/aws/s3/blobstore/config/AWSS3BlobStoreContextModule.java (3) M providers/aws-s3/src/main/java/org/jclouds/aws/s3/blobstore/internal/AWSS3BlobStoreContextImpl.java (7) M providers/aws-s3/src/main/java/org/jclouds/aws/s3/blobstore/strategy/internal/ParallelMultipartUploadStrategy.java (31) R providers/aws-s3/src/main/java/org/jclouds/aws/s3/config/AWSS3HttpApiModule.java (17) M providers/aws-s3/src/test/java/org/jclouds/aws/s3/AWSS3ClientLiveTest.java (4) R providers/aws-s3/src/test/java/org/jclouds/aws/s3/AWSS3ClientTest.java (33) M providers/aws-s3/src/test/java/org/jclouds/aws/s3/binders/BindObjectMetadataToRequestTest.java (6) M providers/aws-s3/src/test/java/org/jclouds/aws/s3/blobstore/AWSS3BlobSignerExpectTest.java (6) M providers/aws-s3/src/test/java/org/jclouds/aws/s3/internal/BaseAWSS3ClientExpectTest.java (6) -- Patch Links -- https://github.com/jclouds/jclouds/pull/538.patch https://github.com/jclouds/jclouds/pull/538.diff --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
[jclouds-pull-requests-java-6 #116](https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests-java-6/116/) SUCCESS This pull request looks good --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57817439
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
[jclouds-pull-requests-java-6 #117](https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests-java-6/117/) SUCCESS This pull request looks good --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57817531
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
[jclouds-pull-requests #1205](https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests/1205/) SUCCESS This pull request looks good --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57824864
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
[jclouds-pull-requests #1206](https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests/1206/) ABORTED --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57829889
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
import org.jclouds.s3.reference.S3Headers; -import com.google.common.collect.ImmutableSet; -import com.google.common.reflect.TypeToken; -import com.google.inject.Module; +import static org.jclouds.Constants.PROPERTY_RELAX_HOSTNAME; +import static org.jclouds.aws.reference.AWSConstants.PROPERTY_AUTH_TAG; +import static org.jclouds.aws.reference.AWSConstants.PROPERTY_HEADER_TAG; +import static org.jclouds.blobstore.reference.BlobStoreConstants.PROPERTY_BLOBSTORE_DIRECTORY_SUFFIX; +import static org.jclouds.blobstore.reference.BlobStoreConstants.PROPERTY_USER_METADATA_PREFIX; +import static org.jclouds.reflect.Reflection2.typeToken; +import static org.jclouds.s3.reference.S3Constants.PROPERTY_S3_SERVICE_PATH; +import static org.jclouds.s3.reference.S3Constants.PROPERTY_S3_VIRTUAL_HOST_BUCKETS; Static imports at the top of the file please. :) Here is the standard format for ordering imports: - `static` - `java` - `javax` - `org` - `com` --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18408829
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
- ListenableFutureVoid enableBucketLogging( -@Bucket @EndpointParam(parser = AssignCorrectHostnameForBucket.class) @BinderParam(BindAsHostPrefixIfConfigured.class) @ParamValidators(BucketNameValidator.class) String bucketName, -@BinderParam(BindBucketLoggingToXmlPayload.class) BucketLogging logging); - - /** -* @see S3Client#putBucketLogging -*/ - @Named(PutBucketLogging) - @PUT - @Path(/) - @QueryParams(keys = logging) - @Produces(MediaType.TEXT_XML) - ListenableFutureVoid disableBucketLogging( -@Bucket @EndpointParam(parser = AssignCorrectHostnameForBucket.class) @BinderParam(BindNoBucketLoggingToXmlPayload.class) @ParamValidators(BucketNameValidator.class) String bucketName); - -} Its so rewarding when these can be removed! --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18408871
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
import org.jclouds.s3.reference.S3Headers; -import com.google.common.collect.ImmutableSet; -import com.google.common.reflect.TypeToken; -import com.google.inject.Module; +import static org.jclouds.Constants.PROPERTY_RELAX_HOSTNAME; +import static org.jclouds.aws.reference.AWSConstants.PROPERTY_AUTH_TAG; +import static org.jclouds.aws.reference.AWSConstants.PROPERTY_HEADER_TAG; +import static org.jclouds.blobstore.reference.BlobStoreConstants.PROPERTY_BLOBSTORE_DIRECTORY_SUFFIX; +import static org.jclouds.blobstore.reference.BlobStoreConstants.PROPERTY_USER_METADATA_PREFIX; +import static org.jclouds.reflect.Reflection2.typeToken; +import static org.jclouds.s3.reference.S3Constants.PROPERTY_S3_SERVICE_PATH; +import static org.jclouds.s3.reference.S3Constants.PROPERTY_S3_VIRTUAL_HOST_BUCKETS; sure --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18408897
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
/** * Provides access to S3 via their REST API. - * p/ - * All commands return a Future of the result from S3. Any exceptions incurred during - * processing will be backend in an {@link ExecutionException} as documented in - * {@link Future#get()}. - * + * * @see a href=http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTAPI.html; / External doc links should be removed. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18408896
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
/** * Retrieves the {@link org.jclouds.s3.domain.internal.BucketListObjectMetadata metadata} of -* the object associated with the key or -* {@link org.jclouds.s3.domain.internal.BucketListObjectMetadata#NOT_FOUND} if not +* the object associated with the key or null if not * available. Pull available up to previous line? --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18408946
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
/** * Retrieves the {@link org.jclouds.s3.domain.internal.BucketListObjectMetadata metadata} of -* the object associated with the key or -* {@link org.jclouds.s3.domain.internal.BucketListObjectMetadata#NOT_FOUND} if not +* the object associated with the key or null if not * available. sure. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18409040
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
/** * Provides access to S3 via their REST API. - * p/ - * All commands return a Future of the result from S3. Any exceptions incurred during - * processing will be backend in an {@link ExecutionException} as documented in - * {@link Future#get()}. - * + * * @see a href=http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTAPI.html; / will do. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18409026
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
*/ - S3Object getObject(String bucketName, String key, GetOptions... options); + @Named(GetObject) [minor] FWIW `@Named(apiName:methodName)` has been an effective naming convention. In this case: `@Named(s3:getObject)` --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18409383
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
@@ -26,7 +26,4 @@ @Override S3BlobStore getBlobStore(); - - @Override - S3AsyncBlobStore getAsyncBlobStore(); Nice! --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18411399
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
public abstract class BaseS3ClientExpectTest extends BaseRestClientExpectTestS3Client { protected static final String CONSTANT_DATE = 2009-11-08T15:54:08.897Z; - @ConfiguresRestClient - private static final class TestS3RestClientModule extends S3RestClientModuleS3Client, S3AsyncClient { + @ConfiguresRestClient I take it that this will go the way of the dodo once all of the `*Client` classes are converted to `*Api`? Meaning, we will primarily use `@ConfiguresHttpApi` moving forward. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18412131
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
} private void addGrantsToACL(AccessControlList acl) { String ownerId = acl.getOwner().getId(); acl.addPermission(GroupGranteeURI.ALL_USERS, Permission.READ); - acl.addPermission(new EmailAddressGrantee(StubS3AsyncClient.TEST_ACL_EMAIL), Permission.READ_ACP); + acl.addPermission(new EmailAddressGrantee(TEST_ACL_EMAIL), Permission.READ_ACP); static import `Permission.READ_ACP` to follow style of `TEST_ACL_EMAIL`? --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18412282
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
[jclouds » jclouds #1686](https://buildhive.cloudbees.com/job/jclouds/job/jclouds/1686/) ABORTED [(what's this?)](https://www.cloudbees.com/what-is-buildhive) --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57837219
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
+import org.jclouds.blobstore.domain.StorageMetadata; +import org.jclouds.blobstore.options.CreateContainerOptions; +import org.jclouds.blobstore.options.GetOptions; +import org.jclouds.blobstore.options.ListContainerOptions; +import org.jclouds.blobstore.options.PutOptions; +import org.jclouds.domain.Location; + +import com.google.common.collect.ForwardingObject; +import com.google.common.util.concurrent.ListenableFuture; +import com.google.common.util.concurrent.ListeningExecutorService; + +/** + * Adapter that allows you to reuse an existing {@link BlobStore} to implement + * the deprecated {@link AsyncBlobStore} interface. + * + * @deprecated will be removed in jclouds 1.7, as async interfaces are no longer In an effort to be mindful of deprecation warnings, this should be updated to `2.0`. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18412485
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
+import org.jclouds.blobstore.options.PutOptions; +import org.jclouds.domain.Location; + +import com.google.common.collect.ForwardingObject; +import com.google.common.util.concurrent.ListenableFuture; +import com.google.common.util.concurrent.ListeningExecutorService; + +/** + * Adapter that allows you to reuse an existing {@link BlobStore} to implement + * the deprecated {@link AsyncBlobStore} interface. + * + * @deprecated will be removed in jclouds 1.7, as async interfaces are no longer + * supported. Please use {@link BlobStore} + */ +@Deprecated +public class SubmissionAsyncBlobStore extends ForwardingObject implements AsyncBlobStore { I argue that we should move this class into the `blobstore` project so that `openstack-swift` can use it as well. WDYT? --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538/files#r18412680
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
+1 Looks good based on the other conversions I have done. Just a few minor comments, thanks @adriancole ! --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57838532
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
[jclouds » jclouds #1687](https://buildhive.cloudbees.com/job/jclouds/job/jclouds/1687/) SUCCESS This pull request looks good [(what's this?)](https://www.cloudbees.com/what-is-buildhive) --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57845969
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
addressed all feedback and re-ran live tests. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57851980
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
[jclouds-pull-requests-java-6 #118](https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests-java-6/118/) SUCCESS This pull request looks good --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57852319
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
[jclouds » jclouds #1688](https://buildhive.cloudbees.com/job/jclouds/job/jclouds/1688/) SUCCESS This pull request looks good [(what's this?)](https://www.cloudbees.com/what-is-buildhive) --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57861528
Re: [jclouds] JCLOUDS-150 add SubmissionAsyncBlobStore; unasync s3 and aws-s3 (#538)
[jclouds-pull-requests #1207](https://jclouds.ci.cloudbees.com/job/jclouds-pull-requests/1207/) SUCCESS This pull request looks good --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/538#issuecomment-57863034