Build failed in Jenkins: jclouds-guava-17-18 » 18.0,OpenJDK 7 (latest) #31
See https://jclouds.ci.cloudbees.com/job/jclouds-guava-17-18/GUAVA_VERSION=18.0,jdk=OpenJDK%207%20(latest)/31/changes Changes: [Andrew Gaul] make xattr work in docker volume -- [...truncated 3874 lines...] at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) ERROR: , completed: 0/2, errors: 1, rate: 0ms/op java.util.concurrent.ExecutionException: java.lang.RuntimeException at com.google.common.util.concurrent.Futures$ImmediateFailedFuture.get(Futures.java:190) at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:123) at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299) at com.google.common.util.concurrent.Futures$ImmediateFuture.addListener(Futures.java:106) at org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:118) at org.jclouds.concurrent.FutureIterables.transformParallel(FutureIterables.java:86) at org.jclouds.concurrent.FutureIterables.transformParallel(FutureIterables.java:70) at org.jclouds.concurrent.FutureIterablesTest.testNormalExceptionPropagatesAsTransformParallelExceptionAndTries5XPerElement(FutureIterablesTest.java:68) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84) at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.RuntimeException at org.jclouds.concurrent.FutureIterablesTest$2.apply(FutureIterablesTest.java:71) at org.jclouds.concurrent.FutureIterablesTest$2.apply(FutureIterablesTest.java:68) at org.jclouds.concurrent.FutureIterables.transformParallel(FutureIterables.java:82) ... 15 more ERROR: , completed: 0/2, errors: 2, rate: 1ms/op java.util.concurrent.ExecutionException: java.lang.RuntimeException at com.google.common.util.concurrent.Futures$ImmediateFailedFuture.get(Futures.java:190) at org.jclouds.concurrent.FutureIterables$1.run(FutureIterables.java:123) at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299) at com.google.common.util.concurrent.Futures$ImmediateFuture.addListener(Futures.java:106) at org.jclouds.concurrent.FutureIterables.awaitCompletion(FutureIterables.java:118) at org.jclouds.concurrent.FutureIterables.transformParallel(FutureIterables.java:86) at org.jclouds.concurrent.FutureIterables.transformParallel(FutureIterables.java:70) at org.jclouds.concurrent.FutureIterablesTest.testNormalExceptionPropagatesAsTransformParallelExceptionAndTries5XPerElement(FutureIterablesTest.java:68) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84) at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at
[jclouds-labs] JCLOUDS-702: JCloud ProfitBricks provider - Server Image API (#119)
Hi! This PR is the implementation of ProfitBricks' API for: * **Server** - [live test](https://gist.github.com/devcsrj/6a17a3c1347f740b0715) * **Image** - [live test](https://gist.github.com/devcsrj/e4ab53864b75afa9655b) As usual, looking forward to awesome code reviews! You can merge this Pull Request by running: git pull https://github.com/devcsrj/jclouds-labs master Or you can view, comment on it, or merge it online at: https://github.com/jclouds/jclouds-labs/pull/119 -- Commit Summary -- * JCLOUDS-702: JCloud ProfitBricks provider implementation * Merge remote-tracking branch 'upstream/master' * Image Api * Server api * Merge branch 'master' of github.com:jclouds/jclouds-labs -- File Changes -- M profitbricks/src/main/java/org/jclouds/profitbricks/ProfitBricksApi.java (8) M profitbricks/src/main/java/org/jclouds/profitbricks/binder/BaseProfitBricksRequestBinder.java (2) A profitbricks/src/main/java/org/jclouds/profitbricks/binder/server/CreateServerRequestBinder.java (57) A profitbricks/src/main/java/org/jclouds/profitbricks/binder/server/UpdateServerRequestBinder.java (56) M profitbricks/src/main/java/org/jclouds/profitbricks/compute/internal/ProvisioningStatusAware.java (2) M profitbricks/src/main/java/org/jclouds/profitbricks/compute/internal/ProvisioningStatusPollingPredicate.java (2) A profitbricks/src/main/java/org/jclouds/profitbricks/domain/AvailabilityZone.java (35) A profitbricks/src/main/java/org/jclouds/profitbricks/domain/Image.java (207) A profitbricks/src/main/java/org/jclouds/profitbricks/domain/OsType.java (30) A profitbricks/src/main/java/org/jclouds/profitbricks/domain/Server.java (424) A profitbricks/src/main/java/org/jclouds/profitbricks/domain/internal/ServerCommonProperties.java (49) A profitbricks/src/main/java/org/jclouds/profitbricks/features/ImageApi.java (65) A profitbricks/src/main/java/org/jclouds/profitbricks/features/ServerApi.java (164) M profitbricks/src/main/java/org/jclouds/profitbricks/http/parser/BaseProfitBricksResponseHandler.java (8) A profitbricks/src/main/java/org/jclouds/profitbricks/http/parser/RequestIdOnlyResponseHandler.java (49) A profitbricks/src/main/java/org/jclouds/profitbricks/http/parser/image/BaseImageResponseHandler.java (76) A profitbricks/src/main/java/org/jclouds/profitbricks/http/parser/image/ImageInfoResponseHandler.java (48) A profitbricks/src/main/java/org/jclouds/profitbricks/http/parser/image/ImageListResponseHandler.java (51) A profitbricks/src/main/java/org/jclouds/profitbricks/http/parser/server/BaseServerResponseHandler.java (75) A profitbricks/src/main/java/org/jclouds/profitbricks/http/parser/server/ServerIdOnlyResponseHandler.java (55) A profitbricks/src/main/java/org/jclouds/profitbricks/http/parser/server/ServerInfoResponseHandler.java (48) A profitbricks/src/main/java/org/jclouds/profitbricks/http/parser/server/ServerListResponseHandler.java (51) A profitbricks/src/test/java/org/jclouds/profitbricks/binder/server/CreateServerRequestBinderTest.java (65) A profitbricks/src/test/java/org/jclouds/profitbricks/binder/server/UpdateServerRequestBinderTest.java (64) M profitbricks/src/test/java/org/jclouds/profitbricks/features/DataCenterApiMockTest.java (2) A profitbricks/src/test/java/org/jclouds/profitbricks/features/ImageApiLiveTest.java (58) A profitbricks/src/test/java/org/jclouds/profitbricks/features/ImageApiMockTest.java (114) A profitbricks/src/test/java/org/jclouds/profitbricks/features/ServerApiLiveTest.java (139) A profitbricks/src/test/java/org/jclouds/profitbricks/features/ServerApiMockTest.java (256) A profitbricks/src/test/java/org/jclouds/profitbricks/http/parser/RequestIdOnlyResponseHandlerTest.java (76) M profitbricks/src/test/java/org/jclouds/profitbricks/http/parser/ServiceFaultResponseHandlerTest.java (4) M profitbricks/src/test/java/org/jclouds/profitbricks/http/parser/datacenter/DataCenterInfoResponseHandlerTest.java (3) A profitbricks/src/test/java/org/jclouds/profitbricks/http/parser/image/ImageInfoResponseHandlerTest.java (65) A profitbricks/src/test/java/org/jclouds/profitbricks/http/parser/image/ImageListResponseHandlerTest.java (183) A profitbricks/src/test/java/org/jclouds/profitbricks/http/parser/server/ServerIdOnlyResponseHandlerTest.java (40) A profitbricks/src/test/java/org/jclouds/profitbricks/http/parser/server/ServerInfoResponseHandlerTest.java (76) A profitbricks/src/test/java/org/jclouds/profitbricks/http/parser/server/ServerListResponseHandlerTest.java (97) A profitbricks/src/test/resources/datacenter/datacenter-not-found.xml (17) M profitbricks/src/test/resources/fault-404.xml (28) A profitbricks/src/test/resources/image/image-not-found.xml (17) A profitbricks/src/test/resources/image/image.xml (26) A
Re: [jclouds] Refactor blobstore filesystem tests for Mac OS X (#643)
@kahing Can you review this? --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/643#issuecomment-69805174
Re: [jclouds] Refactor blobstore filesystem tests for Mac OS X (#643)
Closed #643. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/643#event-218319306
[jclouds] Refactor blobstore filesystem tests for Mac OS X (#643)
This PR refactors the filesystem blobstore tests slightly for Mac OS. - Utilizes class/method level `dataProvider` and `dataProviderClass` in tests where necessary - (minor) 3-space indent cleanup `org.jclouds.util.TestUtils` - pulled up constants from `org.jclouds.filesystem.utils.TestUtils` - pulled up `@DataProvider` for Mac OS/Windows - pulled up `isWindowsOs()` method `org.jclouds.filesystem.integration.FilesystemBlobIntegrationTest` - pulled up description of Mac OS xattrs bug info to class level - added failing methods from `BaseBlobIntegrationTest` - now skips all tests on Mac OS @zack-shoylev Can you please run this on Windows for a sanity check? Thx! You can merge this Pull Request by running: git pull https://github.com/rackspace/jclouds refactor/filesystem-tests-osx Or you can view, comment on it, or merge it online at: https://github.com/jclouds/jclouds/pull/643 -- Commit Summary -- * Refactor blobstore filesystem tests for Mac OS X -- File Changes -- M apis/filesystem/src/test/java/org/jclouds/filesystem/FilesystemBlobStoreTest.java (14) M apis/filesystem/src/test/java/org/jclouds/filesystem/integration/FilesystemBlobIntegrationTest.java (66) M apis/filesystem/src/test/java/org/jclouds/filesystem/integration/FilesystemContainerIntegrationTest.java (17) M apis/filesystem/src/test/java/org/jclouds/filesystem/strategy/internal/FilesystemStorageStrategyImplTest.java (18) M apis/filesystem/src/test/java/org/jclouds/filesystem/utils/TestUtils.java (356) M core/src/test/java/org/jclouds/utils/TestUtils.java (30) -- Patch Links -- https://github.com/jclouds/jclouds/pull/643.patch https://github.com/jclouds/jclouds/pull/643.diff --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/643
Re: [jclouds] Refactor blobstore filesystem tests for Mac OS X (#643)
@Test(dataProvider = ignoreOnMacOSX)
@Override
public void testCreateBlobWithExpiry() throws InterruptedException {
super.testCreateBlobWithExpiry();
}
+
+ @Test(dataProvider = ignoreOnMacOSX)
+ @Override
+ public void testPutByteSource() throws Exception {
+ super.testPutByteSource();
+ }
+
+ @Test(dataProvider = ignoreOnMacOSX)
+ @Override
+ public void testPutInputStream() throws Exception {
+ super.testPutInputStream();
This doesn't seem like the right approach. Instead this should override
`checkContentMetadata` and similar.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/643/files#r22888595
Re: [jclouds] Refactor blobstore filesystem tests for Mac OS X (#643)
There's a lot going on here -- can we break this up into a few different pull requests, e.g., reindentation, Mac OS fixes, etc.? Thanks! --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/643#issuecomment-69806634
Re: [jclouds] Introduce BlobstoreCli (#487)
Now that I have had a look at this, doesn't this make more sense under examples? --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/487#issuecomment-69855141
Re: [jclouds] make xattr work in docker volume (#642)
Pushed to master as 5edd5de38c01f3a1ea962dd9f0a4bd6d060f32de. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/642#issuecomment-69860164
Re: [jclouds-examples] Adds an example blobstore uploader. (#69)
+ fis = new FileInputStream(file);
+ md5Local = DigestUtils.md5Hex(fis);
+ fis.close();
+ } catch (java.io.IOException e) {
+ e.printStackTrace();
+ /**
+ * The file is no longer available on the local FS.
+ * In some application cases, you might also want to retry this
instead of finishing the unit of work.
+ */
+ return;
+ }
+
+ /**
+ * Uploading a file over a network is an inherently fragile
operation. Over thousands of files, especially in
+ * highly parallel jobs that tax upload bandwidth, a small percent
of uploads are guaranteed to fail.
+ */
The retry loop is just for md5 mismatches will the ByteSourcePayload handle
that? Then I won't need to calculate md5 at all. But I don't think it actually
does that...?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/69/files#r22909300
[jclouds] Fixes the exponential backoff for a value of 0 (#644)
You can merge this Pull Request by running: git pull https://github.com/rackspace/jclouds exponential-backoff-for0 Or you can view, comment on it, or merge it online at: https://github.com/jclouds/jclouds/pull/644 -- Commit Summary -- * Fixes the exponential backoff for a value of 0 -- File Changes -- M core/src/main/java/org/jclouds/http/handlers/BackoffLimitedRetryHandler.java (6) -- Patch Links -- https://github.com/jclouds/jclouds/pull/644.patch https://github.com/jclouds/jclouds/pull/644.diff --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/644
Re: [jclouds-site] Update docker.md (#122)
@@ -25,6 +25,7 @@ This involves [dockerizing an SSH daemon service](https://docs.docker.com/exampl // get a context with docker that offers the portable ComputeService api ComputeServiceContext context = ContextBuilder.newBuilder(docker) .credentials(email, password) + .endpoint(http://your-docker-engine-host:2375;) yes as they are officially listed [here](http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=docker) --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds-site/pull/122/files#r22849881
Re: [jclouds-site] Update docker.md (#122)
@demobox I've addressed your questions. Are we good to merge that now? --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds-site/pull/122#issuecomment-69715798
Re: [jclouds-site] add softlayer getting started (#134)
+api.getVirtualGuestApi();
+api.getDatacenterApi();
+api.getSoftwareDescriptionApi();
+api.getVirtualGuestBlockDeviceTemplateGroupApi();
+api.getAccountApi();
+
+// Be sure to close the context when done
+computeServiceContext.close();
+{% endhighlight %}
+
+## SoftLayer Object Storage
+SoftLayer Object Storage is an OpenStack® based blobstore storage system.
+
+{% highlight java %}
+// Get a context with softlayer that offers the portable BlobStore API
+BlobStoreContext context = ContextBuilder.newBuilder(swift)
Not sure about it, @everett-toews
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-site/pull/134/files#r22850062
[jclouds-site] Update users.md to include ClouDesire.com in users page (#150)
You can merge this Pull Request by running: git pull https://github.com/gionn/jclouds-site patch-1 Or you can view, comment on it, or merge it online at: https://github.com/jclouds/jclouds-site/pull/150 -- Commit Summary -- * Update users.md -- File Changes -- M community/users.md (4) -- Patch Links -- https://github.com/jclouds/jclouds-site/pull/150.patch https://github.com/jclouds/jclouds-site/pull/150.diff --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds-site/pull/150
Re: [jclouds-site] Update users.md to include ClouDesire.com in users page (#150)
Go to http://b4400a2e3c89b5675fc1-6f7615d156538bc2d90463b7c9f6fd46.r34.cf5.rackcdn.com/ to review your changes. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds-site/pull/150#issuecomment-69723522
Re: [jclouds] disable directory blob tests in OS X (#640)
@kahing @andrewgaul After pulling this commit and running the tests again, I am
getting two new failures :confused: :
```
Failed tests:
FilesystemBlobIntegrationTestBaseBlobIntegrationTest.testPutByteSource:531-BaseBlobIntegrationTest.testPut:584
Expecting:
{}
to be equal to:
{key1=value1, key2=value2}
but was not.
FilesystemBlobIntegrationTestBaseBlobIntegrationTest.testPutInputStream:539-BaseBlobIntegrationTest.testPut:584
Expecting:
{}
to be equal to:
{key1=value1, key2=value2}
but was not.
```
Any insight is appreciated, thx!
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/640#issuecomment-69770173
Re: [jclouds] Fix for JCLOUDS-655 (#626)
@monitorjbl Sorry I haven't been able to have a look at this. I've been fighting the flu. @nacx Can you help with the unit test of just the adapter? If not, I can look at it later this week or next. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/626#issuecomment-69770083
Build failed in Jenkins: jclouds-guava-17-18 » 18.0,OpenJDK 7 (latest) #30
See https://jclouds.ci.cloudbees.com/job/jclouds-guava-17-18/GUAVA_VERSION=18.0,jdk=OpenJDK%207%20(latest)/30/ -- [...truncated 5622 lines...] at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) [pool-23-thread-1] Test testBreakGraceWhenNodeSocketFailsToOpen(org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMapTest) succeeded: 6ms Test suite progress: tests succeeded: 264, failed: 1, skipped: 0. Starting test testDefaultIsToReturnConfiguredCredential(org.jclouds.compute.strategy.impl.ReturnCredentialsBoundToImageTest) Starting test testDefaultIsToReturnConfiguredCredentialInStore(org.jclouds.compute.strategy.impl.ReturnCredentialsBoundToImageTest) [pool-24-thread-1] Test testDefaultIsToReturnConfiguredCredential(org.jclouds.compute.strategy.impl.ReturnCredentialsBoundToImageTest) succeeded: 0ms Starting test testReturnLoginCredentialAssociatedToOsFamily(org.jclouds.compute.strategy.impl.ReturnCredentialsBoundToImageTest) Test suite progress: tests succeeded: 265, failed: 1, skipped: 0. [pool-24-thread-2] Test testDefaultIsToReturnConfiguredCredentialInStore(org.jclouds.compute.strategy.impl.ReturnCredentialsBoundToImageTest) succeeded: 1ms Starting test testReturnRootWhenNotOnWindows(org.jclouds.compute.strategy.impl.ReturnCredentialsBoundToImageTest) Test suite progress: tests succeeded: 266, failed: 1, skipped: 0. [pool-24-thread-3] Test testReturnLoginCredentialAssociatedToOsFamily(org.jclouds.compute.strategy.impl.ReturnCredentialsBoundToImageTest) succeeded: 0ms Test suite progress: tests succeeded: 267, failed: 1, skipped: 0. [pool-24-thread-4] Test testReturnRootWhenNotOnWindows(org.jclouds.compute.strategy.impl.ReturnCredentialsBoundToImageTest) succeeded: 1ms Test suite progress: tests succeeded: 268, failed: 1, skipped: 0. Starting test testAllContains(org.jclouds.compute.stub.StubApiMetadataTest) Starting test testTransformableToContains(org.jclouds.compute.stub.StubApiMetadataTest) Starting test testWithId(org.jclouds.compute.stub.StubApiMetadataTest) [pool-25-thread-1] Test testAllContains(org.jclouds.compute.stub.StubApiMetadataTest) succeeded: 1ms Test suite progress: tests succeeded: 269, failed: 1, skipped: 0. [pool-25-thread-3] Test testWithId(org.jclouds.compute.stub.StubApiMetadataTest) succeeded: 0ms Test suite progress: tests succeeded: 270, failed: 1, skipped: 0. [pool-25-thread-2] Test testTransformableToContains(org.jclouds.compute.stub.StubApiMetadataTest) succeeded: 2ms Test suite progress: tests succeeded: 271, failed: 1, skipped: 0. Starting test testAllProtocol(org.jclouds.net.util.IpPermissionsTest) Starting test testAllProtocolCidrBound(org.jclouds.net.util.IpPermissionsTest) Starting test testAllProtocolInvalidCidr(org.jclouds.net.util.IpPermissionsTest) Starting test testAllProtocolInvalidCidrMultiple(org.jclouds.net.util.IpPermissionsTest) Starting test testAllProtocolExclusionCidrBound(org.jclouds.net.util.IpPermissionsTest) [pool-26-thread-1] Test testAllProtocol(org.jclouds.net.util.IpPermissionsTest) succeeded: 3ms Test suite progress: tests succeeded: 272, failed: 1, skipped: 0. [pool-26-thread-3] Test testAllProtocolExclusionCidrBound(org.jclouds.net.util.IpPermissionsTest) succeeded: 2ms Test suite progress: tests succeeded: 273, failed: 1, skipped: 0. [pool-26-thread-4] Test testAllProtocolInvalidCidr(org.jclouds.net.util.IpPermissionsTest) succeeded: 2ms Test suite progress: tests succeeded: 274, failed: 1, skipped: 0. [pool-26-thread-2] Test testAllProtocolCidrBound(org.jclouds.net.util.IpPermissionsTest) succeeded: 3ms Starting test testAllProtocolInvalidExclusionCidr(org.jclouds.net.util.IpPermissionsTest) Test suite progress: tests succeeded: 275, failed: 1, skipped: 0. Starting test testAnyProtocol(org.jclouds.net.util.IpPermissionsTest) [pool-26-thread-5] Test testAllProtocolInvalidCidrMultiple(org.jclouds.net.util.IpPermissionsTest) succeeded: 2ms Starting test testAllProtocolInvalidExclusionCidrMultiple(org.jclouds.net.util.IpPermissionsTest) Test suite progress: tests succeeded: 276, failed: 1, skipped: 0. [pool-26-thread-1] Test testAllProtocolInvalidExclusionCidr(org.jclouds.net.util.IpPermissionsTest) succeeded: 1ms Test suite progress: tests succeeded: 277, failed: 1, skipped: 0. Starting test testAnyProtocolWithExcludedCidr(org.jclouds.net.util.IpPermissionsTest) [pool-26-thread-4] Test testAnyProtocol(org.jclouds.net.util.IpPermissionsTest) succeeded: 1ms Starting test testJustProtocolAndCidr(org.jclouds.net.util.IpPermissionsTest) Test suite progress: tests succeeded: 278, failed: 1, skipped: 0. [pool-26-thread-2] Test
Re: [jclouds] disable directory blob tests in OS X (#640)
We should skip those additional two tests as well. I will submit a PR shortly with the changes, thx! --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/640#issuecomment-69775234
Re: [jclouds] Refactor blobstore filesystem tests for Mac OS X (#643)
@andrewgaul sure thing. closing... --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds/pull/643#issuecomment-69807025
Re: [jclouds-examples] Adds an example blobstore uploader. (#69)
+import org.jclouds.Constants;
+import org.jclouds.ContextBuilder;
+import org.jclouds.blobstore.BlobStore;
+import org.jclouds.blobstore.domain.Blob;
+import org.jclouds.domain.Location;
+import org.jclouds.io.Payloads;
+import org.jclouds.openstack.swift.v1.blobstore.RegionScopedBlobStoreContext;
+
+public class BlobUploader implements Runnable {
+ /**
+* ThreadLocal allows us to use 1 container and 1 connection per thread
and reuse them.
+* They need to be static, but here we also have to instantiate them at
runtime, as we pass parameters from
+* the command line.
+*/
+ private static ThreadLocalBlobStore blobStore = new
ThreadLocalBlobStore();
+ private static ThreadLocalString container = new ThreadLocalString();
To be reused by threadpool threads for multiple files. That way the same
blobstore and container is used to upload multiple files.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/69/files#r22901548
Re: [jclouds-examples] Adds an example blobstore uploader. (#69)
+ ExecutorService executor = Executors.newFixedThreadPool(numThreads);
+
+ for(File f : files) {
+ BlobUploader b =
+ new BlobUploader(
+ (String) options.valueOf(username),
+ (String) options.valueOf(password),
+ (String) options.valueOf(provider),
+ (String) options.valueOf(region),
+ f);
+ executor.execute(b);
+ }
+ executor.shutdown();
+
+ try {
+ executor.awaitTermination(1, TimeUnit.DAYS);
Matter of preference. The jobs I used to run with similar code would usually
indicate some kind of issue if this limit was reached: for example, a
consistent error when uploading a file with a very specific file name. With
this code failures get to be retried indefinitely, so sometimes they will sift
down to the last N threads running.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/69/files#r22901785
Re: [jclouds-examples] Adds an example blobstore uploader. (#69)
+ * Instantiate the ThreadLocal variables when this thread runs for the
first time.
+ * Instantiating this in the constructor will not work (different
thread).
+ */
+ if(blobStore.get() == null) {
+ // It is usually a good idea to include the currentThread when
logging parallel tasks.
+ System.out.println(Creating connection for thread +
Thread.currentThread());
+ /**
+ * In some cases, especially when running very large jobs with many
parallel threads, some connections will
+ * break. In that case, we need to be able to obtain a new
connection (and socket) to the service, which is
+ * why this is factored out.
+ */
+ resetBlobstore(username, password, provider, region);
+ }
+
+ if(container.get() == null) {
+ container.set(UUID.randomUUID().toString());
I should probably add some comments to clarify why UUID.
You want a randomized container name (for aws). You also want a container per
thread (makes this much simpler to use with other providers and in code). Does
this make sense?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/69/files#r22902058
Re: [jclouds-examples] Adds an example blobstore uploader. (#69)
BlobstoreCli seems to do a lot more than this simpler example intends to. I rather split BlobstoreCli into multiple command like jars, maybe. Thoughts? Having examples that do similar things (as long as the jclouds code within them is consistent) is fine, I think. --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds-examples/pull/69#issuecomment-69836576
Re: [jclouds-examples] Adds an example blobstore uploader. (#69)
+ CollectionFile files = FileUtils.listFiles(rootDir,
CanReadFileFilter.CAN_READ, TrueFileFilter.TRUE);
+ totalBytes = FileUtils.sizeOfDirectory(rootDir);
+
+ System.out.println(Uploading + rootDir.getName() + +
totalBytes/FileUtils.ONE_MB + MB);
+
+ ExecutorService executor = Executors.newFixedThreadPool(numThreads);
+
+ for(File f : files) {
+ BlobUploader b =
+ new BlobUploader(
+ (String) options.valueOf(username),
+ (String) options.valueOf(password),
+ (String) options.valueOf(provider),
+ (String) options.valueOf(region),
+ f);
+ executor.execute(b);
It is not necessary to do that (usually there is no need to control individual
tasks by their Future here). It might help if you are doing something with
exception handling, but we really want these to be handled in thread or by
jclouds.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/69/files#r22902856
Re: [jclouds-examples] Adds an example blobstore uploader. (#69)
+ +import java.io.File; +import java.io.IOException; +import java.util.Collection; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicLong; + +import org.apache.commons.io.FileUtils; +import org.apache.commons.io.filefilter.CanReadFileFilter; +import org.apache.commons.io.filefilter.TrueFileFilter; + +import joptsimple.OptionException; +import joptsimple.OptionParser; +import joptsimple.OptionSet; I have used it some... it's not bad. joptsimple just looks less like magic (which I like in examples code). --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds-examples/pull/69/files#r22903376
Re: [jclouds-site] Update docker.md (#122)
Are we good to merge that now? Yes, fine by me. Thanks for the ping! --- Reply to this email directly or view it on GitHub: https://github.com/jclouds/jclouds-site/pull/122#issuecomment-69847462
