[
https://issues.apache.org/jira/browse/WHIRR-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13700435#comment-13700435
]
Jeremy Lv commented on WHIRR-733:
---------------------------------
[~abayer]:
I have another similar question that it will be blocked during the operation
about launche-cluster because it can't move the seift blobstore container. I
think the reason is because we delete the related container during the
"Runtime.getRuntime().addShutdownHook(new Thread()".
I have changed the code and remove the container after we upload the local file
to the remote instance, the local file can be uploaded successfully and the
swift blobstore can be remove successfully during the some thread.
Here's attached my modification too.
> Need to support a new option called whirr.user-thread when it comes to the
> Openstack
> ------------------------------------------------------------------------------------
>
> Key: WHIRR-733
> URL: https://issues.apache.org/jira/browse/WHIRR-733
> Project: Whirr
> Issue Type: New Feature
> Components: core
> Affects Versions: 0.8.2
> Environment: Ubuntu
> Openstack
> Reporter: Jeremy Lv
> Fix For: 0.9.0
>
> Attachments: swift_test.zip
>
>
> I found it will be blocked when create a virtual machine based on the
> Openstack(It is because the whirr project can't remove the swift container
> after created the new instance)
> Here's my reproduced steps:
> First of all, you must configrate the Openstack environments including the
> openstack swift
> 1>. Configurate my properties file called zookeeper.properties(In my
> environment, I have put the file in the directory of
> "/home/Jeremy/zookeeper.properties")
> {code}
> whirr.cluster-name=myhadoopcluster
> #whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1
> hadoop-datanode+hadoop-tasktracker
> whirr.instance-templates=1 zookeeper
> whirr.provider=openstack-nova
> whirr.location-id=RegionOne
> whirr.identity=admin:admin
> whirr.credential=hastexo
> whirr.image-id=RegionOne/7cf21d93-d04a-4fb4-a09b-f546a9461726
> whirr.endpoint = http://10.167.157.86:5000/v2.0
> whirr.blobstore-provider=swift-keystone
> whirr.blobstore-endpoint=http://10.167.157.86:5000/v2.0/
> whirr.blobstore-identity=service:swift
> whirr.blobstore-credential=hastexo
> #whirr.blobstore-cache-container=swift
> #whirr.blobstore-location-id=sdb1
> whirr.zookeeper.tarball.url=file:///home/Jeremy/test_sample1.war
> jclouds.openstack-nova.auto-generate-keypairs = true
> #jclouds.openstack-nova.auto-create-floating-ips = true
> whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
> whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub
> {code}
> 2>. Download the whirr-0.8.2.zip to my local
> platform(/home/Jeremy/whirr-0.8.2) and unzip it
> 3>. cd /home/Jeremy/whirr-0.8.2 and execute the following command:
> ./bin/whirr launch-cluster --config ../zookeeper.properties
> 4>.You will found the swift blob container can be created successfully but
> can't be removed automatically after create the new instance.(It will be
> blocked during the process of launch-cluster and can't be canced).
> After a further investigation about the whirr project, I found it is because
> of the following code:
> {code:title=BlobCache.java|borderStyle=solid}
> static {
> /* Ensure that all created containers are removed when the JVM stops */
> Runtime.getRuntime().addShutdownHook(new Thread() {
> @Override
> public void run() {
> BlobCache.dropAndCloseAll();
> }
> });
> }
> {code}
> It is because the default value of the user thread in jclouds side is "0",
> even we create a new thread in the whirr side, we can't delete the container
> here because of the following Exception
> {code}
> Exception in thread "Thread-0"
> java.util.concurrent.RejectedExecutionException: Task
> [delegate=[method=SwiftKeystoneAsyncClient.listContainers, request=GET
> http://10.167.157.86:8080/v1/AUTH_60d91f3ee474439e8edbbf9f15f424c1/?format=json
> HTTP/1.1],
> executionList=com.google.common.util.concurrent.ExecutionList@1b8633e]
> rejected from java.util.concurrent.ThreadPoolExecutor@7b7e61[Terminated, pool
> size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]
> at
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2013)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337)
> at
> org.jclouds.concurrent.config.DescribingExecutorService.execute(DescribingExecutorService.java:105)
> at
> org.jclouds.concurrent.Futures$FutureListener.addListener(Futures.java:122)
> at
> org.jclouds.concurrent.Futures$ListenableFutureAdapter.addListener(Futures.java:153)
> at com.google.common.util.concurrent.Futures.transform(Futures.java:269)
> at com.google.common.util.concurrent.Futures.transform(Futures.java:380)
> at org.jclouds.concurrent.Futures.compose(Futures.java:270)
> at
> org.jclouds.http.TransformingHttpCommandExecutorServiceImpl.submit(TransformingHttpCommandExecutorServiceImpl.java:54)
> at
> org.jclouds.http.TransformingHttpCommandImpl.execute(TransformingHttpCommandImpl.java:73)
> at
> org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFutureForHttpRequestMappedToMethodAndArgs(AsyncRestClientProxy.java:254)
> at
> org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:148)
> at $Proxy70.listContainers(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:170)
> at $Proxy71.listContainers(Unknown Source)
> at com.jclouds.test.JCloudsSwift.deleteConatiner(JCloudsSwift.java:63)
> at com.jclouds.test.JCloudsSwift.access$0(JCloudsSwift.java:60)
> at com.jclouds.test.JCloudsSwift$1.run(JCloudsSwift.java:34)
> {code}
> All in all, I have also write a sample related to the jcloud and found if we
> define the PROPERTY_USER_THREADS in the jclouds side, it will be successful
> to remove the swift container though the
> Runtime.getRuntime().addShutdownHook(new Thread(). If we didn't define the
> PROPERTY_USER_THREADS in the jclouds side and it will be thrown the
> RejectedExecutionException in calling the Blobstore.deleteContainer method
> because of the default user thread is "0"
> So I think we need to define a new option to support this situation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
