Jeremy Lv created WHIRR-733:
-------------------------------
Summary: Need to support a new option called whirr.user-thread
when it comes to the Openstack
Key: WHIRR-733
URL: https://issues.apache.org/jira/browse/WHIRR-733
Project: Whirr
Issue Type: New Feature
Components: core
Affects Versions: 0.8.2
Environment: Ubuntu
Openstack
Reporter: Jeremy Lv
Fix For: 0.9.0
I found it will be blocked when create a virtual machine based on the
Openstack(It is because the whirr project can't remove the swift container
after created the new instance)
Here's my reproduced steps:
First of all, you must configrate the Openstack environments including the
openstack swift
1>. Configurate my properties file called zookeeper.properties(In my
environment, I have put the file in the directory of
"/home/Jeremy/zookeeper.properties")
{code}
whirr.cluster-name=myhadoopcluster
#whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1
hadoop-datanode+hadoop-tasktracker
whirr.instance-templates=1 zookeeper
whirr.provider=openstack-nova
whirr.location-id=RegionOne
whirr.identity=admin:admin
whirr.credential=hastexo
whirr.image-id=RegionOne/7cf21d93-d04a-4fb4-a09b-f546a9461726
whirr.endpoint = http://10.167.157.86:5000/v2.0
whirr.blobstore-provider=swift-keystone
whirr.blobstore-endpoint=http://10.167.157.86:5000/v2.0/
whirr.blobstore-identity=service:swift
whirr.blobstore-credential=hastexo
#whirr.blobstore-cache-container=swift
#whirr.blobstore-location-id=sdb1
whirr.zookeeper.tarball.url=file:///home/Jeremy/test_sample1.war
jclouds.openstack-nova.auto-generate-keypairs = true
#jclouds.openstack-nova.auto-create-floating-ips = true
whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub
{code}
2>. Download the whirr-0.8.2.zip to my local platform(/home/Jeremy/whirr-0.8.2)
and unzip it
3>. cd /home/Jeremy/whirr-0.8.2 and execute the following command:
./bin/whirr launch-cluster --config ../zookeeper.properties
4>.You will found the swift blob container can be created successfully but
can't be removed automatically after create the new instance.(It will be
blocked during the process of launch-cluster and can't be canced).
After a further investigation about the whirr project, I found it is because of
the following code:
{code:title=BlobCache.java|borderStyle=solid}
static {
/* Ensure that all created containers are removed when the JVM stops */
Runtime.getRuntime().addShutdownHook(new Thread() {
@Override
public void run() {
BlobCache.dropAndCloseAll();
}
});
}
{code}
It is because the default value of the user thread in jclouds side is "0", even
we create a new thread in the whirr side, we can't delete the container here
because of the following Exception
{code}
Exception in thread "Thread-0" java.util.concurrent.RejectedExecutionException:
Task [delegate=[method=SwiftKeystoneAsyncClient.listContainers, request=GET
http://10.167.157.86:8080/v1/AUTH_60d91f3ee474439e8edbbf9f15f424c1/?format=json
HTTP/1.1],
executionList=com.google.common.util.concurrent.ExecutionList@1b8633e] rejected
from java.util.concurrent.ThreadPoolExecutor@7b7e61[Terminated, pool size = 0,
active threads = 0, queued tasks = 0, completed tasks = 8]
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2013)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:816)
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1337)
at
org.jclouds.concurrent.config.DescribingExecutorService.execute(DescribingExecutorService.java:105)
at
org.jclouds.concurrent.Futures$FutureListener.addListener(Futures.java:122)
at
org.jclouds.concurrent.Futures$ListenableFutureAdapter.addListener(Futures.java:153)
at com.google.common.util.concurrent.Futures.transform(Futures.java:269)
at com.google.common.util.concurrent.Futures.transform(Futures.java:380)
at org.jclouds.concurrent.Futures.compose(Futures.java:270)
at
org.jclouds.http.TransformingHttpCommandExecutorServiceImpl.submit(TransformingHttpCommandExecutorServiceImpl.java:54)
at
org.jclouds.http.TransformingHttpCommandImpl.execute(TransformingHttpCommandImpl.java:73)
at
org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFutureForHttpRequestMappedToMethodAndArgs(AsyncRestClientProxy.java:254)
at
org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:148)
at $Proxy70.listContainers(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:170)
at $Proxy71.listContainers(Unknown Source)
at com.jclouds.test.JCloudsSwift.deleteConatiner(JCloudsSwift.java:63)
at com.jclouds.test.JCloudsSwift.access$0(JCloudsSwift.java:60)
at com.jclouds.test.JCloudsSwift$1.run(JCloudsSwift.java:34)
{code}
All in all, I have also write a sample related to the jcloud and found if we
define the PROPERTY_USER_THREADS in the jclouds side, it will be successful to
remove the swift container though the Runtime.getRuntime().addShutdownHook(new
Thread(). If we didn't define the PROPERTY_USER_THREADS in the jclouds side and
it will be thrown the RejectedExecutionException in calling the
Blobstore.deleteContainer method because of the default user thread is "0"
So I think we need to define a new option to support this situation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
