Hi all,
I am using whirr to start a hadoop cluster on Cloudstack, I encounter
the following exception
2013-09-10 14:06:49,096 ERROR [org.apache.whirr.compute.StartupProcess]
(pool-3-thread-2) Unexpected error while starting 1 nodes, minimum 1
nodes for [hadoop-namenode, hadoop-jobtracker] of cluster hadoop
java.util.concurrent.ExecutionException:
java.lang.NullPointerException: encoded
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
at java.util.concurrent.FutureTask.get(FutureTask.java:111)
at
org.apache.whirr.compute.StartupProcess.waitForOutcomes(StartupProcess.java:130)
at
org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:83)
at
org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:41)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NullPointerException: encoded
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:208)
at
org.jclouds.compute.internal.FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.firstGroupInPatternOrNull(FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.java:140)
at
org.jclouds.compute.internal.FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.groupInUniqueNameOrNull(FormatSharedNamesAndAppendUniqueStringToThoseWhichRepeat.java:136)
at
org.jclouds.cloudstack.compute.functions.VirtualMachineToNodeMetadata.apply(VirtualMachineToNodeMetadata.java:107)
at
org.jclouds.cloudstack.compute.functions.VirtualMachineToNodeMetadata.apply(VirtualMachineToNodeMetadata.java:61)
at
com.google.common.base.Functions$FunctionComposition.apply(Functions.java:210)
at com.google.common.collect.Iterators$8.transform(Iterators.java:860)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at
com.google.common.collect.Iterators$7.computeNext(Iterators.java:701)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
at com.google.common.collect.Iterators.any(Iterators.java:736)
at com.google.common.collect.Iterables.any(Iterables.java:623)
at
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.getNextNames(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:188)
at
org.jclouds.compute.strategy.impl.CreateNodesWithGroupEncodedIntoNameThenAddToSet.execute(CreateNodesWithGroupEncodedIntoNameThenAddToSet.java:122)
at
org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:212)
at org.apache.whirr.compute.NodeStarter.call(NodeStarter.java:55)
at org.apache.whirr.compute.NodeStarter.call(NodeStarter.java:30)
... 5 more
2013-09-10 14:06:49,106 INFO
[org.apache.whirr.actions.ScriptBasedClusterAction] (main) Finished
running destroy phase scripts on all cluster instances
2013-09-10 14:06:49,106 INFO
[org.apache.whirr.actions.DestroyClusterAction] (main) Destroying hadoop
cluster
2013-09-10 14:06:49,108 DEBUG [jclouds.compute] (main) >> destroying
nodes matching(inGroup(hadoop))
2013-09-10 14:06:49,110 DEBUG
[org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod] (main) >>
invoking listVirtualMachines
2013-09-10 14:06:49,111 DEBUG [jclouds.signature] (main) >> GET
http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true
HTTP/1.1
2013-09-10 14:06:49,111 DEBUG [jclouds.signature] (main) >> Accept:
application/json
2013-09-10 14:06:49,111 DEBUG [jclouds.signature] (main) >>
"apikey=h3dkhc9avlhknuhpythmulhc119qfnqq8xhyjbf_rnu5zl1qeowdw7azrgxvo1vapg6q0ak-a-tqrqszfwnoxq&command=listvirtualmachines&listall=true&response=json"
2013-09-10 14:06:49,112 DEBUG [jclouds.signature] (main) <<
"8CdMYkm15Udov76n0lspNNZG0fQ="
2013-09-10 14:06:49,114 DEBUG [jclouds.signature] (main) << GET
http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true&apiKey=h3DKHC9AVlhKnUhpyThMuLhC119QfNQQ8xhyjbf_rnu5ZL1QeOWdw7aZRGXVO1VApG6q0aK-A-tQRQsZFwnOXQ&signature=8CdMYkm15Udov76n0lspNNZG0fQ%3D
HTTP/1.1
2013-09-10 14:06:49,114 DEBUG [jclouds.signature] (main) << Accept:
application/json
2013-09-10 14:06:49,114 DEBUG
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService] (main)
Sending request 1519465049: GET
http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true&apiKey=h3DKHC9AVlhKnUhpyThMuLhC119QfNQQ8xhyjbf_rnu5ZL1QeOWdw7aZRGXVO1VApG6q0aK-A-tQRQsZFwnOXQ&signature=8CdMYkm15Udov76n0lspNNZG0fQ%3D
HTTP/1.1
2013-09-10 14:06:49,114 DEBUG [jclouds.headers] (main) >> GET
http://localhost:8080/client/api?response=json&command=listVirtualMachines&listAll=true&apiKey=h3DKHC9AVlhKnUhpyThMuLhC119QfNQQ8xhyjbf_rnu5ZL1QeOWdw7aZRGXVO1VApG6q0aK-A-tQRQsZFwnOXQ&signature=8CdMYkm15Udov76n0lspNNZG0fQ%3D
HTTP/1.1
2013-09-10 14:06:49,115 DEBUG [jclouds.headers] (main) >> Accept:
application/json
2013-09-10 14:06:49,131 DEBUG
[org.jclouds.http.internal.JavaUrlHttpCommandExecutorService] (main)
Receiving response 1519465049: HTTP/1.1 200 OK
2013-09-10 14:06:49,131 DEBUG [jclouds.headers] (main) << HTTP/1.1 200
OK
2013-09-10 14:06:49,131 DEBUG [jclouds.headers] (main) << Date: Tue, 10
Sep 2013 18:06:48 GMT
2013-09-10 14:06:49,131 DEBUG [jclouds.headers] (main) << Server:
Apache-Coyote/1.1
2013-09-10 14:06:49,132 DEBUG [jclouds.headers] (main) << Content-Type:
text/javascript;charset=UTF-8
2013-09-10 14:06:49,132 DEBUG [jclouds.headers] (main) <<
Content-Length: 1550
2013-09-10 14:06:49,132 DEBUG [jclouds.wire] (main) << "{
"listvirtualmachinesresponse" : { "count":1 ,"virtualmachine" : [
{"id":"a978e52f-9ee8-433e-abf5-10986fd0c16e","name":"a978e52f-9ee8-433e-abf5-10986fd0c16e","account":"admin","domainid":"611281e6-1a1b-11e3-bce5-0023ae94f722","domain":"ROOT","created":"2013-09-10T09:30:29-0400","state":"Running","haenable":false,"zoneid":"62857481-bb61-4607-ac7d-a450135c83a6","zonename":"zone1","hostid":"08b35a58-6a82-4150-be18-e95f110d44de","hostname":"meng.acis.ufl.edu","templateid":"616bd368-1a1b-11e3-bce5-0023ae94f722","templatename":"CentOS
5.5(64-bit) no GUI (KVM)","templatedisplaytext":"CentOS 5.5(64-bit) no
GUI
(KVM)","passwordenabled":false,"serviceofferingid":"cfd100e5-5f59-4e8a-a8e1-9706c7a11488","serviceofferingname":"Small
Instance","cpunumber":1,"cpuspeed":500,"memory":512,"cpuused":"5.14%","networkkbsread":4479,"networkkbswrite":142,"guestosid":"6716e852-1a1b-11e3-bce5-0023ae94f722","rootdeviceid":0,"rootdevicetype":"ROOT","securitygroup":[{"id":"b414da10-1a1b-11e3-bce5-0023ae94f722","name":"default","description":"Default
Security
Group","account":"admin","ingressrule":[],"egressrule":[],"tags":[]}],"nic":[{"id":"d66a5aac-d15f-4e08-90c5-73a476db1240","networkid":"7da7d83c-ec6b-4e1c-9791-cdc40191eb96","networkname":"defaultGuestNetwork","netmask":"255.255.255.0","gateway":"10.244.18.1","ipaddress":"10.244.18.61","isolationuri":"ec2://untagged","broadcasturi":"vlan://untagged","traffictype":"Guest","type":"Shared","isdefault":true,"macaddress":"06:23:bc:00:00:10"}],"hypervisor":"KVM","instancename":"i-2-3-VM","tags":[]}
] } }"
On the cloudstack management server side, I didn't say any instance
even starting. I couldn't find any DEBUG information on the
management-server.log.
Below is the cluster properties file I use:
# Set the name of your hadoop cluster
whirr.cluster-name=hadoop
whirr.provider=cloudstack
# Change the name of cluster admin user
whirr.cluster-user=${sys:user.name}
# Change the number of machines in the cluster here
# Below we define one hadoop namenode and 3 hadoop datanode
whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,1
hadoop-datanode+hadoop-tasktracker
# Specify which distribution of hadoop you want to use
# Here we choose to use the Cloudera distribution
whirr.env.repo=cdh4
whirr.hadoop.install-function=install_cdh_hadoop
whirr.hadoop.configure-function=configure_cdh_hadoop
# Use a specific instance type.
# Specify the uuid of the CloudStack service offering to use for the
instances of your hadoop cluster
whirr.hardware-id=cfd100e5-5f59-4e8a-a8e1-9706c7a11488
# If you use ssh key pairs to access instances in the cloud
# Specify them like so
whirr.private-key-file=/home/meng/.ssh/id_rsa
whirr.public-key-file=/home/meng/.ssh/id_rsa.pub
# Specify the template to use for the instances
# This is the uuid of the CloudStack template
whirr.image-id=616bd368-1a1b-11e3-bce5-0023ae94f722
whirr.bootstrap-user=root:password
whirr.use-cloudstack-security-group=true
Could someone give me a pointer what might be the possible reason for
this error?
Thank you.
Best Regards,
Meng
