[
https://issues.apache.org/jira/browse/WHIRR-576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zuocheng Ren updated WHIRR-576:
-------------------------------
Attachment: exalt-hbase-ec2.properties
> Starting more instances than specified (and more).
> --------------------------------------------------
>
> Key: WHIRR-576
> URL: https://issues.apache.org/jira/browse/WHIRR-576
> Project: Whirr
> Issue Type: Bug
> Components: core, service/hbase
> Affects Versions: 0.7.1
> Reporter: Zuocheng Ren
> Attachments: exalt-hbase-ec2.properties
>
>
> When starting an hbase cluster with whirr, I encountered the following
> problem:
> This is what instance templates looks like:
> whirr.instance-templates=1
> zookeeper+hadoop-namenode+hadoop-jobtracker+hbase-master,3
> hadoop-datanode+hadoop-tasktracker+hbase-regionserver
> Despite the failure to start the cluster, I noticed that whirr tried to start
> more instances than I specified.
> I took a loot at aws console, indeed 6(1+3+2) instances were started.
> Moreover, whirr wrongly report that the cluster failed to start and destroyed
> the cluster.
> whirr-0.7.1 ren$ bin/whirr launch-cluster --config
> recipes/exalt-hbase-ec2.properties
> Bootstrapping cluster
> Configuring template
> Configuring template
> Starting 1 node(s) with roles [zookeeper, hadoop-namenode, hadoop-jobtracker,
> hbase-master]
> Starting 3 node(s) with roles [hadoop-datanode, hadoop-tasktracker,
> hbase-regionserver]
> Starting 2 node(s) with roles [hadoop-datanode, hadoop-tasktracker,
> hbase-regionserver]
> Nodes started: [[id=us-east-1/i-a4efefc0, providerId=i-a4efefc0,
> group=exalt-hbase, name=exalt-hbase-a4efefc0, location=[id=us-east-1c,
> scope=ZONE, description=us-east-1c, parent=us-east-1, iso3166Codes=[US-VA],
> metadata={}], uri=null, imageId=us-east-1/ami-ab36fbc2, os=[name=null,
> family=ubuntu, version=10.04, arch=paravirtual, is64Bit=false,
> description=099720109477/ebs/ubuntu-images/ubuntu-lucid-10.04-i386-server-20110930],
> state=RUNNING, loginPort=22, hostname=domU-12-31-39-0B-F6-07,
> privateAddresses=[10.214.249.241], publicAddresses=[174.129.105.39],
> hardware=[id=m1.small, providerId=m1.small, name=null,
> processors=[[cores=1.0, speed=1.0]], ram=1740, volumes=[[id=null, type=LOCAL,
> size=150.0, device=/dev/sda2, durable=false, isBootDevice=false],
> [id=vol-63cbc00f, type=SAN, size=null, device=/dev/sda1, durable=true,
> isBootDevice=true]],
> supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,Not(is64Bit())),
> tags=[]], loginUser=ubuntu, userMetadata={Name=exalt-hbase-a4efefc0},
> tags=[]]]
> Destroying failed nodes [us-east-1/i-daefefbe, us-east-1/i-c4f5f5a0,
> us-east-1/i-a6efefc2]
> Destroyed failed nodes [us-east-1/i-daefefbe, us-east-1/i-c4f5f5a0,
> us-east-1/i-a6efefc2]
> Unable to start the cluster. Terminating all nodes.
> java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: Too many instance failed while bootstrapping! 2
> successfully started instances while 3 instances failed
> at
> org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:129)
> at
> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:107)
> at
> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:106)
> at
> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
> at org.apache.whirr.cli.Main.run(Main.java:64)
> at org.apache.whirr.cli.Main.main(Main.java:97)
> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: Too
> many instance failed while bootstrapping! 2 successfully started instances
> while 3 instances failed
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at
> org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:124)
> ... 5 more
> Caused by: java.io.IOException: Too many instance failed while bootstrapping!
> 2 successfully started instances while 3 instances failed
> at org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:92)
> at org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:40)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
> Unable to load cluster state, assuming it has no running nodes.
> java.io.FileNotFoundException: /Users/ren/.whirr/exalt-hbase/instances (No
> such file or directory)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.<init>(FileInputStream.java:120)
> at com.google.common.io.Files$1.getInput(Files.java:100)
> at com.google.common.io.Files$1.getInput(Files.java:97)
> at com.google.common.io.CharStreams$2.getInput(CharStreams.java:91)
> at com.google.common.io.CharStreams$2.getInput(CharStreams.java:88)
> at com.google.common.io.CharStreams.readLines(CharStreams.java:306)
> at com.google.common.io.Files.readLines(Files.java:580)
> at
> org.apache.whirr.state.FileClusterStateStore.load(FileClusterStateStore.java:54)
> at
> org.apache.whirr.state.ClusterStateStore.tryLoadOrEmpty(ClusterStateStore.java:58)
> at
> org.apache.whirr.ClusterController.destroyCluster(ClusterController.java:143)
> at
> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:118)
> at
> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
> at org.apache.whirr.cli.Main.run(Main.java:64)
> at org.apache.whirr.cli.Main.main(Main.java:97)
> Starting to run scripts on cluster for phase destroyinstances:
> Starting to run scripts on cluster for phase destroyinstances:
> Finished running destroy phase scripts on all cluster instances
> Destroying exalt-hbase cluster
> Cluster exalt-hbase destroyed
> Exception in thread "main" java.lang.RuntimeException: java.io.IOException:
> java.util.concurrent.ExecutionException: java.io.IOException: Too many
> instance failed while bootstrapping! 2 successfully started instances while 3
> instances failed
> at
> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:125)
> at
> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
> at org.apache.whirr.cli.Main.run(Main.java:64)
> at org.apache.whirr.cli.Main.main(Main.java:97)
> Caused by: java.io.IOException: java.util.concurrent.ExecutionException:
> java.io.IOException: Too many instance failed while bootstrapping! 2
> successfully started instances while 3 instances failed
> at
> org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:129)
> at
> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:107)
> at
> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:106)
> ... 3 more
> Caused by: java.util.concurrent.ExecutionException: java.io.IOException: Too
> many instance failed while bootstrapping! 2 successfully started instances
> while 3 instances failed
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at
> org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:124)
> ... 5 more
> Caused by: java.io.IOException: Too many instance failed while bootstrapping!
> 2 successfully started instances while 3 instances failed
> at org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:92)
> at org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:40)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:680)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
