[ https://issues.apache.org/jira/browse/CLOUDSTACK-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sangeetha Hariharan reopened CLOUDSTACK-3434: --------------------------------------------- Reopening this issue as per the findings (in the previous comment) from the latest 4.2 build. > Parallel deployment - Xenserver - When deploying 30 Vms in parallel/starting > 30 VMs in parallel, Vms failed because of failing in > "SavePasswordCommand"/"Dhcp Entry". > ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: CLOUDSTACK-3434 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3434 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: Management Server > Affects Versions: 4.2.0 > Environment: Build from 4.2 > Reporter: Sangeetha Hariharan > Assignee: Sheng Yang > Priority: Blocker > Fix For: 4.2.0 > > Attachments: parallel-run.rar, xenparallel.rar > > > Parallel deployment - Xenserver - When deploying 30 Vms in parallel/starting > 30 VMs in parallel, Vms failed because of failing in > "SavePasswordCommand"/"Dhcp Entry". > Steps to reproduce the problem: > Advanced zone set up with Xenserver host. > Deploy 30 Vms in parallel. > Out of 30 vms , 1 vm failed to start successfully and is in "Error" state > because of timing out on the "SavePasswordCommand". > Following is the snippet from management server logs: > 2013-07-09 16:30:28,895 DEBUG [cloud.network.NetworkModelImpl] > (Job-Executor-35:job-35) Service SecurityGroup is not supported in the > network id=204 > 2013-07-09 16:30:28,898 DEBUG > [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-35:job-35) > Applying userdata and password entry in network Ntwk[204|Guest|8] > 2013-07-09 16:30:28,910 DEBUG [agent.transport.Request] > (Job-Executor-35:job-35) Seq 1-729350387: Sending { Cmd , MgmtId: > 7200344900649, via: 1, Ver: v1, Flags: 100011, [{"com.clou > d.agent.api.routing.SavePasswordCommand":{"password":"fnirq_cnffjbeq","vmIpAddress":"10.1.1.148","vmName":"hello-14","executeInSequence":false,"accessDetails":{"router.guest.ip":"10 > .1.1.1","zone.network.type":"Advanced","router.ip":"169.254.3.54","router.name":"r-4-VM"},"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"10.1.1.148","vmName > ":"hello-14","executeInSequence":false,"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.3.54","router.name":"r-4-VM"},"wait":0}}] > } > 2013-07-09 16:30:28,911 DEBUG [agent.transport.Request] > (Job-Executor-35:job-35) Seq 1-729350387: Executing: { Cmd , MgmtId: > 7200344900649, via: 1, Ver: v1, Flags: 100011, [{"com.c > loud.agent.api.routing.SavePasswordCommand":{"password":"fnirq_cnffjbeq","vmIpAddress":"10.1.1.148","vmName":"hello-14","executeInSequence":false,"accessDetails":{"router.guest.ip": > "10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.3.54","router.name":"r-4-VM"},"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":"10.1.1.148","vmN > ame":"hello-14","executeInSequence":false,"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.3.54","router.name":"r-4-VM"},"wait":0}}] > } > ...... > 2013-07-09 16:30:32,224 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-105:null) Seq 1-729350387: Cancelling because one of the answers > is false and it is stop on error. > 2013-07-09 16:30:32,224 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-105:null) Seq 1-729350387: Response Received: > 2013-07-09 16:30:32,225 DEBUG [agent.transport.Request] > (DirectAgent-105:null) Seq 1-729350387: Processing: { Ans: , MgmtId: > 7200344900649, via: 1, Ver: v1, Flags: 10, [{"com.cloud > .agent.api.Answer":{"result":false,"details":"savePassword > failed","wait":0}}] } > 2013-07-09 16:30:32,225 DEBUG [agent.transport.Request] > (Job-Executor-35:job-35) Seq 1-729350387: Received: { Ans: , MgmtId: > 7200344900649, via: 1, Ver: v1, Flags: 10, { Answer } } > 2013-07-09 16:30:32,225 INFO [cloud.vm.VirtualMachineManagerImpl] > (Job-Executor-35:job-35) Unable to contact resource. > com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is > unreachable: Unable to apply userdata and password entry on router > at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3784) > at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyUserData(VirtualNetworkApplianceManagerImpl.java:2977) > at > com.cloud.network.element.VirtualRouterElement.addPasswordAndUserdata(VirtualRouterElement.java:944) > at > com.cloud.network.NetworkManagerImpl.prepareElement(NetworkManagerImpl.java:2006) > at > com.cloud.network.NetworkManagerImpl.prepareNic(NetworkManagerImpl.java:2112) > at > com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2053) > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:849) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557) > at > org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:239) > at > org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3317) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2877) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2863) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155) > at > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:513) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:679) > 2013-07-09 16:30:32,230 DEBUG [cloud.vm.VirtualMachineManagerImpl] > (Job-Executor-35:job-35) Cleaning up resources for the vm VM[User|hello-14] > in Starting state > 2013-07-09 16:30:32,231 DEBUG [agent.transport.Request] > (Job-Executor-35:job-35) Seq 1-729350390: Sending { Cmd , MgmtId: > 7200344900649, via: 1, Ver: v1, Flags: 100011, > [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"i-3-27-VM","wait":0}}] > } -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira