Permission denied on vmdata_kvm.py? How did that happen? Can you look at the file mode on the virtual router in /opt/cloud/bin? The iso build is supposed to set all scripts to executable, regardless of the file mode in git. Might be worth mounting the iso and looking at cloud-scripts.tgz as well. On Jul 30, 2013 7:10 PM, "Toshiaki Hatano (JIRA)" <j...@apache.org> wrote:
> Toshiaki Hatano created CLOUDSTACK-3964: > ------------------------------------------- > > Summary: [KVM] failed to deploy VM in advanced zone > Key: CLOUDSTACK-3964 > URL: > https://issues.apache.org/jira/browse/CLOUDSTACK-3964 > Project: CloudStack > Issue Type: Bug > Security Level: Public (Anyone can view this level - this is the > default.) > Components: KVM, SystemVM > Affects Versions: Future > Environment: master branch > Reporter: Toshiaki Hatano > Assignee: Toshiaki Hatano > Priority: Blocker > > > In agent.log: > 2013-07-31 00:55:47,788 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) Request:Seq 2-96534582: { Cmd , MgmtId: > 161330500648, via: 2, Ver: v1, Flags: 100011, [{"com.c > > loud.agent.api.routing.SavePasswordCommand":{"password":"fnirq_cnffjbeq","vmIpAddress":"172.31.2.252","vmName":"test-vm1","executeInSequence":false,"accessDetails":{" > router.gu > > est.ip":"172.31.2.1","zone.network.type":"Advanced","router.ip":"169.254.1.92"," > router.name > ":"r-4-VM"},"wait":0}},{"com.cloud.agent.api.routing.VmDataCommand":{"vmIpAddress":" > > 172.31.2.252","vmName":"test-vm1","executeInSequence":false,"accessDetails":{"router.guest.ip":"172.31.2.1","zone.network.type":"Advanced","router.ip":"169.254.1.92","router.n > ame":"r-4-VM"},"wait":0}}] } > 2013-07-31 00:55:47,789 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) Processing command: > com.cloud.agent.api.routing.SavePasswordCommand > 2013-07-31 00:55:47,789 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) Executing: > /usr/share/cloudstack-common/scripts/network/domr/save_ > password_to_domr.sh -r 169.254.1.92 -v 172.31.2.252 -p fnirq_cnffjbeq > test-vm1 > 2013-07-31 00:55:47,944 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) Execution is successful. > 2013-07-31 00:55:47,944 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) Processing command: > com.cloud.agent.api.routing.VmDataCommand > 2013-07-31 00:55:47,946 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) JSON > IS:{"172.31.2.252":[["userdata","user-data",null],["metadata" > ,"service-offering","Medium > Instance"],["metadata","availability-zone","AdvancedZone-VXLAN"],["metadata","local-ipv4","172.31.2.252"],["metadata","local-hostname","test-vm1"], > > ["metadata","public-ipv4","198.172.17.227"],["metadata","public-hostname","198.172.17.227"],["metadata","instance-id","67bceb45-a36c-43dd-8da9-11999ccd8373"],["metadata","vm-i > > d","67bceb45-a36c-43dd-8da9-11999ccd8373"],["metadata","public-keys",null],["metadata","cloud-identifier","CloudStack-{81381f9d-8d33-441d-a4df-dcb03bc16e6b}"]]} > 2013-07-31 00:55:47,966 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) Executing: > /usr/share/cloudstack-common/scripts/network/domr/route > r_proxy.sh vmdata_kvm.py 169.254.1.92 -d > eyIxNzIuMzEuMi4yNTIiOltbInVzZXJkYXRhIiwidXNlci1kYXRhIixudWxsXSxbIm1ldGFkYXRhIiwic2VydmljZS1vZmZlcmluZyIsIk1lZGl1bSBJbnN0YW5jZSJdLFsibW > > V0YWRhdGEiLCJhdmFpbGFiaWxpdHktem9uZSIsIkFkdmFuY2VkWm9uZS1WWExBTiJdLFsibWV0YWRhdGEiLCJsb2NhbC1pcHY0IiwiMTcyLjMxLjIuMjUyIl0sWyJtZXRhZGF0YSIsImxvY2FsLWhvc3RuYW1lIiwidGVzdC12bTEiX > > SxbIm1ldGFkYXRhIiwicHVibGljLWlwdjQiLCIxOTguMTcyLjE3LjIyNyJdLFsibWV0YWRhdGEiLCJwdWJsaWMtaG9zdG5hbWUiLCIxOTguMTcyLjE3LjIyNyJdLFsibWV0YWRhdGEiLCJpbnN0YW5jZS1pZCIsIjY3YmNlYjQ1LWEz > > NmMtNDNkZC04ZGE5LTExOTk5Y2NkODM3MyJdLFsibWV0YWRhdGEiLCJ2bS1pZCIsIjY3YmNlYjQ1LWEzNmMtNDNkZC04ZGE5LTExOTk5Y2NkODM3MyJdLFsibWV0YWRhdGEiLCJwdWJsaWMta2V5cyIsbnVsbF0sWyJtZXRhZGF0YSI > > sImNsb3VkLWlkZW50aWZpZXIiLCJDbG91ZFN0YWNrLXs4MTM4MWY5ZC04ZDMzLTQ0MWQtYTRkZi1kY2IwM2JjMTZlNmJ9Il1dfQ== > 2013-07-31 00:55:48,049 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) Exit value is 126 > 2013-07-31 00:55:48,050 DEBUG > [resource.virtualnetwork.VirtualRoutingResource] > (agentRequest-Handler-1:null) bash: /opt/cloud/bin/vmdata_kvm.py: > Permission denied > 2013-07-31 00:55:48,051 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-1:null) Seq 2-96534582: { Ans: , MgmtId: > 161330500648, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent. > api.Answer":{"result":true,"wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"VmDataCommand > failed, check agent logs","wait":0}}] } > 2013-07-31 00:55:48,099 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-3:null) Request:Seq 2-96534583: { Cmd , MgmtId: > 161330500648, via: 2, Ver: v1, Flags: 100011, [{"com.c > loud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"i-2-3-VM","wait":0}}] > } > 2013-07-31 00:55:48,099 DEBUG [cloud.agent.Agent] > (agentRequest-Handler-3:null) Processing command: > com.cloud.agent.api.StopCommand > 2013-07-31 00:55:48,102 DEBUG [kvm.resource.LibvirtConnection] > (agentRequest-Handler-3:null) can't find connection: KVM, for vm: i-2-3-VM, > continue > 2013-07-31 00:55:48,104 DEBUG [kvm.resource.LibvirtConnection] > (agentRequest-Handler-3:null) can't find connection: LXC, for vm: i-2-3-VM, > continue > 2013-07-31 00:55:48,104 DEBUG [kvm.resource.LibvirtConnection] > (agentRequest-Handler-3:null) can't find which hypervisor the vm used , > then use the default hypervisor > 2013-07-31 00:55:48,106 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Failed to get dom xml: > org.libvirt.LibvirtException: Domain not found: no d > omain with matching name 'i-2-3-VM' > 2013-07-31 00:55:48,107 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Failed to get dom xml: > org.libvirt.LibvirtException: Domain not found: no d > omain with matching name 'i-2-3-VM' > 2013-07-31 00:55:48,109 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Failed to get dom xml: > org.libvirt.LibvirtException: Domain not found: no d > omain with matching name 'i-2-3-VM' > 2013-07-31 00:55:48,109 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Executing: > /usr/share/cloudstack-common/scripts/vm/network/security_group.p > y destroy_network_rules_for_vm --vmname i-2-3-VM > 2013-07-31 00:55:48,242 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Execution is successful. > 2013-07-31 00:55:48,243 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Try to stop the vm at first > 2013-07-31 00:55:48,244 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Failed to stop VM :i-2-3-VM : > org.libvirt.LibvirtException: Domain not found: no domain with matching > name 'i-2-3-VM' > at org.libvirt.ErrorHandler.processError(Unknown Source) > at org.libvirt.Connect.processError(Unknown Source) > at org.libvirt.Connect.domainLookupByName(Unknown Source) > at > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:4200) > at > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.stopVM(LibvirtComputingResource.java:4152) > at > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:3029) > at > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1146) > at com.cloud.agent.Agent.processRequest(Agent.java:499) > at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:807) > at com.cloud.utils.nio.Task.run(Task.java:83) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:679) > 2013-07-31 00:55:48,246 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Failed to get vm status:Domain not found: no > domain with matching name 'i-2 > -3-VM' > 2013-07-31 00:55:48,247 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Failed to get vm status:Domain not found: no > domain with matching name 'i-2 > -3-VM' > 2013-07-31 00:55:48,248 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Failed to get vm status:Domain not found: no > domain with matching name 'i-2 > -3-VM' > 2013-07-31 00:55:48,248 DEBUG [kvm.resource.LibvirtComputingResource] > (agentRequest-Handler-3:null) Can't get vm's status, assume it's dead > already > > In management.log: > 2013-07-31 00:55:59,344 ERROR [cloud.async.AsyncJobManagerImpl] > (Job-Executor-23:job-23 = [ fe6837ad-985d-4ff6-ba9c-5da9f36b0df5 ]) > Unexpected exception while executing org.ap > ache.cloudstack.api.command.user.vm.DeployVMCmd > com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM due > to insufficient capacity > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578) > at > org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:237) > at > org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3303) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2884) > at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2870) > at > com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) > at > org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:421) > at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158) > at > com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:538) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable > to create a deployment for VM[User|test-vm1]Scope=interface > com.cloud.dc.DataCenter; id=1 > at > com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:846) > at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:574) > ... 19 more > 2013-07-31 00:55:59,346 DEBUG [cloud.async.AsyncJobManagerImpl] > (Job-Executor-23:job-23 = [ fe6837ad-985d-4ff6-ba9c-5da9f36b0df5 ]) > Complete async job-23 = [ fe6837ad-985d-4ff > 6-ba9c-5da9f36b0df5 ], jobStatus: 2, resultCode: 530, result: Error Code: > 530 Error text: Unable to start a VM due to insufficient capacity > > > > -- > This message is automatically generated by JIRA. > If you think it was sent incorrectly, please contact your JIRA > administrators > For more information on JIRA, see: http://www.atlassian.com/software/jira >