Thank you for your answer and I will try that. -----Original Message----- From: ilya musayev [mailto:ilya.mailing.li...@gmail.com] Sent: Tuesday, September 16, 2014 11:19 PM To: users@cloudstack.apache.org Subject: Re: Unable to start a VM due to insufficient capacity
I've seen this before and had similar issues. Instead of re-installing or dropping DB, you can delete all job_ids that are stuck. For example, this is what i would see in my logs. (1013,VmWorkJobQueue, 2375) is reaching concurrency limit 1 (1013,VmWorkJobQueue, 2375) is reaching concurrency limit 1 (1433,VmWorkJobQueue, 2470) is reaching concurrency limit 1 (1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1 (1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1 (1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1 (1478,VmWorkJobQueue, 596) is reaching concurrency limit 1 (981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 DELETE ALL FROM BELOW: SELECT * FROM cloud.sync_queue_item where queue_id=981 or queue_id=1478 or queue_id=1477 or queue_id=1475 or queue_id=1434 or queue_id=1433 or queue_id=1013 ; SELECT * FROM cloud.sync_queue where id=1433 or id=1434 or id=1475 or id=1477 or id=1478 or id=981; While i did SELECT statement, you can alter to DELETE. Regards ilya On 9/16/14, 6:31 PM, Matthew Midgett wrote: > Ugo, I am having an issue just like you. I can deploy the default > centos template but if I try to deploy a iso it looks like its failing > in the network area. I've been reading your emails to Geoff and I have > restarted the network with a clean and that didn't fix my problem. I > also restarted the management servers during this process. I don't > know what happed but ssvm and console quit working. I've spent all day > deleting the whole thing and setting it back up again. Just to find that it's not working. > > Now I just saw this in the logs > > Cannot delete or update a parent row: a foreign key constraint fails > (`cloud`.`async_job_join_map`, CONSTRAINT > `fk_async_job_join_map__join_job_id` FOREIGN KEY (`join_job_id`) > REFERENCES `async_job` (`id`)) > > I am running a 2 node haproxy with virtual ip in front of a galera sql > cluster. > > I dropped the DB, gave the cloud user more permissions. Deleted > everything and restarted and it's doing this all over again. I am > running xenserver 6.2 and acs4.4. I even tried 4.3 again and it started doing the same thing. > > > > -----Original Message----- > From: Ugo Vasi [mailto:ugo.v...@procne.it] > Sent: Tuesday, September 16, 2014 5:36 AM > To: users@cloudstack.apache.org > Subject: Re: Unable to start a VM due to insufficient capacity > > Hi Geoff, > I restarted the network and now I can create new VM without errors. > > Now, how can I prevent this issue? > > > > > Il 11/09/2014 09:28, Geoff Higginbottom ha scritto: >> Hi Ugo, >> >> Did restarting the network with cleanup option resolve your issue >> last > time, if yes have you tried the same on this network? >> Regards >> >> Geoff Higginbottom >> CTO / Cloud Architect >> >> D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 >> 0540<tel:+442036030540> | M: +447968161581<tel:+447968161581> >> >> geoff.higginbot...@shapeblue.com<mailto:geoff.higginbottom@shapeblue. >> c >> om> | www.shapeblue.com<htp://www.shapeblue.com/> | >> Twitter:@cloudstackguru<https://twitter.com/#!/cloudstackguru> >> >> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N >> 4HS<x-apple-data-detectors://5> >> >> >> On 11 Sep 2014, at 08:17, "Ugo Vasi" > <ugo.v...@procne.it<mailto:ugo.v...@procne.it>> wrote: >> Hi Geoff, >> I still have the problem creating new VMs but in another isolated network. > The problem seems to be the same: >> 2014-09-11 08:55:27,048 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM is being created in >> podId: 1 >> 2014-09-11 08:55:27,050 DEBUG [o.a.c.e.o.NetworkOrchestrator] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Network id=212 is already >> implemented >> 2014-09-11 08:55:27,118 DEBUG [c.c.n.NetworkModelImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is >> not supported in the network id=212 >> 2014-09-11 08:55:27,122 DEBUG [o.a.c.e.o.NetworkOrchestrator] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Changing active number of >> nics for network id=212 on 1 >> 2014-09-11 08:55:27,174 DEBUG [o.a.c.e.o.NetworkOrchestrator] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Asking VirtualRouter to >> prepare for >> Nic[116-86-8429f13f-7c3b-4b8a-aea3-9b8e9a3f4c03-10.1.1.19] >> 2014-09-11 08:55:27,181 DEBUG >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Lock is acquired for >> network id 212 as a part of router startup in >> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->P >> o >> ol(Id))] : >> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(7)-Storage(Volume(98|ROOT-->Pool( >> 6 >> ))] >> 2014-09-11 08:55:27,183 DEBUG >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Lock is released for >> network id 212 as a part of router startup in >> Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->P >> o >> ol(Id))] : >> Dest[Zone(1)-Pod(1)-Cluster(1)-Host(7)-Storage(Volume(98|ROOT-->Pool( >> 6 >> ))] >> 2014-09-11 08:55:27,189 DEBUG [c.c.n.NetworkModelImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is >> not supported in the network id=212 >> 2014-09-11 08:55:27,198 DEBUG [c.c.n.NetworkModelImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is >> not supported in the network id=212 >> 2014-09-11 08:55:27,202 DEBUG >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Applying dhcp entry in >> network Ntwk[212|Guest|8] >> 2014-09-11 08:55:27,215 DEBUG [c.c.a.t.Request] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304056: Sending >> { Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: >> 100011, >> [{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"02:00:15:7 >> 6 >> :00:0e","vmIpAddress":"10.1.1.19","vmName":"giannitest","defaultRoute >> r >> ":"10.1.1.1","defaultDns":"10.1.1.1","duid":"00:03:00:01:02:00:15:76: >> 0 >> 0:0e","isDefault":true,"executeInSequence":false,"accessDetails":{"ro >> u >> ter.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.name": >> "r-60-VM","router.ip":"169.254.1.230"},"wait":0}}] } >> 2014-09-11 08:55:27,941 DEBUG [c.c.s.s.SecondaryStorageManagerImpl] >> (secstorage-1:ctx-327037b2) Zone 1 is ready to launch secondary >> storage VM >> 2014-09-11 08:55:28,022 DEBUG [c.c.c.ConsoleProxyManagerImpl] >> (consoleproxy-1:ctx-a8ba3b1e) Zone 1 is ready to launch console proxy >> 2014-09-11 08:55:28,820 DEBUG [c.c.a.ApiServlet] >> (catalina-exec-16:ctx-58e92382) ===START=== 192.168.130.89 -- GET >> command=queryAsyncJobResult&jobId=195bf80a-9823-4ec3-add4-8907b98fbaf >> 0 >> &response=json&sessionkey=qGZA6KYoc1SB%2Bq2k70HoM%2BBBQTU%3D&_=141041 >> 8 >> 528731 >> 2014-09-11 08:55:29,069 DEBUG [c.c.a.t.Request] >> (AgentManager-Handler-13:null) Seq 7-564304056: Processing: { Ans: , >> MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, >> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] } >> 2014-09-11 08:55:29,069 DEBUG [c.c.a.t.Request] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304056: Received: >> { Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, { Answer >> } } >> 2014-09-11 08:55:29,075 DEBUG [c.c.n.NetworkModelImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is >> not supported in the network id=212 >> 2014-09-11 08:55:29,078 DEBUG >> [c.c.n.r.VirtualNetworkApplianceManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Applying userdata and >> password entry in network Ntwk[212|Guest|8] >> 2014-09-11 08:55:29,089 DEBUG [c.c.a.t.Request] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304057: Sending >> { Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: >> 100011, >> [{"com.cloud.agent.api.routing.SavePasswordCommand":{,"vmIpAddress":" >> 1 >> 0.1.1.19","vmName":"giannitest","executeInSequence":false,"accessDeta >> i >> ls":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","rou >> t >> er.ip":"169.254.1.230","router.name":"r-60-VM"},"wait":0}},{"com.clou >> d >> .agent.api.routing.VmDataCommand":{"vmIpAddress":"10.1.1.19","vmName": >> "giannitest","executeInSequence":false,"accessDetails":{"router.guest. >> ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.1. >> 2 30","router.name":"r-60-VM"},"wait":0}}] } >> 2014-09-11 08:55:29,399 DEBUG [c.c.a.t.Request] >> (AgentManager-Handler-10:null) Seq 7-564304057: Processing: { Ans: , >> MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, >> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}},{"com.cloud. >> a gent.api.Answer":{"result":false,"details":"VmDataCommand failed, >> check agent logs","wait":0}}] } >> 2014-09-11 08:55:29,491 DEBUG [c.c.a.t.Request] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304057: Received: >> { Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, { >> Answer, Answer } } >> 2014-09-11 08:55:29,491 INFO [c.c.v.VirtualMachineManagerImpl] > (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Unable to contact resource. >> com.cloud.exception.ResourceUnavailableException: Resource >> [DataCenter:1] > is unreachable: Unable to apply userdata and password entry on router >> at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules > (Virtu > alNetworkApplianceManagerImpl.java:3915) >> at > com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyUserD > ata(Vi > rtualNetworkApplianceManagerImpl.java:3118) >> at sun.reflect.GeneratedMethodAccessor472.invoke(Unknown >> Source) ... >> 2014-09-11 08:55:29,755 DEBUG [c.c.a.ApiServlet] >> (catalina-exec-16:ctx-58e92382 ctx-ad8ddf67) ===END=== 192.168.130.89 >> -- GET >> command=queryAsyncJobResult&jobId=195bf80a-9823-4ec3-add4-8907b98fbaf >> 0 >> &response=json&sessionkey=qGZA6KYoc1SB%2Bq2k70HoM%2BBBQTU%3D&_=141041 >> 8 >> 528731 >> 2014-09-11 08:55:29,798 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Cleaning up resources for >> the vm VM[User|giannitest] in Starting state >> 2014-09-11 08:55:29,800 DEBUG [c.c.a.t.Request] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Sending >> { Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: >> 100011, >> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequen >> c e":false,"vmName":"i-2-86-VM","wait":0}}] } >> 2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request] >> (AgentManager-Handler-6:null) Seq 7-564304058: Processing: { Ans: , >> MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, >> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] } >> 2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Received: >> { Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, { >> StopAnswer } } >> 2014-09-11 08:55:29,955 DEBUG [c.c.n.NetworkModelImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is >> not supported in the network id=212 >> 2014-09-11 08:55:29,957 DEBUG [o.a.c.e.o.NetworkOrchestrator] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Changing active number of >> nics for network id=212 on -1 >> 2014-09-11 08:55:30,051 DEBUG [o.a.c.e.o.NetworkOrchestrator] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Asking VirtualRouter to >> release >> NicProfile[116-86-8429f13f-7c3b-4b8a-aea3-9b8e9a3f4c03-10.1.1.19-null >> 2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully released >> network resources for the vm VM[User|giannitest] >> 2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully cleanued up >> resources for the vm VM[User|giannitest] in Starting state >> 2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [], >> clusters: [], hosts: [7] >> 2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' > provided is in avoid set, DeploymentPlanner cannot allocate the VM, returning. >> 2014-09-11 08:55:30,214 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from >> :Starting to Stopped with event: OperationFailedvm's original host id: >> null new host id: null host id before state transition: 7 >> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total CPU: >> 36256 and CPU after applying overprovisioning: 36256 >> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total RAM: >> 50639785984 and RAM after applying overprovisioning: 50639785984 >> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release cpu from host: 7, >> old used: 26500,reserved: 0, actual total: 36256, total with >> overprovisioning: 36256; new used: 26000,reserved:0; >> movedfromreserved: false,moveToReserveredfalse >> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release mem from host: 7, >> old used: 21072183296,reserved: 0, total: 50639785984; new used: >> 20535312384,reserved:0; movedfromreserved: >> false,moveToReserveredfalse >> 2014-09-11 08:55:30,302 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from >> :Stopped to Starting with event: StartRequestedvm's original host id: >> null new host id: null host id before state transition: null >> 2014-09-11 08:55:30,302 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully transitioned >> to start state for VM[User|giannitest] reservation id = >> 4b6c2f8d-a34c-4b86-a0d4-acd5ea87353a >> 2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Trying to deploy VM, vm >> has dcId: 1 and podId: 1 >> 2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [], >> clusters: [], hosts: [7] >> 2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [], >> clusters: [], hosts: [7] >> 2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' > provided is in avoid set, DeploymentPlanner cannot allocate the VM, returning. >> 2014-09-11 08:55:30,435 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from >> :Starting to Stopped with event: OperationFailedvm's original host id: >> null new host id: null host id before state transition: null >> 2014-09-11 08:55:30,477 DEBUG [c.c.v.UserVmManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Destroying vm >> VM[User|giannitest] as it failed to create on Host with Id:null >> 2014-09-11 08:55:30,609 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from >> :Stopped to Error with event: OperationFailedToErrorvm's original >> host >> id: null new host id: null host id before state transition: null >> 2014-09-11 08:55:29,755 DEBUG [c.c.a.ApiServlet] >> (catalina-exec-16:ctx-58e92382 ctx-ad8ddf67) ===END=== 192.168.130.89 >> -- GET >> command=queryAsyncJobResult&jobId=195bf80a-9823-4ec3-add4-8907b98fbaf >> 0 >> &response=json&sessionkey=qGZA6KYoc1SB%2Bq2k70HoM%2BBBQTU%3D&_=141041 >> 8 >> 528731 >> 2014-09-11 08:55:29,798 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Cleaning up resources for >> the vm VM[User|giannitest] in Starting state >> 2014-09-11 08:55:29,800 DEBUG [c.c.a.t.Request] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Sending >> { Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: >> 100011, >> [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequen >> c e":false,"vmName":"i-2-86-VM","wait":0}}] } >> 2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request] >> (AgentManager-Handler-6:null) Seq 7-564304058: Processing: { Ans: , >> MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, >> [{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] } >> 2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Received: >> { Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, { >> StopAnswer } } >> 2014-09-11 08:55:29,955 DEBUG [c.c.n.NetworkModelImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is >> not supported in the network id=212 >> 2014-09-11 08:55:29,957 DEBUG [o.a.c.e.o.NetworkOrchestrator] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Changing active number of >> nics for network id=212 on -1 >> 2014-09-11 08:55:30,051 DEBUG [o.a.c.e.o.NetworkOrchestrator] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Asking VirtualRouter to >> release >> NicProfile[116-86-8429f13f-7c3b-4b8a-aea3-9b8e9a3f4c03-10.1.1.19-null >> 2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully released >> network resources for the vm VM[User|giannitest] >> 2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully cleanued up >> resources for the vm VM[User|giannitest] in Starting state >> 2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [], >> clusters: [], hosts: [7] >> 2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' > provided is in avoid set, DeploymentPlanner cannot allocate the VM, returning. >> 2014-09-11 08:55:30,214 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from >> :Starting to Stopped with event: OperationFailedvm's original host id: >> null new host id: null host id before state transition: 7 >> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total CPU: >> 36256 and CPU after applying overprovisioning: 36256 >> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total RAM: >> 50639785984 and RAM after applying overprovisioning: 50639785984 >> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release cpu from host: 7, >> old used: 26500,reserved: 0, actual total: 36256, total with >> overprovisioning: 36256; new used: 26000,reserved:0; >> movedfromreserved: false,moveToReserveredfalse >> 2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release mem from host: 7, >> old used: 21072183296,reserved: 0, total: 50639785984; new used: >> 20535312384,reserved:0; movedfromreserved: >> false,moveToReserveredfalse >> 2014-09-11 08:55:30,302 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from >> :Stopped to Starting with event: StartRequestedvm's original host id: >> null new host id: null host id before state transition: null >> 2014-09-11 08:55:30,302 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully transitioned >> to start state for VM[User|giannitest] reservation id = >> 4b6c2f8d-a34c-4b86-a0d4-acd5ea87353a >> 2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Trying to deploy VM, vm >> has dcId: 1 and podId: 1 >> 2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [], >> clusters: [], hosts: [7] >> 2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [], >> clusters: [], hosts: [7] >> 2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl] > (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' > provided is in avoid set, DeploymentPlanner cannot allocate the VM, returning. >> 2014-09-11 08:55:30,435 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from >> :Starting to Stopped with event: OperationFailedvm's original host id: >> null new host id: null host id before state transition: null >> 2014-09-11 08:55:30,477 DEBUG [c.c.v.UserVmManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Destroying vm >> VM[User|giannitest] as it failed to create on Host with Id:null >> 2014-09-11 08:55:30,609 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from >> :Stopped to Error with event: OperationFailedToErrorvm's original >> host >> id: null new host id: null host id before state transition: null >> 2014-09-11 08:55:30,809 WARN [o.a.c.alerts] >> (Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) alertType:: 8 // >> dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Failed >> to deploy Vm with Id: 86, on Host with Id: null >> 2014-09-11 08:55:30,932 DEBUG [c.c.a.m.AgentManagerImpl] >> (AgentManager-Handler-1:null) Ping from 6 >> 2014-09-11 08:55:31,078 ERROR [c.c.a.ApiAsyncJobDispatcher] >> (Job-Executor-55:ctx-b5c34ec7) Unexpected exception while executing >> org.apache.cloudstack.api.command.user.vm.DeployVMCmd >> com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM >> due > to insufficient capacity >> at > com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java: > 605) >> at > org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.depl > oyVirt > ualMachine(VMEntityManagerImpl.java:237) >> at > org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl > .deplo > y(VirtualMachineEntityImpl.java:207) >> at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.j > ava:35 > 81) >> at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.j > ava:31 > 61) >> at > com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.j > ava:31 > 47) >> at sun.reflect.GeneratedMethodAccessor587.invoke(Unknown >> Source) ... >> >> >> Il 01/08/2014 11:44, Geoff Higginbottom ha scritto: >> Ugo, >> >> Login to the VR and tail the messages log then attempt to start the >> VM, > you should see various messages re DHCP updating etc in the VR. If > you don't see the VR update try restarting the network Iain the 'cleanup' > option, this will create a new VR. >> Regards >> >> Geoff Higginbottom >> CTO / Cloud Architect >> >> D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 >> 0540<tel:+442036030540> | M: +447968161581<tel:+447968161581> >> >> geoff.higginbot...@shapeblue.com<mailto:geoff.higginbottom@shapeblue. >> c >> om><mailto:geoff.higginbot...@shapeblue.com> | >> www.shapeblue.com<http://www.shapeblue.com><htp://www.shapeblue.com/> >> | Twitter:@cloudstackguru<https://twitter.com/#!/cloudstackguru> >> >> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N >> 4HS<x-apple-data-detectors://5> >> >> >> On 1 Aug 2014, at 09:06, "Ugo Vasi" > <ugo.v...@procne.it<mailto:ugo.v...@procne.it><mailto:ugo.vasi@procne. > it>> > wrote: >> Hi Geoff, >> I tried restarting r-29-VM but the problem is still blocking creation >> of > new VM in that isolated network. >> Creating a new VM in another isolated network, it works. Where can I >> look > at to find the problem? >> >> >> Il 31/07/2014 21:53, Geoff Higginbottom ha scritto: >> Ugo, >> >> Looks like CS is failing to update the VR with details for the new >> VM, try > restarting the VR. >> Regards >> >> Geoff Higginbottom >> CTO / Cloud Architect >> >> >> D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603 >> 0540<tel:+442036030540> | M: +447968161581<tel:+447968161581> >> >> geoff.higginbot...@shapeblue.com<mailto:geoff.higginbottom@shapeblue. >> c >> om><mailto:geoff.higginbot...@shapeblue.com><mailto:geoff.higginbotto >> om>m >> @shapeblue.com> | >> www.shapeblue.com<http://www.shapeblue.com><http://www.shapeblue.com> >> < >> htp://www.shapeblue.com/> >> >> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N >> 4HS<x-apple-data-detectors://37> >> >> >> >> On 31 Jul 2014, at 15:40, "Ugo Vasi" > <ugo.v...@procne.it<mailto:ugo.v...@procne.it><mailto:ugo.vasi@procne. > it><ma > ilto:ugo.v...@procne.it>> wrote: >> Hi everybody, >> I installed a CloudStack 4.3.0 system with a manager (in a virtual > machine), two NFS primary storage, one NFS secondary storare and two > host agent on ubuntu distribution. >> The cluster have one advanced zone without security group >> configuration > with KVM hipervisors. >> I create some VM but now does not allow me to create more giving the >> error > message "Unable to start a VM due to insufficient capacity". >> I am sure that the resources (ip, ram, cpu and storage) are enough to > create new instances. In dashboard I see all ok. >> >From the management-server.log I see this errors messages (see >> >attachment > for full version) when I try to deploy a new virtual machine with > 1x1GHz cpu, 1GB or RAM and 20GB disk space: >> 2014-07-31 14:59:11,983 DEBUG [c.c.a.t.Request] >> (AgentManager-Handler-12:null) Seq 1-684594119: Processing: { Ans: , >> MgmtId: 77916390353436, via: 1, Ver: v1, Flags: 10, >> [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}},{"com.cloud. >> a gent.api.Answer":{"result":false,"details":"VmDataCommand failed, >> check agent logs","wait":0}}] } >> 2014-07-31 14:59:11,983 INFO [c.c.v.VirtualMachineManagerImpl] > (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) Unable to contact resource. >> com.cloud.exception.ResourceUnavailableException: Resource >> [DataCenter:1] is unreachable: Unable to apply userdata and password >> entry on router >> 2014-07-31 14:59:12,453 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) VM state transitted from >> :Starting to Stopped with event: OperationFailedvm's original host id: >> null new host id: null host id before state transition: 7 >> 2014-07-31 14:59:12,677 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) VM state transitted from >> :Starting to Stopped with event: OperationFailedvm's original host id: >> null new host id: null host id before state transition: null >> 2014-07-31 14:59:12,708 DEBUG [c.c.v.UserVmManagerImpl] >> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) Destroying vm >> VM[User|csweb2] as it failed to create on Host with Id:null >> 2014-07-31 14:59:12,738 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) VM state transitted from >> :Stopped to Error with event: OperationFailedToErrorvm's original >> host >> id: null new host id: null host id before state transition: null >> 2014-07-31 14:59:12,929 WARN [o.a.c.alerts] >> (Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) alertType:: 8 // >> dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Failed >> to deploy Vm with Id: 70, on Host with Id: null >> 2014-07-31 14:59:13,159 ERROR [c.c.a.ApiAsyncJobDispatcher] >> (Job-Executor-26:ctx-ab3aee38) Unexpected exception while executing >> org.apache.cloudstack.api.command.user.vm.DeployVMCmd >> com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM >> due to insufficient capacity Caused by: >> com.cloud.exception.InsufficientServerCapacityException: Unable to >> create a deployment for VM[User|csweb2]Scope=interface >> com.cloud.dc.DataCenter; id=1 >> 2014-07-31 14:59:13,161 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> (Job-Executor-26:ctx-ab3aee38) Complete async job-783, jobStatus: >> FAILED, resultCode: 530, result: >> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList": >> [],"errorcode":530,"errortext":"Unable to start a VM due to >> insufficient capacity"} >> 2014-07-31 14:59:46,280 DEBUG [c.c.v.VirtualMachineManagerImpl] >> (Job-Executor-27:ctx-82be396e ctx-41f0cf24) Stopped called on >> VM[User|csweb2] but the state is Error >> 2014-07-31 14:59:46,330 DEBUG [c.c.c.CapacityManagerImpl] >> (Job-Executor-27:ctx-82be396e ctx-41f0cf24) VM state transitted from >> :Error to Expunging with event: DestroyRequestedvm's original host id: >> null new host id: null host id before state transition: null >> >> I tried to search in others log file ("VmDataCommand failed, check >> agent > logs"), but I don't found nothing... >> Can anyone Help me? >> >> -- >> >> U g o V a s i > <ugo.v...@procne.it<mailto:ugo.v...@procne.it><mailto:ugo.vasi@procne. > it><ma > ilto:ugo.v...@procne.it>> >> P r o c n e s.r.l >) >> via Cotonificio 45 33010 Tavagnacco IT >> phone: +390432486523 fax: +390432486523 >> >> Le informazioni contenute in questo messaggio sono riservate e >> confidenziali ed ? vietata la diffusione in qualunque modo eseguita. >> Qualora Lei non fosse la persona a cui il presente messaggio ? >> destinato, La invitiamo ad eliminarlo e a non leggerlo, dandocene >> gentilmente comunicazione. >> Per qualsiasi informazione si prega di contattare > supp...@procne.it<mailto:supp...@procne.it><mailto:supp...@procne.it>< > mailto > :supp...@procne.it> . >> Rif. D.L. 196/2003 >> >> >> >> >> <management-server.log> >> Find out more about ShapeBlue and our range of CloudStack related >> services >> >> IaaS Cloud Design & >> Build<http://shapeblue.com/iaas-cloud-design-and-build//> >> CSForge - rapid IaaS deployment >> framework<http://shapeblue.com/csforge/> >> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> >> CloudStack Infrastructure >> Support<http://shapeblue.com/cloudstack-infrastructure-support/> >> CloudStack Bootcamp Training >> Courses<http://shapeblue.com/cloudstack-training/> >> >> This email and any attachments to it may be confidential and are >> intended > solely for the use of the individual to whom it is addressed. Any > views or opinions expressed are solely those of the author and do not > necessarily represent those of Shape Blue Ltd or related companies. If > you are not the intended recipient of this email, you must neither > take any action based upon its contents, nor copy or show it to > anyone. Please contact the sender if you believe you have received > this email in error. Shape Blue Ltd is a company incorporated in > England & Wales. ShapeBlue Services India LLP is a company > incorporated in India and is operated under license from Shape Blue > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in > Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA > Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. >> >> >> >> >> >> -- >> >> U g o V a s i > <ugo.v...@procne.it<mailto:ugo.v...@procne.it><mailto:ugo.vasi@procne. > it>> >> P r o c n e s.r.l >) >> via Cotonificio 45 33010 Tavagnacco IT >> phone: +390432486523 fax: +390432486523 >> >> Le informazioni contenute in questo messaggio sono riservate e >> confidenziali ed ? vietata la diffusione in qualunque modo eseguita. >> Qualora Lei non fosse la persona a cui il presente messaggio ? >> destinato, La invitiamo ad eliminarlo e a non leggerlo, dandocene >> gentilmente comunicazione. >> Per qualsiasi informazione si prega di contattare > supp...@procne.it<mailto:supp...@procne.it><mailto:supp...@procne.it> . >> Rif. D.L. 196/2003 >> >> >> >> >> >> Find out more about ShapeBlue and our range of CloudStack related >> services >> >> IaaS Cloud Design & >> Build<http://shapeblue.com/iaas-cloud-design-and-build//> >> CSForge - rapid IaaS deployment >> framework<http://shapeblue.com/csforge/> >> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> >> CloudStack Infrastructure >> Support<http://shapeblue.com/cloudstack-infrastructure-support/> >> CloudStack Bootcamp Training >> Courses<http://shapeblue.com/cloudstack-training/> >> >> This email and any attachments to it may be confidential and are >> intended > solely for the use of the individual to whom it is addressed. Any > views or opinions expressed are solely those of the author and do not > necessarily represent those of Shape Blue Ltd or related companies. If > you are not the intended recipient of this email, you must neither > take any action based upon its contents, nor copy or show it to > anyone. Please contact the sender if you believe you have received > this email in error. Shape Blue Ltd is a company incorporated in > England & Wales. ShapeBlue Services India LLP is a company > incorporated in India and is operated under license from Shape Blue > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in > Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA > Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. >> >> >> >> >> >> -- >> >> U g o V a s i <ugo.v...@procne.it<mailto:ugo.v...@procne.it>> >> P r o c n e s.r.l >) >> via Cotonificio 45 33010 Tavagnacco IT >> phone: +390432486523 fax: +390432486523 >> >> Le informazioni contenute in questo messaggio sono riservate e >> confidenziali ed ? vietata la diffusione in qualunque modo eseguita. >> Qualora Lei non fosse la persona a cui il presente messaggio ? >> destinato, La invitiamo ad eliminarlo e a non leggerlo, dandocene >> gentilmente comunicazione. >> Per qualsiasi informazione si prega di contattare > supp...@procne.it<mailto:supp...@procne.it> . >> Rif. D.L. 196/2003 >> >> >> Find out more about ShapeBlue and our range of CloudStack related >> services >> >> IaaS Cloud Design & >> Build<http://shapeblue.com/iaas-cloud-design-and-build//> >> CSForge - rapid IaaS deployment >> framework<http://shapeblue.com/csforge/> >> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> >> CloudStack Infrastructure >> Support<http://shapeblue.com/cloudstack-infrastructure-support/> >> CloudStack Bootcamp Training >> Courses<http://shapeblue.com/cloudstack-training/> >> >> This email and any attachments to it may be confidential and are >> intended > solely for the use of the individual to whom it is addressed. Any > views or opinions expressed are solely those of the author and do not > necessarily represent those of Shape Blue Ltd or related companies. If > you are not the intended recipient of this email, you must neither > take any action based upon its contents, nor copy or show it to > anyone. Please contact the sender if you believe you have received > this email in error. Shape Blue Ltd is a company incorporated in > England & Wales. ShapeBlue Services India LLP is a company > incorporated in India and is operated under license from Shape Blue > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in > Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA > Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark. >> >> >> >
