I've seen this before and had similar issues.

Instead of re-installing or dropping DB, you can delete all job_ids that are stuck.

For example, this is what i would see in my logs.

(1013,VmWorkJobQueue, 2375) is reaching concurrency limit 1
(1013,VmWorkJobQueue, 2375) is reaching concurrency limit 1
(1433,VmWorkJobQueue, 2470) is reaching concurrency limit 1
(1434,VmWorkJobQueue, 2471) is reaching concurrency limit 1
(1475,VmWorkJobQueue, 2483) is reaching concurrency limit 1
(1477,VmWorkJobQueue, 2484) is reaching concurrency limit 1
(1478,VmWorkJobQueue, 596) is reaching concurrency limit 1
(981,VmWorkJobQueue, 2371) is reaching concurrency limit 1

DELETE ALL FROM BELOW:
SELECT * FROM cloud.sync_queue_item where queue_id=981 or queue_id=1478 or queue_id=1477 or queue_id=1475 or queue_id=1434 or queue_id=1433 or queue_id=1013 ; SELECT * FROM cloud.sync_queue where id=1433 or id=1434 or id=1475 or id=1477 or id=1478 or id=981;

While i did SELECT statement, you can alter to DELETE.

Regards
ilya

On 9/16/14, 6:31 PM, Matthew Midgett wrote:
Ugo, I am having an issue just like you. I can deploy the default centos
template but if I try to deploy a iso it looks like its failing in the
network area. I've been reading your emails to Geoff and I have restarted
the network with a clean and that didn't fix my problem. I also restarted
the management servers during this process. I don't know what happed but
ssvm and console quit working. I've spent all day deleting the whole thing
and setting it back up again. Just to find that it's not working.

Now I just saw this in the logs

Cannot delete or update a parent row: a foreign key constraint fails
(`cloud`.`async_job_join_map`, CONSTRAINT
`fk_async_job_join_map__join_job_id` FOREIGN KEY (`join_job_id`) REFERENCES
`async_job` (`id`))

I am running a 2 node haproxy with virtual ip in front of a galera sql
cluster.

I dropped the DB, gave the cloud user more permissions. Deleted everything
and restarted and it's doing this all over again. I am running xenserver 6.2
and acs4.4. I even tried 4.3 again and it started doing the same thing.



-----Original Message-----
From: Ugo Vasi [mailto:ugo.v...@procne.it]
Sent: Tuesday, September 16, 2014 5:36 AM
To: users@cloudstack.apache.org
Subject: Re: Unable to start a VM due to insufficient capacity

Hi Geoff,
I restarted the network and now I can create new VM without errors.

Now, how can I prevent this issue?




Il 11/09/2014 09:28, Geoff Higginbottom ha scritto:
Hi Ugo,

Did restarting the network with cleanup option resolve your issue last
time, if yes have you tried the same on this network?
Regards

Geoff Higginbottom
CTO / Cloud Architect

D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603
0540<tel:+442036030540> | M: +447968161581<tel:+447968161581>

geoff.higginbot...@shapeblue.com<mailto:geoff.higginbottom@shapeblue.c
om> | www.shapeblue.com<htp://www.shapeblue.com/> |
Twitter:@cloudstackguru<https://twitter.com/#!/cloudstackguru>

ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N
4HS<x-apple-data-detectors://5>


On 11 Sep 2014, at 08:17, "Ugo Vasi"
<ugo.v...@procne.it<mailto:ugo.v...@procne.it>> wrote:
Hi Geoff,
I still have the problem creating new VMs but in another isolated network.
The problem seems to be the same:
2014-09-11 08:55:27,048 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM is being created in
podId: 1
2014-09-11 08:55:27,050 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Network id=212 is already
implemented
2014-09-11 08:55:27,118 DEBUG [c.c.n.NetworkModelImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
not supported in the network id=212
2014-09-11 08:55:27,122 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Changing active number of
nics for network id=212 on 1
2014-09-11 08:55:27,174 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Asking VirtualRouter to
prepare for Nic[116-86-8429f13f-7c3b-4b8a-aea3-9b8e9a3f4c03-10.1.1.19]
2014-09-11 08:55:27,181 DEBUG
[c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Lock is acquired for
network id 212 as a part of router startup in
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Po
ol(Id))] :
Dest[Zone(1)-Pod(1)-Cluster(1)-Host(7)-Storage(Volume(98|ROOT-->Pool(6
))]
2014-09-11 08:55:27,183 DEBUG
[c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Lock is released for
network id 212 as a part of router startup in
Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Po
ol(Id))] :
Dest[Zone(1)-Pod(1)-Cluster(1)-Host(7)-Storage(Volume(98|ROOT-->Pool(6
))]
2014-09-11 08:55:27,189 DEBUG [c.c.n.NetworkModelImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
not supported in the network id=212
2014-09-11 08:55:27,198 DEBUG [c.c.n.NetworkModelImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
not supported in the network id=212
2014-09-11 08:55:27,202 DEBUG
[c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Applying dhcp entry in
network Ntwk[212|Guest|8]
2014-09-11 08:55:27,215 DEBUG [c.c.a.t.Request]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304056: Sending
{ Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.routing.DhcpEntryCommand":{"vmMac":"02:00:15:76
:00:0e","vmIpAddress":"10.1.1.19","vmName":"giannitest","defaultRouter
":"10.1.1.1","defaultDns":"10.1.1.1","duid":"00:03:00:01:02:00:15:76:0
0:0e","isDefault":true,"executeInSequence":false,"accessDetails":{"rou
ter.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.name":
"r-60-VM","router.ip":"169.254.1.230"},"wait":0}}] }
2014-09-11 08:55:27,941 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-327037b2) Zone 1 is ready to launch secondary
storage VM
2014-09-11 08:55:28,022 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-a8ba3b1e) Zone 1 is ready to launch console proxy
2014-09-11 08:55:28,820 DEBUG [c.c.a.ApiServlet]
(catalina-exec-16:ctx-58e92382) ===START===  192.168.130.89 -- GET
command=queryAsyncJobResult&jobId=195bf80a-9823-4ec3-add4-8907b98fbaf0
&response=json&sessionkey=qGZA6KYoc1SB%2Bq2k70HoM%2BBBQTU%3D&_=1410418
528731
2014-09-11 08:55:29,069 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-13:null) Seq 7-564304056: Processing:  { Ans: ,
MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}}] }
2014-09-11 08:55:29,069 DEBUG [c.c.a.t.Request]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304056: Received:
{ Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, { Answer
} }
2014-09-11 08:55:29,075 DEBUG [c.c.n.NetworkModelImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
not supported in the network id=212
2014-09-11 08:55:29,078 DEBUG
[c.c.n.r.VirtualNetworkApplianceManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Applying userdata and
password entry in network Ntwk[212|Guest|8]
2014-09-11 08:55:29,089 DEBUG [c.c.a.t.Request]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304057: Sending
{ Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.routing.SavePasswordCommand":{,"vmIpAddress":"1
0.1.1.19","vmName":"giannitest","executeInSequence":false,"accessDetai
ls":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","rout
er.ip":"169.254.1.230","router.name":"r-60-VM"},"wait":0}},{"com.cloud
.agent.api.routing.VmDataCommand":{"vmIpAddress":"10.1.1.19","vmName":
"giannitest","executeInSequence":false,"accessDetails":{"router.guest.
ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.1.2
30","router.name":"r-60-VM"},"wait":0}}] }
2014-09-11 08:55:29,399 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-10:null) Seq 7-564304057: Processing:  { Ans: ,
MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}},{"com.cloud.a
gent.api.Answer":{"result":false,"details":"VmDataCommand failed,
check agent logs","wait":0}}] }
2014-09-11 08:55:29,491 DEBUG [c.c.a.t.Request]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304057: Received:
{ Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, { Answer,
Answer } }
2014-09-11 08:55:29,491 INFO  [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Unable to contact resource.
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1]
is unreachable: Unable to apply userdata and password entry on router
         at
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(Virtu
alNetworkApplianceManagerImpl.java:3915)
         at
com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyUserData(Vi
rtualNetworkApplianceManagerImpl.java:3118)
         at sun.reflect.GeneratedMethodAccessor472.invoke(Unknown
Source) ...
2014-09-11 08:55:29,755 DEBUG [c.c.a.ApiServlet]
(catalina-exec-16:ctx-58e92382 ctx-ad8ddf67) ===END=== 192.168.130.89
-- GET
command=queryAsyncJobResult&jobId=195bf80a-9823-4ec3-add4-8907b98fbaf0
&response=json&sessionkey=qGZA6KYoc1SB%2Bq2k70HoM%2BBBQTU%3D&_=1410418
528731
2014-09-11 08:55:29,798 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Cleaning up resources for
the vm VM[User|giannitest] in Starting state
2014-09-11 08:55:29,800 DEBUG [c.c.a.t.Request]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Sending
{ Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequenc
e":false,"vmName":"i-2-86-VM","wait":0}}] }
2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-6:null) Seq 7-564304058: Processing:  { Ans: ,
MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Received:
{ Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, {
StopAnswer } }
2014-09-11 08:55:29,955 DEBUG [c.c.n.NetworkModelImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
not supported in the network id=212
2014-09-11 08:55:29,957 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Changing active number of
nics for network id=212 on -1
2014-09-11 08:55:30,051 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Asking VirtualRouter to
release
NicProfile[116-86-8429f13f-7c3b-4b8a-aea3-9b8e9a3f4c03-10.1.1.19-null
2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully released
network resources for the vm VM[User|giannitest]
2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully cleanued up
resources for the vm VM[User|giannitest] in Starting state
2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
clusters: [], hosts: [7]
2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' provided is
in avoid set, DeploymentPlanner cannot allocate the VM, returning.
2014-09-11 08:55:30,214 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
:Starting to Stopped with event: OperationFailedvm's original host id:
null new host id: null host id before state transition: 7
2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total CPU:
36256 and CPU after applying overprovisioning: 36256
2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total RAM:
50639785984 and RAM after applying overprovisioning: 50639785984
2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release cpu from host: 7,
old used: 26500,reserved: 0, actual total: 36256, total with
overprovisioning: 36256; new used: 26000,reserved:0;
movedfromreserved: false,moveToReserveredfalse
2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release mem from host: 7,
old used: 21072183296,reserved: 0, total: 50639785984; new used:
20535312384,reserved:0; movedfromreserved: false,moveToReserveredfalse
2014-09-11 08:55:30,302 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
:Stopped to Starting with event: StartRequestedvm's original host id:
null new host id: null host id before state transition: null
2014-09-11 08:55:30,302 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully transitioned
to start state for VM[User|giannitest] reservation id =
4b6c2f8d-a34c-4b86-a0d4-acd5ea87353a
2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Trying to deploy VM, vm
has dcId: 1 and podId: 1
2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
clusters: [], hosts: [7]
2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
clusters: [], hosts: [7]
2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' provided is
in avoid set, DeploymentPlanner cannot allocate the VM, returning.
2014-09-11 08:55:30,435 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
:Starting to Stopped with event: OperationFailedvm's original host id:
null new host id: null host id before state transition: null
2014-09-11 08:55:30,477 DEBUG [c.c.v.UserVmManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Destroying vm
VM[User|giannitest] as it failed to create on Host with Id:null
2014-09-11 08:55:30,609 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
:Stopped to Error with event: OperationFailedToErrorvm's original host
id: null new host id: null host id before state transition: null
2014-09-11 08:55:29,755 DEBUG [c.c.a.ApiServlet]
(catalina-exec-16:ctx-58e92382 ctx-ad8ddf67) ===END=== 192.168.130.89
-- GET
command=queryAsyncJobResult&jobId=195bf80a-9823-4ec3-add4-8907b98fbaf0
&response=json&sessionkey=qGZA6KYoc1SB%2Bq2k70HoM%2BBBQTU%3D&_=1410418
528731
2014-09-11 08:55:29,798 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Cleaning up resources for
the vm VM[User|giannitest] in Starting state
2014-09-11 08:55:29,800 DEBUG [c.c.a.t.Request]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Sending
{ Cmd , MgmtId: 77916390353436, via: 7(csh1b), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequenc
e":false,"vmName":"i-2-86-VM","wait":0}}] }
2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-6:null) Seq 7-564304058: Processing:  { Ans: ,
MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StopAnswer":{"result":true,"wait":0}}] }
2014-09-11 08:55:29,950 DEBUG [c.c.a.t.Request]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Seq 7-564304058: Received:
{ Ans: , MgmtId: 77916390353436, via: 7, Ver: v1, Flags: 10, {
StopAnswer } }
2014-09-11 08:55:29,955 DEBUG [c.c.n.NetworkModelImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Service SecurityGroup is
not supported in the network id=212
2014-09-11 08:55:29,957 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Changing active number of
nics for network id=212 on -1
2014-09-11 08:55:30,051 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Asking VirtualRouter to
release
NicProfile[116-86-8429f13f-7c3b-4b8a-aea3-9b8e9a3f4c03-10.1.1.19-null
2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully released
network resources for the vm VM[User|giannitest]
2014-09-11 08:55:30,051 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully cleanued up
resources for the vm VM[User|giannitest] in Starting state
2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
clusters: [], hosts: [7]
2014-09-11 08:55:30,058 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' provided is
in avoid set, DeploymentPlanner cannot allocate the VM, returning.
2014-09-11 08:55:30,214 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
:Starting to Stopped with event: OperationFailedvm's original host id:
null new host id: null host id before state transition: 7
2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total CPU:
36256 and CPU after applying overprovisioning: 36256
2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Hosts's actual total RAM:
50639785984 and RAM after applying overprovisioning: 50639785984
2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release cpu from host: 7,
old used: 26500,reserved: 0, actual total: 36256, total with
overprovisioning: 36256; new used: 26000,reserved:0;
movedfromreserved: false,moveToReserveredfalse
2014-09-11 08:55:30,218 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) release mem from host: 7,
old used: 21072183296,reserved: 0, total: 50639785984; new used:
20535312384,reserved:0; movedfromreserved: false,moveToReserveredfalse
2014-09-11 08:55:30,302 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
:Stopped to Starting with event: StartRequestedvm's original host id:
null new host id: null host id before state transition: null
2014-09-11 08:55:30,302 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Successfully transitioned
to start state for VM[User|giannitest] reservation id =
4b6c2f8d-a34c-4b86-a0d4-acd5ea87353a
2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Trying to deploy VM, vm
has dcId: 1 and podId: 1
2014-09-11 08:55:30,340 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
clusters: [], hosts: [7]
2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Deploy avoids pods: [],
clusters: [], hosts: [7]
2014-09-11 08:55:30,348 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) DataCenter id = '1' provided is
in avoid set, DeploymentPlanner cannot allocate the VM, returning.
2014-09-11 08:55:30,435 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
:Starting to Stopped with event: OperationFailedvm's original host id:
null new host id: null host id before state transition: null
2014-09-11 08:55:30,477 DEBUG [c.c.v.UserVmManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) Destroying vm
VM[User|giannitest] as it failed to create on Host with Id:null
2014-09-11 08:55:30,609 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de) VM state transitted from
:Stopped to Error with event: OperationFailedToErrorvm's original host
id: null new host id: null host id before state transition: null
2014-09-11 08:55:30,809 WARN  [o.a.c.alerts]
(Job-Executor-55:ctx-b5c34ec7 ctx-3715f5de)  alertType:: 8 //
dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Failed
to deploy Vm with Id: 86, on Host with Id: null
2014-09-11 08:55:30,932 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-1:null) Ping from 6
2014-09-11 08:55:31,078 ERROR [c.c.a.ApiAsyncJobDispatcher]
(Job-Executor-55:ctx-b5c34ec7) Unexpected exception while executing
org.apache.cloudstack.api.command.user.vm.DeployVMCmd
com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM due
to insufficient capacity
         at
com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:
605)
         at
org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirt
ualMachine(VMEntityManagerImpl.java:237)
         at
org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deplo
y(VirtualMachineEntityImpl.java:207)
         at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:35
81)
         at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:31
61)
         at
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:31
47)
         at sun.reflect.GeneratedMethodAccessor587.invoke(Unknown
Source) ...


Il 01/08/2014 11:44, Geoff Higginbottom ha scritto:
Ugo,

Login to the VR and tail the messages log then attempt to start the VM,
you should see various messages re DHCP updating etc in the VR.  If you
don't see the VR update try restarting the network Iain the 'cleanup'
option, this will create a new VR.
Regards

Geoff Higginbottom
CTO / Cloud Architect

D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603
0540<tel:+442036030540> | M: +447968161581<tel:+447968161581>

geoff.higginbot...@shapeblue.com<mailto:geoff.higginbottom@shapeblue.c
om><mailto:geoff.higginbot...@shapeblue.com> |
www.shapeblue.com<http://www.shapeblue.com><htp://www.shapeblue.com/>
| Twitter:@cloudstackguru<https://twitter.com/#!/cloudstackguru>

ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N
4HS<x-apple-data-detectors://5>


On 1 Aug 2014, at 09:06, "Ugo Vasi"
<ugo.v...@procne.it<mailto:ugo.v...@procne.it><mailto:ugo.v...@procne.it>>
wrote:
Hi Geoff,
I tried restarting r-29-VM but the problem is still blocking creation of
new VM in that isolated network.
Creating a new VM in another isolated network, it works. Where can I look
at to find the problem?


Il 31/07/2014 21:53, Geoff Higginbottom ha scritto:
Ugo,

Looks like CS is failing to update the VR with details for the new VM, try
restarting the VR.
Regards

Geoff Higginbottom
CTO / Cloud Architect


D: +44 20 3603 0542<tel:+442036030542> | S: +44 20 3603
0540<tel:+442036030540> | M: +447968161581<tel:+447968161581>

geoff.higginbot...@shapeblue.com<mailto:geoff.higginbottom@shapeblue.c
om><mailto:geoff.higginbot...@shapeblue.com><mailto:geoff.higginbottom
@shapeblue.com> |
www.shapeblue.com<http://www.shapeblue.com><http://www.shapeblue.com><
htp://www.shapeblue.com/>

ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N
4HS<x-apple-data-detectors://37>



On 31 Jul 2014, at 15:40, "Ugo Vasi"
<ugo.v...@procne.it<mailto:ugo.v...@procne.it><mailto:ugo.v...@procne.it><ma
ilto:ugo.v...@procne.it>> wrote:
Hi everybody,
I installed a CloudStack 4.3.0 system with a manager (in a virtual
machine), two NFS primary storage, one NFS secondary storare and two host
agent on ubuntu distribution.
The cluster have one advanced zone without security group configuration
with KVM hipervisors.
I create some VM but now does not allow me to create more giving the error
message "Unable to start a VM due to insufficient capacity".
I am sure that the resources (ip, ram, cpu and storage) are enough to
create new instances. In dashboard I see all ok.
>From the management-server.log I see this errors messages (see attachment
for full version) when I try to deploy a new virtual machine with 1x1GHz
cpu, 1GB or RAM and 20GB disk space:
2014-07-31 14:59:11,983 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-12:null) Seq 1-684594119: Processing:  { Ans: ,
MgmtId: 77916390353436, via: 1, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":true,"wait":0}},{"com.cloud.a
gent.api.Answer":{"result":false,"details":"VmDataCommand failed,
check agent logs","wait":0}}] }
2014-07-31 14:59:11,983 INFO  [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) Unable to contact resource.
com.cloud.exception.ResourceUnavailableException: Resource
[DataCenter:1] is unreachable: Unable to apply userdata and password
entry on router
2014-07-31 14:59:12,453 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) VM state transitted from
:Starting to Stopped with event: OperationFailedvm's original host id:
null new host id: null host id before state transition: 7
2014-07-31 14:59:12,677 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) VM state transitted from
:Starting to Stopped with event: OperationFailedvm's original host id:
null new host id: null host id before state transition: null
2014-07-31 14:59:12,708 DEBUG [c.c.v.UserVmManagerImpl]
(Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) Destroying vm
VM[User|csweb2] as it failed to create on Host with Id:null
2014-07-31 14:59:12,738 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-26:ctx-ab3aee38 ctx-d8600c16) VM state transitted from
:Stopped to Error with event: OperationFailedToErrorvm's original host
id: null new host id: null host id before state transition: null
2014-07-31 14:59:12,929 WARN  [o.a.c.alerts]
(Job-Executor-26:ctx-ab3aee38 ctx-d8600c16)  alertType:: 8 //
dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Failed
to deploy Vm with Id: 70, on Host with Id: null
2014-07-31 14:59:13,159 ERROR [c.c.a.ApiAsyncJobDispatcher]
(Job-Executor-26:ctx-ab3aee38) Unexpected exception while executing
org.apache.cloudstack.api.command.user.vm.DeployVMCmd
com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM
due to insufficient capacity Caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to
create a deployment for VM[User|csweb2]Scope=interface
com.cloud.dc.DataCenter; id=1
2014-07-31 14:59:13,161 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(Job-Executor-26:ctx-ab3aee38) Complete async job-783, jobStatus:
FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
[],"errorcode":530,"errortext":"Unable to start a VM due to
insufficient capacity"}
2014-07-31 14:59:46,280 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Job-Executor-27:ctx-82be396e ctx-41f0cf24) Stopped called on
VM[User|csweb2] but the state is Error
2014-07-31 14:59:46,330 DEBUG [c.c.c.CapacityManagerImpl]
(Job-Executor-27:ctx-82be396e ctx-41f0cf24) VM state transitted from
:Error to Expunging with event: DestroyRequestedvm's original host id:
null new host id: null host id before state transition: null

I tried to search in others log file ("VmDataCommand failed, check agent
logs"), but I don't found nothing...
Can anyone Help me?

--

   U g o   V a s i
<ugo.v...@procne.it<mailto:ugo.v...@procne.it><mailto:ugo.v...@procne.it><ma
ilto:ugo.v...@procne.it>>
   P r o c n e  s.r.l    >)
   via Cotonificio 45  33010 Tavagnacco IT
   phone: +390432486523 fax: +390432486523

Le informazioni contenute in questo messaggio sono riservate e
confidenziali ed ? vietata la diffusione in qualunque modo eseguita.
Qualora Lei non fosse la persona a cui il presente messaggio ?
destinato, La invitiamo ad eliminarlo e a non leggerlo, dandocene
gentilmente comunicazione.
Per qualsiasi informazione si prega di contattare
supp...@procne.it<mailto:supp...@procne.it><mailto:supp...@procne.it><mailto
:supp...@procne.it> .
Rif. D.L. 196/2003




<management-server.log>
Find out more about ShapeBlue and our range of CloudStack related
services

IaaS Cloud Design &
Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge - rapid IaaS deployment
framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure
Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training
Courses<http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or
opinions expressed are solely those of the author and do not necessarily
represent those of Shape Blue Ltd or related companies. If you are not the
intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender
if you believe you have received this email in error. Shape Blue Ltd is a
company incorporated in England & Wales. ShapeBlue Services India LLP is a
company incorporated in India and is operated under license from Shape Blue
Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a
company registered by The Republic of South Africa and is traded under
license from Shape Blue Ltd. ShapeBlue is a registered trademark.





--

   U g o   V a s i
<ugo.v...@procne.it<mailto:ugo.v...@procne.it><mailto:ugo.v...@procne.it>>
   P r o c n e  s.r.l    >)
   via Cotonificio 45  33010 Tavagnacco IT
   phone: +390432486523 fax: +390432486523

Le informazioni contenute in questo messaggio sono riservate e
confidenziali ed ? vietata la diffusione in qualunque modo eseguita.
Qualora Lei non fosse la persona a cui il presente messaggio ?
destinato, La invitiamo ad eliminarlo e a non leggerlo, dandocene
gentilmente comunicazione.
Per qualsiasi informazione si prega di contattare
supp...@procne.it<mailto:supp...@procne.it><mailto:supp...@procne.it> .
Rif. D.L. 196/2003





Find out more about ShapeBlue and our range of CloudStack related
services

IaaS Cloud Design &
Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge - rapid IaaS deployment
framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure
Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training
Courses<http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or
opinions expressed are solely those of the author and do not necessarily
represent those of Shape Blue Ltd or related companies. If you are not the
intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender
if you believe you have received this email in error. Shape Blue Ltd is a
company incorporated in England & Wales. ShapeBlue Services India LLP is a
company incorporated in India and is operated under license from Shape Blue
Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a
company registered by The Republic of South Africa and is traded under
license from Shape Blue Ltd. ShapeBlue is a registered trademark.





--

   U g o   V a s i    <ugo.v...@procne.it<mailto:ugo.v...@procne.it>>
   P r o c n e  s.r.l    >)
   via Cotonificio 45  33010 Tavagnacco IT
   phone: +390432486523 fax: +390432486523

Le informazioni contenute in questo messaggio sono riservate e
confidenziali ed ? vietata la diffusione in qualunque modo eseguita.
Qualora Lei non fosse la persona a cui il presente messaggio ?
destinato, La invitiamo ad eliminarlo e a non leggerlo, dandocene
gentilmente comunicazione.
Per qualsiasi informazione si prega di contattare
supp...@procne.it<mailto:supp...@procne.it> .
Rif. D.L. 196/2003


Find out more about ShapeBlue and our range of CloudStack related
services

IaaS Cloud Design &
Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge - rapid IaaS deployment
framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure
Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training
Courses<http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or
opinions expressed are solely those of the author and do not necessarily
represent those of Shape Blue Ltd or related companies. If you are not the
intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender
if you believe you have received this email in error. Shape Blue Ltd is a
company incorporated in England & Wales. ShapeBlue Services India LLP is a
company incorporated in India and is operated under license from Shape Blue
Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a
company registered by The Republic of South Africa and is traded under
license from Shape Blue Ltd. ShapeBlue is a registered trademark.





Reply via email to