Hi,
I wonder if somebody experienced a similar issue. We have a very busy dev
cloudstack installation with 10 hosts, 3 clusters and more than 300 user VM's
running across the clusters. The cpu over-provisioning is set to 3. From time
to time we are getting the following an error when starting the VM.
2015-01-26 01:45:25,547 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-364:ctx-578a4e5d) Task failed! Task record: uuid:
ff81a41e-3340-e7d5-6f8c-c99d4a910bb0
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Mon Jan 26 01:45:24 GMT 2015
finished: Mon Jan 26 01:45:24 GMT 2015
status: failure
residentOn: com.xensource.xenapi.Host@fcaebca8
progress: 1.0
type: <none/>
result:
errorInfo: [HOST_NOT_ENOUGH_FREE_MEMORY, 1587544064, 1446559744]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
I traced the job executor for the VM and it looks like the process have
reported that the host have enough RAM 2060627968 to start the VM, which
requestes 1572864000 but for some reason the command failed on the Xen server.
2015-01-26 01:45:20,005 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Work-Job-Executor-8:ctx-c47b7a2f job-205846/job-205847 ctx-ea6dc373)
Deployment found - P0=VM[User|i-3-160816-VM],
P0=Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))]
:
Dest[Zone(1)-Pod(1)-Cluster(5)-Host(35)-Storage(Volume(158708|ROOT-->Pool(30))]
2015-01-26 01:45:20,124 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-8:ctx-c47b7a2f job-205846/job-205847 ctx-ea6dc373) VM state
transitted from :Starting to Starting with event: OperationRetryvm's original
host id: null new host id: 35 host id before state transition: null
2015-01-26 01:45:20,138 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-8:ctx-c47b7a2f job-205846/job-205847 ctx-ea6dc373) Hosts's
actual total CPU: 44688 and CPU after applying overprovisioning: 58094
2015-01-26 01:45:20,138 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-8:ctx-c47b7a2f job-205846/job-205847 ctx-ea6dc373) We are
allocating VM, increasing the used capacity of this host:35
2015-01-26 01:45:20,138 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-8:ctx-c47b7a2f job-205846/job-205847 ctx-ea6dc373) Current
Used CPU: 52500 , Free CPU:5594 ,Requested CPU: 1500
2015-01-26 01:45:20,138 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-8:ctx-c47b7a2f job-205846/job-205847 ctx-ea6dc373) Current
Used RAM: 61220061184 , Free RAM:2060627968 ,Requested RAM: 1572864000
2015-01-26 01:45:20,138 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-8:ctx-c47b7a2f job-205846/job-205847 ctx-ea6dc373) CPU STATS
after allocation: for host: 35, old used: 52500, old reserved: 0, actual total:
44688, total with o
verprovisioning: 58094; new used:54000, reserved:0; requested
cpu:1500,alloc_from_last:false
2015-01-26 01:45:20,138 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-8:ctx-c47b7a2f job-205846/job-205847 ctx-ea6dc373) RAM STATS
after allocation: for host: 35, old used: 61220061184, old reserved: 0, total:
63280689152; new use
d: 62792925184, reserved: 0; requested mem: 1572864000,alloc_from_last:false
Please let me know if somebody had a similar problem and managed to fix it. We
are running xenserver6.2 with cloudstack 4.3.0.1
Bast regards
Yuri
