We've upgrade our Cloudstack environment from 4.1 to 4.2.1, it is currently
managing XenServer 6.0.2 clusters.
1. Since we have upgraded to 4.2.1 we noticed that Secondary Storage Name
have been rename and we now see UUID's of SS.
2. On one cluster that is using local-Storage and Shared Storage we have
removed on host from the XenServer Pool, since then, this error keep
repeating in the management-server.log:
2014-01-16 17:01:17,898 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-437:null) Seq 1-590413869: Response Received:
2014-01-16 17:01:17,898 DEBUG [agent.transport.Request]
(StatsCollector-2:null) Seq 1-590413869: Received: { Ans: , MgmtId:
152067623530867, via: 1, Ver: v1, Flags: 10, { GetVmStatsAnswer } }
2014-01-16 17:01:17,949 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-73:null) Seq 2-1599340596: Executing request
2014-01-16 17:01:18,191 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-320:null) Ping from 69
2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-73:null) Vm cpu utilization 0.006406249999999999
2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-73:null) Vm cpu utilization 12.9175
2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-73:null) Vm cpu utilization 24.08
2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-73:null) Vm cpu utilization 0.29500000000000004
2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-73:null) Vm cpu utilization 23.8075
2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-73:null) Vm cpu utilization 10.09875
2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-73:null) Vm cpu utilization 0.29828125
2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-73:null) Vm cpu utilization 2.6500000000000004
2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase]
(DirectAgent-73:null) Vm cpu utilization 0.8375
2014-01-16 17:01:18,507 WARN [xen.resource.CitrixResourceBase]
(DirectAgent-73:null) Error while collecting disk stats from :
You gave an invalid object reference. The object may have recently been
deleted. The class parameter gives the type of reference given, and the
handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:209)
at com.xensource.xenapi.Connection.dispatch(Connection.java:368)
at
com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909)
at com.xensource.xenapi.VBDMetrics.getIoReadKbs(VBDMetrics.java:210)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getVmStats(CitrixResourceBase.java:2791)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2691)
at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:497)
at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)
at
com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
2014-01-16 17:01:18,509 DEBUG [agent.manager.DirectAgentAttache]
(DirectAgent-73:null) Seq 2-1599340596: Response Received:
Also, Because we were planning to upgrade the pool using Local-Storage we
start moving VM into another Primary Storage using regular procedure of
turning off Instances, and moving them as Admin account. this keep failing
but also delete Instance DISK which result of having data lost.
Now, we can't see our hosts list and System VM list from the Infrastructure
tab (see attachements), we only see ERROR.
here is the apilog.log when trying to show hosts:
2014-01-16 17:13:35,337 INFO [cloud.api.ApiServer] (catalina-exec-18:null)
(userId=2 accountId=2 sessionId=4F907F8E77BE76ADF46BB4FBE6B23BC0)
172.24.0.20 -- GET
command=listHosts&response=json&sessionkey=tPdf1uSuogiM1ubmRFuPuFV%2FLzM%3D&page=1&pageSize=20&type=routing&listAll=true&_=1389910415201
530 null
is their any other logs to look at to understands what's broken because the
upper stacktrace is clueless for me and their is no errors in the
management-server.log when trying to display hosts.
Regards,
Pierre-Luc Dion
Architecte de Solution Cloud | Cloud Solutions Architect
514-447-3456, 1101
- - -
*CloudOps*420 rue Guy
Montréal QC H3J 1S6
www.cloudops.com
@CloudOps_
