Hi, We've upgrade our Cloudstack environment from 4.1 to 4.2.1, it is currently managing XenServer 6.0.2 clusters.
1. Since we have upgraded to 4.2.1 we noticed that Secondary Storage Name have been rename and we now see UUID's of SS. 2. On one cluster that is using local-Storage and Shared Storage we have removed on host from the XenServer Pool, since then, this error keep repeating in the management-server.log: 2014-01-16 17:01:17,898 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-437:null) Seq 1-590413869: Response Received: 2014-01-16 17:01:17,898 DEBUG [agent.transport.Request] (StatsCollector-2:null) Seq 1-590413869: Received: { Ans: , MgmtId: 152067623530867, via: 1, Ver: v1, Flags: 10, { GetVmStatsAnswer } } 2014-01-16 17:01:17,949 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-73:null) Seq 2-1599340596: Executing request 2014-01-16 17:01:18,191 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-320:null) Ping from 69 2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-73:null) Vm cpu utilization 0.006406249999999999 2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-73:null) Vm cpu utilization 12.9175 2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-73:null) Vm cpu utilization 24.08 2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-73:null) Vm cpu utilization 0.29500000000000004 2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-73:null) Vm cpu utilization 23.8075 2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-73:null) Vm cpu utilization 10.09875 2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-73:null) Vm cpu utilization 0.29828125 2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-73:null) Vm cpu utilization 2.6500000000000004 2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase] (DirectAgent-73:null) Vm cpu utilization 0.8375 2014-01-16 17:01:18,507 WARN [xen.resource.CitrixResourceBase] (DirectAgent-73:null) Error while collecting disk stats from : You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given. at com.xensource.xenapi.Types.checkResponse(Types.java:209) at com.xensource.xenapi.Connection.dispatch(Connection.java:368) at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909) at com.xensource.xenapi.VBDMetrics.getIoReadKbs(VBDMetrics.java:210) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getVmStats(CitrixResourceBase.java:2791) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2691) at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:497) at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59) at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701) 2014-01-16 17:01:18,509 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-73:null) Seq 2-1599340596: Response Received: Also, Because we were planning to upgrade the pool using Local-Storage we start moving VM into another Primary Storage using regular procedure of turning off Instances, and moving them as Admin account. this keep failing but also delete Instance DISK which result of having data lost. Now, we can't see our hosts list and System VM list from the Infrastructure tab (see attachements), we only see ERROR. here is the apilog.log when trying to show hosts: 2014-01-16 17:13:35,337 INFO [cloud.api.ApiServer] (catalina-exec-18:null) (userId=2 accountId=2 sessionId=4F907F8E77BE76ADF46BB4FBE6B23BC0) 172.24.0.20 -- GET command=listHosts&response=json&sessionkey=tPdf1uSuogiM1ubmRFuPuFV%2FLzM%3D&page=1&pageSize=20&type=routing&listAll=true&_=1389910415201 530 null is their any other logs to look at to understands what's broken because the upper stacktrace is clueless for me and their is no errors in the management-server.log when trying to display hosts. Regards, Pierre-Luc Dion Architecte de Solution Cloud | Cloud Solutions Architect 514-447-3456, 1101 - - - *CloudOps*420 rue Guy Montréal QC H3J 1S6 www.cloudops.com @CloudOps_