same issue: CLOUDSTACK-5834
2014/1/17 Pierre-Luc Dion <pd...@cloudops.com> > Hi, > > We've upgrade our Cloudstack environment from 4.1 to 4.2.1, it is > currently managing XenServer 6.0.2 clusters. > > 1. Since we have upgraded to 4.2.1 we noticed that Secondary Storage Name > have been rename and we now see UUID's of SS. > 2. On one cluster that is using local-Storage and Shared Storage we have > removed on host from the XenServer Pool, since then, this error keep > repeating in the management-server.log: > > 2014-01-16 17:01:17,898 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-437:null) Seq 1-590413869: Response Received: > 2014-01-16 17:01:17,898 DEBUG [agent.transport.Request] > (StatsCollector-2:null) Seq 1-590413869: Received: { Ans: , MgmtId: > 152067623530867, via: 1, Ver: v1, Flags: 10, { GetVmStatsAnswer } } > 2014-01-16 17:01:17,949 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-73:null) Seq 2-1599340596: Executing request > 2014-01-16 17:01:18,191 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-320:null) Ping from 69 > 2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase] > (DirectAgent-73:null) Vm cpu utilization 0.006406249999999999 > 2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase] > (DirectAgent-73:null) Vm cpu utilization 12.9175 > 2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase] > (DirectAgent-73:null) Vm cpu utilization 24.08 > 2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase] > (DirectAgent-73:null) Vm cpu utilization 0.29500000000000004 > 2014-01-16 17:01:18,369 DEBUG [xen.resource.CitrixResourceBase] > (DirectAgent-73:null) Vm cpu utilization 23.8075 > 2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase] > (DirectAgent-73:null) Vm cpu utilization 10.09875 > 2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase] > (DirectAgent-73:null) Vm cpu utilization 0.29828125 > 2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase] > (DirectAgent-73:null) Vm cpu utilization 2.6500000000000004 > 2014-01-16 17:01:18,370 DEBUG [xen.resource.CitrixResourceBase] > (DirectAgent-73:null) Vm cpu utilization 0.8375 > 2014-01-16 17:01:18,507 WARN [xen.resource.CitrixResourceBase] > (DirectAgent-73:null) Error while collecting disk stats from : > You gave an invalid object reference. The object may have recently been > deleted. The class parameter gives the type of reference given, and the > handle parameter echoes the bad value given. > at com.xensource.xenapi.Types.checkResponse(Types.java:209) > at com.xensource.xenapi.Connection.dispatch(Connection.java:368) > at > com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:909) > at com.xensource.xenapi.VBDMetrics.getIoReadKbs(VBDMetrics.java:210) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.getVmStats(CitrixResourceBase.java:2791) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2691) > at > com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:497) > at > com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59) > at > com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:701) > 2014-01-16 17:01:18,509 DEBUG [agent.manager.DirectAgentAttache] > (DirectAgent-73:null) Seq 2-1599340596: Response Received: > > > Also, Because we were planning to upgrade the pool using Local-Storage we > start moving VM into another Primary Storage using regular procedure of > turning off Instances, and moving them as Admin account. this keep failing > but also delete Instance DISK which result of having data lost. > > Now, we can't see our hosts list and System VM list from the > Infrastructure tab (see attachements), we only see ERROR. > > here is the apilog.log when trying to show hosts: > > 2014-01-16 17:13:35,337 INFO [cloud.api.ApiServer] > (catalina-exec-18:null) (userId=2 accountId=2 > sessionId=4F907F8E77BE76ADF46BB4FBE6B23BC0) 172.24.0.20 -- GET > command=listHosts&response=json&sessionkey=tPdf1uSuogiM1ubmRFuPuFV%2FLzM%3D&page=1&pageSize=20&type=routing&listAll=true&_=1389910415201 > 530 null > > is their any other logs to look at to understands what's broken because > the upper stacktrace is clueless for me and their is no errors in the > management-server.log when trying to display hosts. > > Regards, > > Pierre-Luc Dion > Architecte de Solution Cloud | Cloud Solutions Architect > 514-447-3456, 1101 > - - - > > *CloudOps*420 rue Guy > Montréal QC H3J 1S6 > www.cloudops.com > @CloudOps_ > > > >