Chandan Purushothama created CLOUDSTACK-7725: ------------------------------------------------
Summary: [Automation][HyperV] After sometime System VM 'agents' get disconnected due to Ping Issues and get shut down Key: CLOUDSTACK-7725 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7725 Project: CloudStack Issue Type: Bug Security Level: Public (Anyone can view this level - this is the default.) Components: Automation Affects Versions: 4.5.0 Reporter: Chandan Purushothama Priority: Blocker Fix For: 4.5.0 There is something still wrong with the Agents in System VMs on HyperV Setup. The System VM agents got disconnected due to Ping Issues and VMs got shut down. I found this observation when I noticed that all the SSVM Test cases failed. Someone has to look at what is happening between management server and agents communication. I don’t think it is anything wrong with the Setup based on the success of other test suites. Disconnection Information Log is as shown below: 2014-10-14 19:21:50,378 INFO [c.c.a.m.AgentManagerImpl] (AgentMonitor-1:ctx-72fa526b) Found the following agents behind on ping: [3, 4] 2014-10-14 19:21:50,380 WARN [c.c.a.m.AgentManagerImpl] (AgentMonitor-1:ctx-72fa526b) Disconnect agent for CPVM/SSVM due to physical connection close. host: 3 2014-10-14 19:21:50,381 INFO [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Host 3 is disconnecting with event PingTimeout 2014-10-14 19:21:50,382 WARN [c.c.a.m.AgentManagerImpl] (AgentMonitor-1:ctx-72fa526b) Disconnect agent for CPVM/SSVM due to physical connection close. host: 4 2014-10-14 19:21:50,383 INFO [c.c.a.m.AgentManagerImpl] (AgentTaskPool-6:ctx-f560f275) Host 4 is disconnecting with event PingTimeout 2014-10-14 19:21:50,384 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) The next status of agent 3 is Alert, current status is Up 2014-10-14 19:21:50,384 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Deregistering link for 3 with state Alert 2014-10-14 19:21:50,384 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Remove Agent : 3 2014-10-14 19:21:50,384 DEBUG [c.c.a.m.ConnectedAgentAttache] (AgentTaskPool-5:ctx-693d2497) Processing Disconnect. 2014-10-14 19:21:50,384 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.hypervisor.xenserver.discoverer.XcpServerDiscoverer 2014-10-14 19:21:50,384 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.hypervisor.hyperv.discoverer.HypervServerDiscoverer 2014-10-14 19:21:50,384 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.storage.secondary.SecondaryStorageListener 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-6:ctx-f560f275) The next status of agent 4 is Alert, current status is Up 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-6:ctx-f560f275) Deregistering link for 4 with state Alert 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-6:ctx-f560f275) Remove Agent : 4 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.ConnectedAgentAttache] (AgentTaskPool-6:ctx-f560f275) Processing Disconnect. 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.vm.ClusteredVirtualMachineManagerImpl 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: org.apache.cloudstack.engine.orchestration.NetworkOrchestrator 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.storage.listener.StoragePoolMonitor 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.hypervisor.vmware.manager.VmwareManagerImpl 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.deploy.DeploymentPlanningManagerImpl 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-6:ctx-f560f275) Sending Disconnect to listener: com.cloud.hypervisor.xenserver.discoverer.XcpServerDiscoverer 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-6:ctx-f560f275) Sending Disconnect to listener: com.cloud.hypervisor.hyperv.discoverer.HypervServerDiscoverer 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-6:ctx-f560f275) Sending Disconnect to listener: com.cloud.storage.secondary.SecondaryStorageListener 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.network.security.SecurityGroupListener 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.network.SshKeysDistriMonitor 2014-10-14 19:21:50,385 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl 2014-10-14 19:21:50,386 DEBUG [c.c.a.m.AgentManagerImpl] (AgentTaskPool-5:ctx-693d2497) Sending Disconnect to listener: com.cloud.consoleproxy.ConsoleProxyListener 2014-10-14 19:21:50,387 INFO [c.c.c.ConsoleProxyManagerImpl] (AgentTaskPool-5:ctx-693d2497) Console proxy agent disconnected, proxy: v-1-VM 2014-10-14 19:21:50,391 WARN [o.a.c.f.j.AsyncJobExecutionContext] (AgentTaskPool-5:ctx-693d2497) Job is executed without a context, setup psudo job for the executing thread 2014-10-14 19:21:50,397 WARN [o.a.c.f.j.AsyncJobExecutionContext] (AgentTaskPool-6:ctx-f560f275) Job is executed without a context, setup psudo job for the executing thread Thank you, Chandan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)