Ok. I don’t think searching for errors/failures would help. This may not be an error situation from cloudstack perspective.
We need to search for these - [SimpleInvestigator, XenServerInvestigator, KVMInvestigator, PingInvestigator, ManagementIPSysVMInvestigator, HypervInvestigator, VMwareInvestigator]. These investigators are responsible for determining the state of the VM at host disconnect based on which decision to HA is made. Regards, Somesh -----Original Message----- From: Luciano Castro [mailto:luciano.cas...@gmail.com] Sent: Thursday, July 16, 2015 8:34 AM To: users@cloudstack.apache.org Subject: Re: HA feature - KVM - CloudStack 4.5.1 Hi! * Yes, Offer HA. * The status os KVM hosts reported is "Disconnected", and curiously the VM status is still "Running..." I did the same test again: 1) shutdown KVM host 12:13 UTC 2) Logs (management server) [root@1q2 ~]# grep -i -E 'warn|exception|unable|fail|error' /var/log/cloudstack/management/management-server.log|tail -50 2015-07-16 12:13:21,195 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-14:ctx-b1ba1de6 ctx-5c5114f1) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid 2015-07-16 12:13:31,395 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-78cb8329 ctx-ea77f742) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid 2015-07-16 12:13:40,975 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-6e2ab11e ctx-f0bc3e8d) Received unknown parameters for command expungeVirtualMachine. Unknown parameters : projectid 2015-07-16 12:13:40,998 WARN [c.c.a.d.ParamGenericValidationWorker] (API-Job-Executor-54:ctx-a6765444 job-407 ctx-e7d4ae6e) Received unknown parameters for command expungeVirtualMachine. Unknown parameters : projectid 2015-07-16 12:13:44,162 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-ca71fefa ctx-3a46d16f) Received unknown parameters for command queryAsyncJobResult. Unknown parameters : projectid 2015-07-16 12:13:47,163 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-2:ctx-9773fe5c ctx-20f2c53d) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid 2015-07-16 12:13:51,478 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-986e5695 ctx-cfd9dc2a) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid 2015-07-16 12:13:57,139 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-21:ctx-3ed5162d ctx-0b378da3) Received unknown parameters for command listZones. Unknown parameters : projectid 2015-07-16 12:13:58,786 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-ef9b46ae ctx-1ec7c374) Received unknown parameters for command listZones. Unknown parameters : projectid 2015-07-16 12:13:58,991 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-c40727ff ctx-129cff5b) Received unknown parameters for command listHosts. Unknown parameters : projectid 2015-07-16 12:14:01,369 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-9:ctx-d39d30d5 ctx-7d1296f3) Received unknown parameters for command listHosts. Unknown parameters : projectid 2015-07-16 12:14:01,553 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-6:ctx-f02aae15 ctx-b675547d) Received unknown parameters for command listDedicatedHosts. Unknown parameters : projectid 2015-07-16 12:14:01,725 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-25:ctx-4de9059a ctx-15ac62c3) Received unknown parameters for command listConfigurations. Unknown parameters : projectid 2015-07-16 12:14:01,908 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-23:ctx-25fb3589 ctx-d4274e7c) Received unknown parameters for command listOsCategories. Unknown parameters : projectid 2015-07-16 12:14:02,081 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-2:ctx-a90cf3ca ctx-6e7a01fc) Received unknown parameters for command listHosts. Unknown parameters : projectid 2015-07-16 12:14:02,266 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-12:ctx-b8f987fa ctx-8a7f2922) Received unknown parameters for command listDedicatedHosts. Unknown parameters : projectid 2015-07-16 12:14:02,442 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-7:ctx-64be7d4b ctx-36a9864d) Received unknown parameters for command listConfigurations. Unknown parameters : projectid 2015-07-16 12:14:02,650 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-1:ctx-9b0043f1 ctx-36482b14) Received unknown parameters for command listOsCategories. Unknown parameters : projectid 2015-07-16 12:14:14,981 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-5d62538b ctx-31cf1646) Received unknown parameters for command listZones. Unknown parameters : projectid 2015-07-16 12:14:17,445 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-ef67f80f ctx-e953915f) Received unknown parameters for command listZones. Unknown parameters : projectid 2015-07-16 12:14:17,654 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-11:ctx-7a3e6ce3 ctx-7d06a5a4) Received unknown parameters for command listHosts. Unknown parameters : projectid 2015-07-16 12:14:24,537 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-596c9e4b ctx-90642717) Received unknown parameters for command findHostsForMigration. Unknown parameters : projectid 2015-07-16 12:14:28,684 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-20:ctx-2b17bb98 ctx-a489da47) Received unknown parameters for command migrateVirtualMachine. Unknown parameters : projectid 2015-07-16 12:14:28,708 WARN [c.c.a.d.ParamGenericValidationWorker] (API-Job-Executor-55:ctx-0b76e8ae job-409 ctx-d346cca9) Received unknown parameters for command migrateVirtualMachine. Unknown parameters : projectid 2015-07-16 12:14:31,880 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-10:ctx-b9b78739 ctx-bc42f26e) Received unknown parameters for command queryAsyncJobResult. Unknown parameters : projectid 2015-07-16 12:14:34,873 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-22941163 ctx-737b322d) Received unknown parameters for command queryAsyncJobResult. Unknown parameters : projectid 2015-07-16 12:15:47,758 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-25:ctx-bd9f0a98 ctx-97c7b112) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid 2015-07-16 12:15:50,466 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-c10f59b2 ctx-405d4cf3) Received unknown parameters for command listZones. Unknown parameters : projectid 2015-07-16 12:15:54,480 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-9bb4f185 ctx-9f2b37a9) Received unknown parameters for command listZones. Unknown parameters : projectid 2015-07-16 12:15:54,684 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-11:ctx-6a51a841 ctx-a0d37d54) Received unknown parameters for command listHosts. Unknown parameters : projectid 2015-07-16 12:15:58,044 WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-656c8c86 ctx-aef2da97) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid 2015-07-16 12:16:37,381 DEBUG [c.c.c.CapacityManagerImpl] (API-Job-Executor-57:ctx-e1ed3c62 job-412 ctx-e861a024) STATS: Failed to alloc resource from host: 3 reservedCpu: 0, requested cpu: 4000, reservedMem: 0, requested mem: 2147483648 2015-07-16 12:21:18,239 WARN [c.c.a.m.AgentManagerImpl] (AgentMonitor-1:ctx-9eef326e) Disconnect agent for CPVM/SSVM due to physical connection close. host: 9 2015-07-16 12:21:18,243 INFO [c.c.u.e.CSExceptionErrorCode] (StatsCollector-3:ctx-eafd1ed4) Could not find exception: com.cloud.exception.OperationTimedoutException in error code list for exceptions 2015-07-16 12:21:18,243 WARN [c.c.a.m.AgentAttache] (StatsCollector-3:ctx-eafd1ed4) Seq 9-8454382400481264659: Timed out on null 2015-07-16 12:21:18,243 DEBUG [o.a.c.s.RemoteHostEndPoint] (StatsCollector-3:ctx-eafd1ed4) Failed to send command, due to Agent:9, com.cloud.exception.OperationTimedoutException: Commands 8454382400481264659 to Host 9 timed out after 3600 2015-07-16 12:21:18,243 ERROR [c.c.s.StatsCollector] (StatsCollector-3:ctx-eafd1ed4) Error trying to retrieve storage stats com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:9, com.cloud.exception.OperationTimedoutException: Commands 8454382400481264659 to Host 9 timed out after 3600 2015-07-16 12:22:09,951 WARN [o.a.c.alerts] (HA-1:ctx-6b60e29b) alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: No usage server process running [root@1q2 ~]# grep -i -E 'warn|exception|unable|fail|error' /var/log/cloudstack/management/catalina.out|tail -50 WARN [o.a.c.alerts] (HA-1:ctx-830dab68) alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: No usage server process running WARN [o.a.c.alerts] (HA-1:ctx-e45484a1) alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: No usage server process running WARN [o.a.c.alerts] (HA-1:ctx-c27a224e) alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: No usage server process running WARN [o.a.c.alerts] (HA-1:ctx-259fdd28) alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: No usage server process running WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-40e06aca ctx-d419dc55) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-4b3562fd ctx-3341445a) Received unknown parameters for command listZones. Unknown parameters : projectid WARN [o.a.c.alerts] (HA-1:ctx-f34fcc40) alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: No usage server process running WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-8f8e296e ctx-a6f598c1) Received unknown parameters for command listZones. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-8825ea69 ctx-fb7312c0) Received unknown parameters for command listZones. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-9:ctx-d6a4922e ctx-b3bad36d) Received unknown parameters for command listHosts. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-f7e28168 ctx-e966ba7a) Received unknown parameters for command listServiceOfferings. Unknown parameters : listall WARN [o.a.c.alerts] (HA-1:ctx-42103a07) alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: No usage server process running WARN [o.a.c.alerts] (HA-1:ctx-5cac1938) alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: No usage server process running WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-14:ctx-b1ba1de6 ctx-5c5114f1) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-78cb8329 ctx-ea77f742) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-6e2ab11e ctx-f0bc3e8d) Received unknown parameters for command expungeVirtualMachine. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (API-Job-Executor-54:ctx-a6765444 job-407 ctx-e7d4ae6e) Received unknown parameters for command expungeVirtualMachine. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-ca71fefa ctx-3a46d16f) Received unknown parameters for command queryAsyncJobResult. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-2:ctx-9773fe5c ctx-20f2c53d) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-986e5695 ctx-cfd9dc2a) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-21:ctx-3ed5162d ctx-0b378da3) Received unknown parameters for command listZones. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-19:ctx-ef9b46ae ctx-1ec7c374) Received unknown parameters for command listZones. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-3:ctx-c40727ff ctx-129cff5b) Received unknown parameters for command listHosts. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-9:ctx-d39d30d5 ctx-7d1296f3) Received unknown parameters for command listHosts. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-6:ctx-f02aae15 ctx-b675547d) Received unknown parameters for command listDedicatedHosts. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-25:ctx-4de9059a ctx-15ac62c3) Received unknown parameters for command listConfigurations. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-23:ctx-25fb3589 ctx-d4274e7c) Received unknown parameters for command listOsCategories. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-2:ctx-a90cf3ca ctx-6e7a01fc) Received unknown parameters for command listHosts. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-12:ctx-b8f987fa ctx-8a7f2922) Received unknown parameters for command listDedicatedHosts. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-7:ctx-64be7d4b ctx-36a9864d) Received unknown parameters for command listConfigurations. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-1:ctx-9b0043f1 ctx-36482b14) Received unknown parameters for command listOsCategories. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-5d62538b ctx-31cf1646) Received unknown parameters for command listZones. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-ef67f80f ctx-e953915f) Received unknown parameters for command listZones. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-11:ctx-7a3e6ce3 ctx-7d06a5a4) Received unknown parameters for command listHosts. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-4:ctx-596c9e4b ctx-90642717) Received unknown parameters for command findHostsForMigration. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-20:ctx-2b17bb98 ctx-a489da47) Received unknown parameters for command migrateVirtualMachine. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (API-Job-Executor-55:ctx-0b76e8ae job-409 ctx-d346cca9) Received unknown parameters for command migrateVirtualMachine. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-10:ctx-b9b78739 ctx-bc42f26e) Received unknown parameters for command queryAsyncJobResult. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-22941163 ctx-737b322d) Received unknown parameters for command queryAsyncJobResult. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-25:ctx-bd9f0a98 ctx-97c7b112) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-16:ctx-c10f59b2 ctx-405d4cf3) Received unknown parameters for command listZones. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-17:ctx-9bb4f185 ctx-9f2b37a9) Received unknown parameters for command listZones. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-11:ctx-6a51a841 ctx-a0d37d54) Received unknown parameters for command listHosts. Unknown parameters : projectid WARN [c.c.a.d.ParamGenericValidationWorker] (catalina-exec-22:ctx-656c8c86 ctx-aef2da97) Received unknown parameters for command listNetworkOfferings. Unknown parameters : projectid WARN [c.c.a.m.AgentManagerImpl] (AgentMonitor-1:ctx-9eef326e) Disconnect agent for CPVM/SSVM due to physical connection close. host: 9 INFO [c.c.u.e.CSExceptionErrorCode] (StatsCollector-3:ctx-eafd1ed4) Could not find exception: com.cloud.exception.OperationTimedoutException in error code list for exceptions WARN [c.c.a.m.AgentAttache] (StatsCollector-3:ctx-eafd1ed4) Seq 9-8454382400481264659: Timed out on null ERROR [c.c.s.StatsCollector] (StatsCollector-3:ctx-eafd1ed4) Error trying to retrieve storage stats com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due to Agent:9, com.cloud.exception.OperationTimedoutException: Commands 8454382400481264659 to Host 9 timed out after 3600 WARN [o.a.c.alerts] (HA-1:ctx-6b60e29b) alertType:: 13 // dataCenterId:: 0 // podId:: 0 // clusterId:: null // message:: No usage server process running PS: I was loggen in the vm with offer HA, and I started a ping. I lost the connection after 30 icmp_seq. Thanks! On Wed, Jul 15, 2015 at 7:11 PM, Somesh Naidu <somesh.na...@citrix.com> wrote: > "TAG H.A checked" - did you mean "Offer HA"? > > Also, after you shutdown host A, what was the status of the host reported > in Cloudstack? Also, did you check the management server logs to see what > the investigators reported? > > Regards, > Somesh > > -----Original Message----- > From: Luciano Castro [mailto:luciano.cas...@gmail.com] > Sent: Wednesday, July 15, 2015 2:46 PM > To: users@cloudstack.apache.org > Subject: Fwd: HA feature - KVM - CloudStack 4.5.1 > > Hi there, > > I´m new in the list, and I´m using 4.5.1 CloudStack version, and I > started making some tests with H.A feature for VMs, > but I didn´t have success. I don´t know if it´s a bug, or something that > I´m doing wrong. > > Scenario: I have 1 manager server and 2 KVM hosts ('A' and 'B'). > > > > 1) I created a "service offering" (computing offering) with the TAG H.A > checked. > > 2) then, I created a VM with this offering, and the VM is in KVM 'A'. > > 3) I did a shutdown in the KVM host 'A', and I expected that this VM, after > some minutes will be available in another KVM host ('B') > > But it didn´t happened. > > Any ideias?? > > thanks! > > -- > Luciano Castro > > > > -- > Luciano Castro > -- Luciano Castro
