Re: HA State: Ineligible
Hi Jeremy, I remember seeing this is the documentation, the section about Host HA, is just a duplication of VM HA. I think this was copied and pasted in error by the author. For Host HA to work, you would need at least 3 hosts, all with Out of band management enabled. Regards Darrin On 2021/06/11 09:03:03, Jeremy Hansen wrote: > > I'm trying to play with HA. I've enabled it via the interface but the HA > state is labeled as Ineligible.> > > I'm specifically interested in this:> > > HA for Hosts> > > The user can specify a virtual machine as HA-enabled. By default, all virtual > router VMs and Elastic Load Balancing VMs are automatically configured as > HA-enabled. When an HA-enabled VM crashes, CloudStack detects the crash and > restarts the VM automatically within the same Availability Zone. HA is never > performed across different Availability Zones. CloudStack has a conservative > policy towards restarting VMs and ensures that there will never be two > instances of the same VM running at the same time. The Management Server > attempts to start the VM on another Host in the same cluster.> > > > My assumption is if a VM Host dies, whatever guests that were running on that > host would automatically move to an available VM host. Maybe I'm > misinterpreting.> > > Thanks> > -jeremy> >
Re: HA State: Ineligible
because it might still run or be restarted on the unreachable host. On Fri, Jun 18, 2021 at 12:06 PM Jeremy Hansen wrote: > Just seems strange. If I have centralized storage and I can migrate live > vm’s, why wouldn’t I be able to take the risk to migrate a dead vm guest, > as in, just start it on another available host and start it and consider it > migrated. If the original vm host happens to come back up and the instance > has already migrated, just leave it alone. Seems like that would go a long > way for high availability. I assume i/o fencing becomes an issue. I’d > prefer some kind of hook so if a VM host seems dead beyond, say, 5 minutes, > i/o fence it by triggering a power off from the PDU and bring up the VM > guests somewhere else. > > -jeremy > > > > On Jun 18, 2021, at 2:42 AM, Daan Hoogland > wrote: > > > > :D > > I'm sorry, when your bike is broken you have to walk. detection if the > > exact situation is the issue here, good luck in your search! > > > > On Fri, Jun 18, 2021 at 11:31 AM Jeremy Hansen > wrote: > > > >> I guess I’m looking for a recovery scenario where the dead vm host is > not > >> coming back, failed disk, caught on fire, and a reboot isn’t going to > help. > >> > >> Thanks > >> > >>> On Jun 18, 2021, at 1:41 AM, Daan Hoogland > >> wrote: > >>> > >>> Jemery, > >>> If you don't have IPMI then ACS can not know for sure that the VM won't > >>> come back. If it comes back the VM would be running twice and this must > >> be > >>> prevented at all costs. Maybe I am missing some functionality, and > >> someone > >>> else can give additional options. > >>> > On Fri, Jun 18, 2021 at 10:21 AM Jeremy Hansen > >> wrote: > > I pasted that from the documentation. My end goal is if a VM host > dies > completely and I’m not available to fix it, I would like the VM guests > >> that > were running on the failed host to automatically migrate to an > >> available VM > host so the guest instances continue to run. Perhaps that’s not how > it > works. The hosts I’m using for testing do not have any kind of IPMI > supported out of band management. They do have network enabled PDUs > but > let’s just say the VM host is gone completely. How do I get the VM > >> guests > that were running on the failed host back up and running without my > intervention? I guess I wrongly assumed Cloudstack would handle this > >> case > by just starting the VMs on another available host machine after some > >> kind > of failed heartbeat threshold. > > Thanks > -jeremy > > > On Jun 18, 2021, at 1:09 AM, Daan Hoogland > wrote: > > > > Jeremy, > > I don't fully understand your question. You say you are interested in > > HostHA specifically but then you ask about restarting VMs when a host > dies. > > This would not be safe as we can't be sure a host really dies unless > >> you > > have HostHA enabled. Consequently you can't guarantee the VM won't > suddenly > > re-apear when the host is seen running again. So keep these things > > separated. > > HostHA is for rebooting suspect hosts, not for moving VMs around. I > am > not > > aware of the connection between the two, that you seem to look for. > > > > > > On Fri, Jun 11, 2021 at 11:03 AM Jeremy Hansen > wrote: > > > >> > >> I’m trying to play with HA. I’ve enabled it via the interface but > the > HA > >> state is labeled as Ineligible. > >> > >> I’m specifically interested in this: > >> > >> HA for Hosts > >> > >> The user can specify a virtual machine as HA-enabled. By default, > all > >> virtual router VMs and Elastic Load Balancing VMs are automatically > >> configured as HA-enabled. When an HA-enabled VM crashes, CloudStack > detects > >> the crash and restarts the VM automatically within the same > >> Availability > >> Zone. HA is never performed across different Availability Zones. > CloudStack > >> has a conservative policy towards restarting VMs and ensures that > >> there > >> will never be two instances of the same VM running at the same time. > >> The > >> Management Server attempts to start the VM on another Host in the > same > >> cluster. > >> > >> > >> My assumption is if a VM Host dies, whatever guests that were > running > >> on > >> that host would automatically move to an available VM host. Maybe > I’m > >> misinterpreting. > >> > >> Thanks > >> -jeremy > >> > > > > > > -- > > Daan > > > >>> > >>> -- > >>> Daan > >> > >> > > > > -- > > Daan > > -- Daan
Re: HA State: Ineligible
Just seems strange. If I have centralized storage and I can migrate live vm’s, why wouldn’t I be able to take the risk to migrate a dead vm guest, as in, just start it on another available host and start it and consider it migrated. If the original vm host happens to come back up and the instance has already migrated, just leave it alone. Seems like that would go a long way for high availability. I assume i/o fencing becomes an issue. I’d prefer some kind of hook so if a VM host seems dead beyond, say, 5 minutes, i/o fence it by triggering a power off from the PDU and bring up the VM guests somewhere else. -jeremy > On Jun 18, 2021, at 2:42 AM, Daan Hoogland wrote: > > :D > I'm sorry, when your bike is broken you have to walk. detection if the > exact situation is the issue here, good luck in your search! > > On Fri, Jun 18, 2021 at 11:31 AM Jeremy Hansen wrote: > >> I guess I’m looking for a recovery scenario where the dead vm host is not >> coming back, failed disk, caught on fire, and a reboot isn’t going to help. >> >> Thanks >> >>> On Jun 18, 2021, at 1:41 AM, Daan Hoogland >> wrote: >>> >>> Jemery, >>> If you don't have IPMI then ACS can not know for sure that the VM won't >>> come back. If it comes back the VM would be running twice and this must >> be >>> prevented at all costs. Maybe I am missing some functionality, and >> someone >>> else can give additional options. >>> On Fri, Jun 18, 2021 at 10:21 AM Jeremy Hansen >> wrote: I pasted that from the documentation. My end goal is if a VM host dies completely and I’m not available to fix it, I would like the VM guests >> that were running on the failed host to automatically migrate to an >> available VM host so the guest instances continue to run. Perhaps that’s not how it works. The hosts I’m using for testing do not have any kind of IPMI supported out of band management. They do have network enabled PDUs but let’s just say the VM host is gone completely. How do I get the VM >> guests that were running on the failed host back up and running without my intervention? I guess I wrongly assumed Cloudstack would handle this >> case by just starting the VMs on another available host machine after some >> kind of failed heartbeat threshold. Thanks -jeremy > On Jun 18, 2021, at 1:09 AM, Daan Hoogland wrote: > > Jeremy, > I don't fully understand your question. You say you are interested in > HostHA specifically but then you ask about restarting VMs when a host dies. > This would not be safe as we can't be sure a host really dies unless >> you > have HostHA enabled. Consequently you can't guarantee the VM won't suddenly > re-apear when the host is seen running again. So keep these things > separated. > HostHA is for rebooting suspect hosts, not for moving VMs around. I am not > aware of the connection between the two, that you seem to look for. > > > On Fri, Jun 11, 2021 at 11:03 AM Jeremy Hansen wrote: > >> >> I’m trying to play with HA. I’ve enabled it via the interface but the HA >> state is labeled as Ineligible. >> >> I’m specifically interested in this: >> >> HA for Hosts >> >> The user can specify a virtual machine as HA-enabled. By default, all >> virtual router VMs and Elastic Load Balancing VMs are automatically >> configured as HA-enabled. When an HA-enabled VM crashes, CloudStack detects >> the crash and restarts the VM automatically within the same >> Availability >> Zone. HA is never performed across different Availability Zones. CloudStack >> has a conservative policy towards restarting VMs and ensures that >> there >> will never be two instances of the same VM running at the same time. >> The >> Management Server attempts to start the VM on another Host in the same >> cluster. >> >> >> My assumption is if a VM Host dies, whatever guests that were running >> on >> that host would automatically move to an available VM host. Maybe I’m >> misinterpreting. >> >> Thanks >> -jeremy >> > > > -- > Daan >>> >>> -- >>> Daan >> >> > > -- > Daan signature.asc Description: Message signed with OpenPGP
Re: HA State: Ineligible
:D I'm sorry, when your bike is broken you have to walk. detection if the exact situation is the issue here, good luck in your search! On Fri, Jun 18, 2021 at 11:31 AM Jeremy Hansen wrote: > I guess I’m looking for a recovery scenario where the dead vm host is not > coming back, failed disk, caught on fire, and a reboot isn’t going to help. > > Thanks > > > On Jun 18, 2021, at 1:41 AM, Daan Hoogland > wrote: > > > > Jemery, > > If you don't have IPMI then ACS can not know for sure that the VM won't > > come back. If it comes back the VM would be running twice and this must > be > > prevented at all costs. Maybe I am missing some functionality, and > someone > > else can give additional options. > > > >> On Fri, Jun 18, 2021 at 10:21 AM Jeremy Hansen > wrote: > >> > >> I pasted that from the documentation. My end goal is if a VM host dies > >> completely and I’m not available to fix it, I would like the VM guests > that > >> were running on the failed host to automatically migrate to an > available VM > >> host so the guest instances continue to run. Perhaps that’s not how it > >> works. The hosts I’m using for testing do not have any kind of IPMI > >> supported out of band management. They do have network enabled PDUs but > >> let’s just say the VM host is gone completely. How do I get the VM > guests > >> that were running on the failed host back up and running without my > >> intervention? I guess I wrongly assumed Cloudstack would handle this > case > >> by just starting the VMs on another available host machine after some > kind > >> of failed heartbeat threshold. > >> > >> Thanks > >> -jeremy > >> > >>> On Jun 18, 2021, at 1:09 AM, Daan Hoogland > >> wrote: > >>> > >>> Jeremy, > >>> I don't fully understand your question. You say you are interested in > >>> HostHA specifically but then you ask about restarting VMs when a host > >> dies. > >>> This would not be safe as we can't be sure a host really dies unless > you > >>> have HostHA enabled. Consequently you can't guarantee the VM won't > >> suddenly > >>> re-apear when the host is seen running again. So keep these things > >>> separated. > >>> HostHA is for rebooting suspect hosts, not for moving VMs around. I am > >> not > >>> aware of the connection between the two, that you seem to look for. > >>> > >>> > >>> On Fri, Jun 11, 2021 at 11:03 AM Jeremy Hansen > >> wrote: > >>> > > I’m trying to play with HA. I’ve enabled it via the interface but the > >> HA > state is labeled as Ineligible. > > I’m specifically interested in this: > > HA for Hosts > > The user can specify a virtual machine as HA-enabled. By default, all > virtual router VMs and Elastic Load Balancing VMs are automatically > configured as HA-enabled. When an HA-enabled VM crashes, CloudStack > >> detects > the crash and restarts the VM automatically within the same > Availability > Zone. HA is never performed across different Availability Zones. > >> CloudStack > has a conservative policy towards restarting VMs and ensures that > there > will never be two instances of the same VM running at the same time. > The > Management Server attempts to start the VM on another Host in the same > cluster. > > > My assumption is if a VM Host dies, whatever guests that were running > on > that host would automatically move to an available VM host. Maybe I’m > misinterpreting. > > Thanks > -jeremy > > >>> > >>> > >>> -- > >>> Daan > >> > >> > > > > -- > > Daan > > -- Daan
Re: HA State: Ineligible
I guess I’m looking for a recovery scenario where the dead vm host is not coming back, failed disk, caught on fire, and a reboot isn’t going to help. Thanks > On Jun 18, 2021, at 1:41 AM, Daan Hoogland wrote: > > Jemery, > If you don't have IPMI then ACS can not know for sure that the VM won't > come back. If it comes back the VM would be running twice and this must be > prevented at all costs. Maybe I am missing some functionality, and someone > else can give additional options. > >> On Fri, Jun 18, 2021 at 10:21 AM Jeremy Hansen wrote: >> >> I pasted that from the documentation. My end goal is if a VM host dies >> completely and I’m not available to fix it, I would like the VM guests that >> were running on the failed host to automatically migrate to an available VM >> host so the guest instances continue to run. Perhaps that’s not how it >> works. The hosts I’m using for testing do not have any kind of IPMI >> supported out of band management. They do have network enabled PDUs but >> let’s just say the VM host is gone completely. How do I get the VM guests >> that were running on the failed host back up and running without my >> intervention? I guess I wrongly assumed Cloudstack would handle this case >> by just starting the VMs on another available host machine after some kind >> of failed heartbeat threshold. >> >> Thanks >> -jeremy >> >>> On Jun 18, 2021, at 1:09 AM, Daan Hoogland >> wrote: >>> >>> Jeremy, >>> I don't fully understand your question. You say you are interested in >>> HostHA specifically but then you ask about restarting VMs when a host >> dies. >>> This would not be safe as we can't be sure a host really dies unless you >>> have HostHA enabled. Consequently you can't guarantee the VM won't >> suddenly >>> re-apear when the host is seen running again. So keep these things >>> separated. >>> HostHA is for rebooting suspect hosts, not for moving VMs around. I am >> not >>> aware of the connection between the two, that you seem to look for. >>> >>> >>> On Fri, Jun 11, 2021 at 11:03 AM Jeremy Hansen >> wrote: >>> I’m trying to play with HA. I’ve enabled it via the interface but the >> HA state is labeled as Ineligible. I’m specifically interested in this: HA for Hosts The user can specify a virtual machine as HA-enabled. By default, all virtual router VMs and Elastic Load Balancing VMs are automatically configured as HA-enabled. When an HA-enabled VM crashes, CloudStack >> detects the crash and restarts the VM automatically within the same Availability Zone. HA is never performed across different Availability Zones. >> CloudStack has a conservative policy towards restarting VMs and ensures that there will never be two instances of the same VM running at the same time. The Management Server attempts to start the VM on another Host in the same cluster. My assumption is if a VM Host dies, whatever guests that were running on that host would automatically move to an available VM host. Maybe I’m misinterpreting. Thanks -jeremy >>> >>> >>> -- >>> Daan >> >> > > -- > Daan
Re: HA State: Ineligible
Jemery, If you don't have IPMI then ACS can not know for sure that the VM won't come back. If it comes back the VM would be running twice and this must be prevented at all costs. Maybe I am missing some functionality, and someone else can give additional options. On Fri, Jun 18, 2021 at 10:21 AM Jeremy Hansen wrote: > I pasted that from the documentation. My end goal is if a VM host dies > completely and I’m not available to fix it, I would like the VM guests that > were running on the failed host to automatically migrate to an available VM > host so the guest instances continue to run. Perhaps that’s not how it > works. The hosts I’m using for testing do not have any kind of IPMI > supported out of band management. They do have network enabled PDUs but > let’s just say the VM host is gone completely. How do I get the VM guests > that were running on the failed host back up and running without my > intervention? I guess I wrongly assumed Cloudstack would handle this case > by just starting the VMs on another available host machine after some kind > of failed heartbeat threshold. > > Thanks > -jeremy > > > On Jun 18, 2021, at 1:09 AM, Daan Hoogland > wrote: > > > > Jeremy, > > I don't fully understand your question. You say you are interested in > > HostHA specifically but then you ask about restarting VMs when a host > dies. > > This would not be safe as we can't be sure a host really dies unless you > > have HostHA enabled. Consequently you can't guarantee the VM won't > suddenly > > re-apear when the host is seen running again. So keep these things > > separated. > > HostHA is for rebooting suspect hosts, not for moving VMs around. I am > not > > aware of the connection between the two, that you seem to look for. > > > > > > On Fri, Jun 11, 2021 at 11:03 AM Jeremy Hansen > wrote: > > > >> > >> I’m trying to play with HA. I’ve enabled it via the interface but the > HA > >> state is labeled as Ineligible. > >> > >> I’m specifically interested in this: > >> > >> HA for Hosts > >> > >> The user can specify a virtual machine as HA-enabled. By default, all > >> virtual router VMs and Elastic Load Balancing VMs are automatically > >> configured as HA-enabled. When an HA-enabled VM crashes, CloudStack > detects > >> the crash and restarts the VM automatically within the same Availability > >> Zone. HA is never performed across different Availability Zones. > CloudStack > >> has a conservative policy towards restarting VMs and ensures that there > >> will never be two instances of the same VM running at the same time. The > >> Management Server attempts to start the VM on another Host in the same > >> cluster. > >> > >> > >> My assumption is if a VM Host dies, whatever guests that were running on > >> that host would automatically move to an available VM host. Maybe I’m > >> misinterpreting. > >> > >> Thanks > >> -jeremy > >> > > > > > > -- > > Daan > > -- Daan
Re: HA State: Ineligible
I pasted that from the documentation. My end goal is if a VM host dies completely and I’m not available to fix it, I would like the VM guests that were running on the failed host to automatically migrate to an available VM host so the guest instances continue to run. Perhaps that’s not how it works. The hosts I’m using for testing do not have any kind of IPMI supported out of band management. They do have network enabled PDUs but let’s just say the VM host is gone completely. How do I get the VM guests that were running on the failed host back up and running without my intervention? I guess I wrongly assumed Cloudstack would handle this case by just starting the VMs on another available host machine after some kind of failed heartbeat threshold. Thanks -jeremy > On Jun 18, 2021, at 1:09 AM, Daan Hoogland wrote: > > Jeremy, > I don't fully understand your question. You say you are interested in > HostHA specifically but then you ask about restarting VMs when a host dies. > This would not be safe as we can't be sure a host really dies unless you > have HostHA enabled. Consequently you can't guarantee the VM won't suddenly > re-apear when the host is seen running again. So keep these things > separated. > HostHA is for rebooting suspect hosts, not for moving VMs around. I am not > aware of the connection between the two, that you seem to look for. > > > On Fri, Jun 11, 2021 at 11:03 AM Jeremy Hansen wrote: > >> >> I’m trying to play with HA. I’ve enabled it via the interface but the HA >> state is labeled as Ineligible. >> >> I’m specifically interested in this: >> >> HA for Hosts >> >> The user can specify a virtual machine as HA-enabled. By default, all >> virtual router VMs and Elastic Load Balancing VMs are automatically >> configured as HA-enabled. When an HA-enabled VM crashes, CloudStack detects >> the crash and restarts the VM automatically within the same Availability >> Zone. HA is never performed across different Availability Zones. CloudStack >> has a conservative policy towards restarting VMs and ensures that there >> will never be two instances of the same VM running at the same time. The >> Management Server attempts to start the VM on another Host in the same >> cluster. >> >> >> My assumption is if a VM Host dies, whatever guests that were running on >> that host would automatically move to an available VM host. Maybe I’m >> misinterpreting. >> >> Thanks >> -jeremy >> > > > -- > Daan signature.asc Description: Message signed with OpenPGP
Re: HA State: Ineligible
Jeremy, I don't fully understand your question. You say you are interested in HostHA specifically but then you ask about restarting VMs when a host dies. This would not be safe as we can't be sure a host really dies unless you have HostHA enabled. Consequently you can't guarantee the VM won't suddenly re-apear when the host is seen running again. So keep these things separated. HostHA is for rebooting suspect hosts, not for moving VMs around. I am not aware of the connection between the two, that you seem to look for. On Fri, Jun 11, 2021 at 11:03 AM Jeremy Hansen wrote: > > I’m trying to play with HA. I’ve enabled it via the interface but the HA > state is labeled as Ineligible. > > I’m specifically interested in this: > > HA for Hosts > > The user can specify a virtual machine as HA-enabled. By default, all > virtual router VMs and Elastic Load Balancing VMs are automatically > configured as HA-enabled. When an HA-enabled VM crashes, CloudStack detects > the crash and restarts the VM automatically within the same Availability > Zone. HA is never performed across different Availability Zones. CloudStack > has a conservative policy towards restarting VMs and ensures that there > will never be two instances of the same VM running at the same time. The > Management Server attempts to start the VM on another Host in the same > cluster. > > > My assumption is if a VM Host dies, whatever guests that were running on > that host would automatically move to an available VM host. Maybe I’m > misinterpreting. > > Thanks > -jeremy > -- Daan
HA State: Ineligible
I’m trying to play with HA. I’ve enabled it via the interface but the HA state is labeled as Ineligible. I’m specifically interested in this: HA for Hosts The user can specify a virtual machine as HA-enabled. By default, all virtual router VMs and Elastic Load Balancing VMs are automatically configured as HA-enabled. When an HA-enabled VM crashes, CloudStack detects the crash and restarts the VM automatically within the same Availability Zone. HA is never performed across different Availability Zones. CloudStack has a conservative policy towards restarting VMs and ensures that there will never be two instances of the same VM running at the same time. The Management Server attempts to start the VM on another Host in the same cluster. My assumption is if a VM Host dies, whatever guests that were running on that host would automatically move to an available VM host. Maybe I’m misinterpreting. Thanks -jeremy signature.asc Description: Message signed with OpenPGP
