Re: Cluster HA - KVM Hosts

[Jafar Aghabalayev]

Hello,
OOBM also configured using IPMI and when i shut the host, power state show 
correct state and im able to control (power off, power on) host using OOBM on 
cloudstack

Sent from Outlook for Android

From: Simon Weller 
Sent: Friday, February 10, 2023 11:07:15 PM
To: users@cloudstack.apache.org 
Subject: Re: Cluster HA - KVM Hosts

CAUTION: This email originated from outside the organization. Do not click 
links or open attachments unless you recognize the sender and know the content 
is safe. Please report all suspicious emails to s...@pasha-technology.com.

Hey Jafar,

For HA to work on KVM, you need to configure each host with out-of-band
management using ipmitool or Redfish. This is so Cloudstack can fence your
nodes.

This covers the concept -
https://docs.cloudstack.apache.org/en/latest/adminguide/reliability.html?highlight=fencing#ha-enabled-hosts
Out-of-band documentation is here -
https://docs.cloudstack.apache.org/en/latest/adminguide/hosts.html#out-of-band-management
For details on how the feature works, check out the original design
document - https://cwiki.apache.org/confluence/display/CLOUDSTACK/Host+HA

-Si




On Fri, Feb 10, 2023 at 12:32 PM Jafar Aghabalayev <
jafar.aghabala...@pasha-technology.com> wrote:

> Hello Community,
>
> I have configured cluster with 2 KVM Hosts.
> HA enabled for hosts, ha.tag attribute on global configuration is the same
> as for ha.tag attribute for hosts. VMs are running with HA enabled
> offering. NFS used as primary storage.
> I tried to shutdown one of the hosts, however HA not performed. State and
> power state indicated as down, but resource state show as up and VM located
> at the failed host show in running state (in real it is unaccessible). I
> tried to stop libvirtd service and the result same.
>
> Can anyone help me with this issue?
>
>
>

Re: Cluster HA - KVM Hosts

[pspa...@hotmail.com]

Hi Jafar,
FYI.
VM HA does not work when server is stopped gracefully · Issue #4211 · 
apache/cloudstack  
|  
|   
|   
|   ||

   |

  |
|  
|   |  
VM HA does not work when server is stopped gracefully · Issue #4211 · 
apache/cloudstack
 
When we stop a server (hypervisor) gracefully, vms on it will NOT be started on 
other servers even HA is enabled. ISSUE TYPE Bug Report COMPONENT NAME VM HA 
CLOUDSTACK VERSION 4.14/4.15 CONFIGURATI...
  |   |

  |

  |

  

Thanks Pradeep 

Sent from Yahoo Mail on Android 
 
  On Sat, 11 Feb 2023 at 12:02 am, Jafar 
Aghabalayev wrote:   Hello Community,

I have configured cluster with 2 KVM Hosts.
HA enabled for hosts, ha.tag attribute on global configuration is the same as 
for ha.tag attribute for hosts. VMs are running with HA enabled offering. NFS 
used as primary storage.
I tried to shutdown one of the hosts, however HA not performed. State and power 
state indicated as down, but resource state show as up and VM located at the 
failed host show in running state (in real it is unaccessible). I tried to stop 
libvirtd service and the result same.

Can anyone help me with this issue?


  

Re: Cluster HA - KVM Hosts

[Simon Weller]

Hey Jafar,

For HA to work on KVM, you need to configure each host with out-of-band
management using ipmitool or Redfish. This is so Cloudstack can fence your
nodes.

This covers the concept -
https://docs.cloudstack.apache.org/en/latest/adminguide/reliability.html?highlight=fencing#ha-enabled-hosts
Out-of-band documentation is here -
https://docs.cloudstack.apache.org/en/latest/adminguide/hosts.html#out-of-band-management
For details on how the feature works, check out the original design
document - https://cwiki.apache.org/confluence/display/CLOUDSTACK/Host+HA

-Si




On Fri, Feb 10, 2023 at 12:32 PM Jafar Aghabalayev <
jafar.aghabala...@pasha-technology.com> wrote:

> Hello Community,
>
> I have configured cluster with 2 KVM Hosts.
> HA enabled for hosts, ha.tag attribute on global configuration is the same
> as for ha.tag attribute for hosts. VMs are running with HA enabled
> offering. NFS used as primary storage.
> I tried to shutdown one of the hosts, however HA not performed. State and
> power state indicated as down, but resource state show as up and VM located
> at the failed host show in running state (in real it is unaccessible). I
> tried to stop libvirtd service and the result same.
>
> Can anyone help me with this issue?
>
>
>

Cluster HA - KVM Hosts

[Jafar Aghabalayev]

Hello Community,

I have configured cluster with 2 KVM Hosts.
HA enabled for hosts, ha.tag attribute on global configuration is the same as 
for ha.tag attribute for hosts. VMs are running with HA enabled offering. NFS 
used as primary storage.
I tried to shutdown one of the hosts, however HA not performed. State and power 
state indicated as down, but resource state show as up and VM located at the 
failed host show in running state (in real it is unaccessible). I tried to stop 
libvirtd service and the result same.

Can anyone help me with this issue?

Re: noob question about Templates

[Simon Weller]

Yeah, I don't think you need to worry about it. Without the modprobe flag
configured on the KVM host, it won't allow a nested VM to be deployed.

-Si

On Fri, Feb 10, 2023 at 10:51 AM Gary Dixon
 wrote:

>
> Hi Simon
>
> Thanks for the info - its quite interesting !!
>
>
> We are running KVM hypervisor on Ubuntu 20.04 on AMD EPYC cpu's
>
> We haven't been enabling HVM on templates as we don't want users to be
> able to nest hypervisors within their VM's
>
> Gary Dixon​
> Senior Technical Consultant
> T:  +44 161 537 4990
> E:  *v* <+44%207989717661>ms@quadris‑support.com
> W: www.quadris.co.uk
> The information contained in this e-mail from Quadris may be confidential
> and privileged for the private use of the named recipient.  The contents of
> this e-mail may not necessarily represent the official views of Quadris.
> If you have received this information in error you must not copy,
> distribute or take any action or reliance on its contents.  Please destroy
> any hard copies and delete this message.
> -Original Message-
> From: Simon Weller 
> Sent: 10 February 2023 16:23
> To: users@cloudstack.apache.org
> Subject: Re: noob question about Templates
>
> Hey Gary,
>
> Welcome to the list.
>
> Firstly, which hypervisor are you using?
>
> HVM used to mean use hardware accelerated virtualization back in the day
> when it was still fairly new. It was required for some hypervisors, but not
> all.
> These days everything is hardware accelerated.
>
> If you are running KVM, you can enable the ability to run nested
> virilization by doing this -
>
> Intel -
> modprobe -r kvm_intel
> modprobe kvm_intel nested=1
>
> Make it permanent by adding the following line to /etc/modprobe.d/kvm.conf:
> options kvm_intel nested=1
>
> AMD -
> modprobe -r kvm_amd
> modprobe kvm_amd nested=1
>
> Make it permanent by adding the following line to /etc/modprobe.d/kvm.conf:
> options kvm_amd nested=1
>
> Please note that CloudStack will not manage any nested virtualization
> unless Cloudstack itself is nested (e.g. a lab environment).
>
> -Si
>
>
> On Fri, Feb 10, 2023 at 8:28 AM Gary Dixon
> 
> wrote:
>
> > HI
> >
> >
> >
> > Sorry for the newbie question but …..
> >
> >
> >
> > When creating a template – does enabling the HVM option allow the
> > virtual machine to also be a hypervisor so that nested virtualization
> > can be achieved?
> > Gary Dixon​
> > Senior Technical Consultant
> > T: +44 161 537 4990
> > E: *v* <+44%207989717661>ms@quadris‑support.com
> > W:
> > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.q
> > uadris.co.uk%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7C425ae13b53
> > 394cabd18808db0b831363%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C0%7C63
> > 8116429828439052%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV
> > 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2GprAUWMFJ
> > cOjv2PGTIr6gZdnFIDOFAZQd61NGvMGBc%3D&reserved=0
> > The information contained in this e-mail from Quadris may be
> > confidential and privileged for the private use of the named
> > recipient. The contents of this e-mail may not necessarily represent the
> official views of Quadris.
> > If you have received this information in error you must not copy,
> > distribute or take any action or reliance on its contents. Please
> > destroy any hard copies and delete this message.
> >
>

RE: noob question about Templates

[Gary Dixon]

Hi Simon

Thanks for the info - its quite interesting !!


We are running KVM hypervisor on Ubuntu 20.04 on AMD EPYC cpu's

We haven't been enabling HVM on templates as we don't want users to be able to 
nest hypervisors within their VM's


Gary Dixon
Senior Technical Consultant
T:  +44 161 537 4990
E:  v...@quadris-support.com
W: www.quadris.co.uk
The information contained in this e-mail from Quadris may be confidential and 
privileged for the private use of the named recipient.  The contents of this 
e-mail may not necessarily represent the official views of Quadris.  If you 
have received this information in error you must not copy, distribute or take 
any action or reliance on its contents.  Please destroy any hard copies and 
delete this message.
-Original Message-
From: Simon Weller 
Sent: 10 February 2023 16:23
To: users@cloudstack.apache.org
Subject: Re: noob question about Templates

Hey Gary,

Welcome to the list.

Firstly, which hypervisor are you using?

HVM used to mean use hardware accelerated virtualization back in the day when 
it was still fairly new. It was required for some hypervisors, but not all.
These days everything is hardware accelerated.

If you are running KVM, you can enable the ability to run nested virilization 
by doing this -

Intel -
modprobe -r kvm_intel
modprobe kvm_intel nested=1

Make it permanent by adding the following line to /etc/modprobe.d/kvm.conf:
options kvm_intel nested=1

AMD -
modprobe -r kvm_amd
modprobe kvm_amd nested=1

Make it permanent by adding the following line to /etc/modprobe.d/kvm.conf:
options kvm_amd nested=1

Please note that CloudStack will not manage any nested virtualization unless 
Cloudstack itself is nested (e.g. a lab environment).

-Si


On Fri, Feb 10, 2023 at 8:28 AM Gary Dixon 
wrote:

> HI
>
>
>
> Sorry for the newbie question but …..
>
>
>
> When creating a template – does enabling the HVM option allow the
> virtual machine to also be a hypervisor so that nested virtualization
> can be achieved?
> Gary Dixon​
> Senior Technical Consultant
> T:  +44 161 537 4990
> E:  *v* <+44%207989717661>ms@quadris‑support.com
> W:
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.q
> uadris.co.uk%2F&data=05%7C01%7CGary.Dixon%40quadris.co.uk%7C425ae13b53
> 394cabd18808db0b831363%7Cf1d6abf3d3b44894ae16db0fb93a96a2%7C0%7C0%7C63
> 8116429828439052%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV
> 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2GprAUWMFJ
> cOjv2PGTIr6gZdnFIDOFAZQd61NGvMGBc%3D&reserved=0
> The information contained in this e-mail from Quadris may be
> confidential and privileged for the private use of the named
> recipient.  The contents of this e-mail may not necessarily represent the 
> official views of Quadris.
> If you have received this information in error you must not copy,
> distribute or take any action or reliance on its contents.  Please
> destroy any hard copies and delete this message.
>

Re: noob question about Templates

[Simon Weller]

Hey Gary,

Welcome to the list.

Firstly, which hypervisor are you using?

HVM used to mean use hardware accelerated virtualization back in the day
when it was still fairly new. It was required for some hypervisors, but not
all.
These days everything is hardware accelerated.

If you are running KVM, you can enable the ability to run nested
virilization by doing this -

Intel -
modprobe -r kvm_intel
modprobe kvm_intel nested=1

Make it permanent by adding the following line to /etc/modprobe.d/kvm.conf:
options kvm_intel nested=1

AMD -
modprobe -r kvm_amd
modprobe kvm_amd nested=1

Make it permanent by adding the following line to /etc/modprobe.d/kvm.conf:
options kvm_amd nested=1

Please note that CloudStack will not manage any nested virtualization
unless Cloudstack itself is nested (e.g. a lab environment).

-Si


On Fri, Feb 10, 2023 at 8:28 AM Gary Dixon 
wrote:

> HI
>
>
>
> Sorry for the newbie question but …..
>
>
>
> When creating a template – does enabling the HVM option allow the virtual
> machine to also be a hypervisor so that nested virtualization can be
> achieved?
> Gary Dixon​
> Senior Technical Consultant
> T:  +44 161 537 4990
> E:  *v* <+44%207989717661>ms@quadris‑support.com
> W: www.quadris.co.uk
> The information contained in this e-mail from Quadris may be confidential
> and privileged for the private use of the named recipient.  The contents of
> this e-mail may not necessarily represent the official views of Quadris.
> If you have received this information in error you must not copy,
> distribute or take any action or reliance on its contents.  Please destroy
> any hard copies and delete this message.
>

System monitoring

[S.Fuller]

Obviously a bit of a broad topic, but I'm curious what metrics people are
using to monitor the health and performance of their Cloudstack
installations? Disk space / total CPU load / load per core are the obvious
ones. What are any of you doing from a network standpoint? Do you dig into
traffic on the bridged interfaces in any detail? Is there a way to tie that
back to specific instances without having monitoring agents on the
instances themselves?

-- 
Steve Fuller
steveful...@gmail.com

noob question about Templates

[Gary Dixon]

HI

Sorry for the newbie question but .

When creating a template - does enabling the HVM option allow the virtual 
machine to also be a hypervisor so that nested virtualization can be achieved?

Gary Dixon
Senior Technical Consultant
T:  +44 161 537 4990
E:  v...@quadris-support.com
W: www.quadris.co.uk
The information contained in this e-mail from Quadris may be confidential and 
privileged for the private use of the named recipient.  The contents of this 
e-mail may not necessarily represent the official views of Quadris.  If you 
have received this information in error you must not copy, distribute or take 
any action or reliance on its contents.  Please destroy any hard copies and 
delete this message.