Re: VM Randomly Freezing

2023-10-25 Thread Ricardo Pertuz 
Check your primary storage, it happened to me and it was at 100%


Atte,

Ricardo Pertuz


On 25 Oct 2023 at 1:22 AM -0500, Granwille Strauss 
, wrote:
> Hi Guys
> Any help will be appreciated, my VMs on my KVM hyp (Almalinux 8) randomly 
> freezes. It does not happen with all of them at once but at least once a week 
> one VM randomly freezes. It does not respond to console nor can I check 
> /var/log/libvirt/qemu/i-2-118-VM.log in the KVM there is nothing. I also 
> check cloudstack and agent logs and I am not seeing anything obvious. I then 
> investigate the VM too. There too I see nothing journalctl only has the 
> latest boot information once the VM is forced reboot via Cloudstack UI, which 
> is the only way I can bring it up again.
>
> What's also weird, I have Ha enabled for all my VMs it works when I for 
> example shutdown the VM inside the VM then Ha bring it up again. But in this 
> case, Ha doesn't seem to detect the VM is in a "frozen" state to reboot it.
> If anyone has had a similar experience before, I will appreciate any advice 
> to help troubleshoot this, please.
> --
> Regards / Groete
>
> Granwille Strauss  //  Senior Systems Admin
>
> e: granwi...@namhost.com
> m: +264 81 323 1260
> w: www.namhost.com
>
>
>
>
> Namhost Internet Services (Pty) Ltd,
> 24 Black Eagle Rd, Hermanus, 7210, RSA
>
>
> The content of this message is confidential. If you have received it by 
> mistake, please inform us by email reply and then delete the message. It is 
> forbidden to copy, forward, or in any way reveal the contents of this message 
> to anyone without our explicit consent. The integrity and security of this 
> email cannot be guaranteed over the Internet. Therefore, the sender will not 
> be held liable for any damage caused by the message. For our full privacy 
> policy and disclaimers, please go to https://www.namhost.com/privacy-policy
>

Re: VM Randomly Freezing

2023-10-25 Thread Granwille Strauss 
Thank you for the feedback. The issue is very weird, the KVM host is 
under no stress whatsoever while the VM was down. The monitoring tools 
we have in place also show zero spikes on the VM load, RAM or network 
prior to it "freezing". Virsh in KVM at the time still saw the VM 
instance as running, and the fact that I could not find anything in 
/var/log/libvirt/qemu/i-2-118-VM.log prob means there's no issue with 
libvirt itself. Or was there possible other log locations that I may not 
be aware of?


On 10/25/23 11:04, Boris Stoyanov wrote:

The way HA works in Cloudstack is it reads the hypervisor report, so if your 
kvm hosts says it’s working, Cloudstack wouldn’t do anything, despite being 
frozen.

Cloudstack manages the provisioning of the VMs at the host. How the VM behaves 
is dependent on the hardware and hypervisor (libvirt), unfortunately I cannot 
relate this issue to Cloudstack, especially no errors/exceptions hound in logs. 
Sounds like a libvirt issue that you’re hitting or some hardware related 
problem..

Thanks,
Bobby.

From: Granwille Strauss
Date: Wednesday, 25 October 2023, 9:21
To:users@cloudstack.apache.org  
Subject: VM Randomly Freezing

Hi Guys

Any help will be appreciated, my VMs on my KVM hyp (Almalinux 8) randomly 
freezes. It does not happen with all of them at once but at least once a week 
one VM randomly freezes. It does not respond to console nor can I check 
/var/log/libvirt/qemu/i-2-118-VM.log in the KVM there is nothing. I also check 
cloudstack and agent logs and I am not seeing anything obvious. I then 
investigate the VM too. There too I see nothing journalctl only has the latest 
boot information once the VM is forced reboot via Cloudstack UI, which is the 
only way I can bring it up again.

What's also weird, I have Ha enabled for all my VMs it works when I for example shutdown 
the VM inside the VM then Ha bring it up again. But in this case, Ha doesn't seem to 
detect the VM is in a "frozen" state to reboot it.

If anyone has had a similar experience before, I will appreciate any advice to 
help troubleshoot this, please.
--

Regards / Groete
[Image removed by sender.]<https://www.namhost.com/>
Granwille Strauss  //  Senior Systems Admin

e:granwi...@namhost.com<mailto:granwi...@namhost.com>
m: +264 81 323 1260
w:www.namhost.com<https://www.namhost.com/>

[Image removed by sender.]<https://www.facebook.com/namhost>[Image removed by 
sender.]<https://twitter.com/namhost>[Image removed by 
sender.]<https://www.instagram.com/namhostinternetservices/>[Image removed by 
sender.]<https://www.linkedin.com/company/namhos>[Image removed by 
sender.]<https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>

[Image removed by 
sender.]<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
Namhost Internet Services (Pty) Ltd,

24 Black Eagle Rd, Hermanus, 7210, RSA

The content of this message is confidential. If you have received it by 
mistake, please inform us by email reply and then delete the message. It is 
forbidden to copy, forward, or in any way reveal the contents of this message 
to anyone without our explicit consent. The integrity and security of this 
email cannot be guaranteed over the Internet. Therefore, the sender will not be 
held liable for any damage caused by the message. For our full privacy policy 
and disclaimers, please go tohttps://www.namhost.com/privacy-policy

[Image removed by sender. Powered by 
AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>

  




--
Regards / Groete

<https://www.namhost.com> Granwille Strauss  // Senior Systems Admin

*e:* granwi...@namhost.com
*m:* +264 81 323 1260 
*w:* www.namhost.com <https://www.namhost.com/>

<https://www.facebook.com/namhost><https://twitter.com/namhost><https://www.instagram.com/namhostinternetservices/><https://www.linkedin.com/company/namhos><https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>

<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>

Namhost Internet Services (Pty) Ltd,

24 Black Eagle Rd, Hermanus, 7210, RSA



The content of this message is confidential. If you have received it by 
mistake, please inform us by email reply and then delete the message. It 
is forbidden to copy, forward, or in any way reveal the contents of this 
message to anyone without our explicit consent. The integrity and 
security of this email cannot be guaranteed over the Internet. 
Therefore, the sender will not be held liable for any damage caused by 
the message. For our full privacy policy and disclaimers, please go to 
https://www.namhost.com/privacy-policy


Powered by AdSigner 
<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>

smime.p7s
Description: S/MIME Cryptographic Signature

Re: VM Randomly Freezing

2023-10-25 Thread Boris Stoyanov 
The way HA works in Cloudstack is it reads the hypervisor report, so if your 
kvm hosts says it’s working, Cloudstack wouldn’t do anything, despite being 
frozen.

Cloudstack manages the provisioning of the VMs at the host. How the VM behaves 
is dependent on the hardware and hypervisor (libvirt), unfortunately I cannot 
relate this issue to Cloudstack, especially no errors/exceptions hound in logs. 
Sounds like a libvirt issue that you’re hitting or some hardware related 
problem..

Thanks,
Bobby.

From: Granwille Strauss 
Date: Wednesday, 25 October 2023, 9:21
To: users@cloudstack.apache.org 
Subject: VM Randomly Freezing

Hi Guys

Any help will be appreciated, my VMs on my KVM hyp (Almalinux 8) randomly 
freezes. It does not happen with all of them at once but at least once a week 
one VM randomly freezes. It does not respond to console nor can I check 
/var/log/libvirt/qemu/i-2-118-VM.log in the KVM there is nothing. I also check 
cloudstack and agent logs and I am not seeing anything obvious. I then 
investigate the VM too. There too I see nothing journalctl only has the latest 
boot information once the VM is forced reboot via Cloudstack UI, which is the 
only way I can bring it up again.

What's also weird, I have Ha enabled for all my VMs it works when I for example 
shutdown the VM inside the VM then Ha bring it up again. But in this case, Ha 
doesn't seem to detect the VM is in a "frozen" state to reboot it.

If anyone has had a similar experience before, I will appreciate any advice to 
help troubleshoot this, please.
--

Regards / Groete
[Image removed by sender.]<https://www.namhost.com/>
Granwille Strauss  //  Senior Systems Admin

e: granwi...@namhost.com<mailto:granwi...@namhost.com>
m: +264 81 323 1260
w: www.namhost.com<https://www.namhost.com/>

[Image removed by sender.]<https://www.facebook.com/namhost>[Image removed by 
sender.]<https://twitter.com/namhost>[Image removed by 
sender.]<https://www.instagram.com/namhostinternetservices/>[Image removed by 
sender.]<https://www.linkedin.com/company/namhos>[Image removed by 
sender.]<https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>

[Image removed by 
sender.]<https://www.adsigner.com/v1/l/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818/banner>
Namhost Internet Services (Pty) Ltd,

24 Black Eagle Rd, Hermanus, 7210, RSA

The content of this message is confidential. If you have received it by 
mistake, please inform us by email reply and then delete the message. It is 
forbidden to copy, forward, or in any way reveal the contents of this message 
to anyone without our explicit consent. The integrity and security of this 
email cannot be guaranteed over the Internet. Therefore, the sender will not be 
held liable for any damage caused by the message. For our full privacy policy 
and disclaimers, please go to https://www.namhost.com/privacy-policy

[Image removed by sender. Powered by 
AdSigner]<https://www.adsigner.com/v1/c/631091998d4670001fe43ec2/621c9b76c140bb001ed0f818>

VM Randomly Freezing

2023-10-25 Thread Granwille Strauss 

Hi Guys

Any help will be appreciated, my VMs on my KVM hyp (Almalinux 8) 
randomly freezes. It does not happen with all of them at once but at 
least once a week one VM randomly freezes. It does not respond to 
console nor can I check /var/log/libvirt/qemu/i-2-118-VM.log in the KVM 
there is nothing. I also check cloudstack and agent logs and I am not 
seeing anything obvious. I then investigate the VM too. There too I see 
nothing journalctl only has the latest boot information once the VM is 
forced reboot via Cloudstack UI, which is the only way I can bring it up 
again.


What's also weird, I have Ha enabled for all my VMs it works when I for 
example shutdown the VM inside the VM then Ha bring it up again. But in 
this case, Ha doesn't seem to detect the VM is in a "frozen" state to 
reboot it.


If anyone has had a similar experience before, I will appreciate any 
advice to help troubleshoot this, please.


--
Regards / Groete

 Granwille Strauss  // Senior Systems Admin

*e:* granwi...@namhost.com
*m:* +264 81 323 1260 
*w:* www.namhost.com 





Namhost Internet Services (Pty) Ltd,

24 Black Eagle Rd, Hermanus, 7210, RSA



The content of this message is confidential. If you have received it by 
mistake, please inform us by email reply and then delete the message. It 
is forbidden to copy, forward, or in any way reveal the contents of this 
message to anyone without our explicit consent. The integrity and 
security of this email cannot be guaranteed over the Internet. 
Therefore, the sender will not be held liable for any damage caused by 
the message. For our full privacy policy and disclaimers, please go to 
https://www.namhost.com/privacy-policy


Powered by AdSigner 


smime.p7s
Description: S/MIME Cryptographic Signature