GitHub user prashanthr2 added a comment to the discussion: application vm are crashing in a cluster with mixed CPU model hardware
@gvaishya Thanks for the report. This does not looks like a bug, ideally should be a discussions thread. A few things to separate out. - host-passthrough is unsafe for this cluster. Your hosts mix Skylake-SP (6148, 6154) with Cascade Lake-SP (6254). host-passthrough passes the exact host CPU to the guest, so a VM's visible CPUID changes depending on which host it starts on. Pin to the lowest common denominator, identically on all hosts: > guest.cpu.mode=custom > guest.cpu.model=Skylake-Server **NOTE**: Before changing, confirm the exact model string with virsh domcapabilities on a 6148 host (you may need -IBRS or -noTSX-IBRS). Needs an agent restart, and existing VMs need a stop/start (not a guest reboot). - guest.cpu.features=vmx , is this intentional? This emits <feature policy='require' name='vmx'/>, exposing nested virtualisation to every guest on that host. AFAIK for VDI there's usually no reason for this. This is the first thing I'd remove and check - Also, Check the 1 CPU x 2.4 GHz offering for CPU cap: `SELECT id, name, cpu, speed, limit_cpu_use FROM cloud.service_offering WHERE uuid = <offering_id_from_UI>;` If limit_cpu_use = 1, a CFS <quota> is applied (virsh dumpxml <vm> | grep -A4 cputune). 2.4 GHz is ~77% of a core on a 3.1 GHz host but ~100% on a 2.4 GHz host, so the same offering throttles differently by placement. For VDI that presents as hangs/timeouts users report as crashes. To identify the actual cause, for one crash with a timestamp ( **collect/check below if the above suggested doesn't resolve the issue)** **1. Do crashes correlate with one host? If so this is a faulty host, not a mixed-CPU issue** --> This is the first thing I will check to isolate the issue further 2. /var/log/libvirt/qemu/i-<acct>-<vmid>-VM.log , a QEMU abort or reason=crashed points at guest/QEMU; nothing in the log means the host killed the process 3. Host dmesg -T / journalctl -k . OOM killer reaping qemu-kvm, or MCE entries 4. Guest-side: Windows bugcheck code, or Linux kernel panic Could you paste grep -v '^#' /etc/cloudstack/agent/agent.properties | grep -v '^$' from two hosts (one 6254, one 6148)? That confirms the setting is actually applied and consistent. **Summary**: fix the CPU mode regardless, but with no migration occurring I'd rank vmx and CPU-cap throttling as the likelier causes. The libvirt/QEMU log plus guest bugcheck will help isolate it further. GitHub link: https://github.com/apache/cloudstack/discussions/13992#discussioncomment-18176674 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
