Public bug reported:
Real human writing:
#### Problem:
Booting Xen hypervisor with Linux Dom0.
GRUB loads, Xen kernel messages load up to "Using 6 CPU"
# From this point:
Next screen goes black and stays blank for 10+minutes.
Have to force reboot and choose non XEN kernel booting option via GRUB.
# From this point expected behaviour without bug:
Next screen goes black and stays so for 30-70sec and Dom0 kernel messages
appear,
Next login is possible and system is running, "xl info" and other xl commands
function.
#### Cause:
I spent 2 days rebooting to narrow this down..
This happens if we set both settings same time:
NVME io polling and XEN kernel CPU limits, via default grub configs:
GRUB_CMDLINE_LINUX_DEFAULT="nvme.poll_queues=24"
# Can also be set via /mnt/oldroot/etc/modprobe.d/nvme.conf
# For proper polling setup nvme.io_poll=1 should also be set,
# but symptom already appears if we set only nvme.poll_queues=24
and
GRUB_CMDLINE_XEN_DEFAULT="dom0_max_vcpus=6 dom0_vcpus_pin"
#### Affected software:
24.04.4 LTS (Noble Numbat)
Kernel: 6.8.0-136-generic x86_64
And HWE: 7.0.0-38-generic x86_64
xen-hypervisor-4.17-amd64 4.17.3+10-g091466ba55-1.1ubuntu3
## updating was attempted, but same symptoms still:
26.04.1 LTS (Resolute Raccoon)
Kernel: 7.0.0-38-generic x86_64
xen-hypervisor-4.20-amd64 4.20.2+7-g1badcf5035-2build2
#### Hardware:
Supermicro MB,
AMD EPYC Zen5 CPU with 48threads,
(horrendeus 2+min boot time to GRUB, server MB with BMC/IPMI)
BIOS (not sure how relevant):
ACPI SRAT L3 Cache As NUMA Domain - Enabled
IOMMU - enabled
DMA Protection - enabled
DMAr Support - disabled (if enabled we get +50sec to boot time)
DRTM Virtual Device Support - enabeld
DRTM Memory Reservation - enabeld
SEV-SNP Support - enabled
ASPM Support - enabled for 24.04 testing later set it to disabled for 26.04
testing.
Gen5 NVME disks connected via bifurcation cards:
5pieces: Crucial T700 / PN: CT2000T700SSD5
4pieces: FireCuda 540 / PN: 3JT301-300
1piece: Lexar NM1090 / PN: LNM1090002T-RNANG
Sadly not my personal system, so I can not do further heavy testing on it.
io polling does give very good performance boost to Gen5 NVME, and even Gen4.
Sad we cant use it, as CPU pinning is more important for XEN hypervizor system.
** Affects: linux (Ubuntu)
Importance: Undecided
Status: New
** Tags: kernel-bug nvme xen
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2170270
Title:
Xen Dom0 fails to boot with "nvme.poll_queues=24" and
"dom0_max_vcpus=6 dom0_vcpus_pin"
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2170270/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
