This is an invalid bug report.
But it might still be interesting because the problem seems to be some
change after the upgrade which is reserving some of the exactly 16GiB of
hugepages I reserved via sysctl.conf
I did not see this in 22.04.
So, I think the problem was the amount of memory I was requesting exceeded
free pages due to something else reserving some of the hugepage pool.
Firstly ...
#Topic 1: Unnecessary entry in fstab and /det/libvirt/qemu.conf edit are
now reverted
#Topic 2: there is no indication of apparmor problems, I was just
eliminating it.
guest config which fails to start: "FailConfig":
<memory unit="KiB">16777216</memory>
<currentMemory unit="KiB">16777216</currentMemory>
<memoryBacking>
<hugepages>
<page size="2048" unit="KiB"/>
</hugepages>
<source type="memfd"/>
<access mode="shared"/>
</memoryBacking>
guest config which worked "SucceedConfig":
<memory unit="KiB">8388608</memory>
<currentMemory unit="KiB">8388608</currentMemory>
<memoryBacking>
<hugepages>
<page size="2048" unit="KiB"/>
</hugepages>
<source type="memfd"/>
<access mode="shared"/>
</memoryBacking>
tim@black:~$ hugeadm --list-all-mounts
Mount Point Options
/dev/hugepages rw,relatime,gid=1002,mode=1770,pagesize=2M
Prior to starting the VM:
tim@black:~$ grep -i huge /proc/meminfo
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
FileHugePages: 0 kB
HugePages_Total: 8192
HugePages_Free: 8180
HugePages_Rsvd: 61
HugePages_Surp: 0
Hugepagesize: 2048 kB
Hugetlb: 16777216 kB
(I do not remember noticing the reservation of 61 pages in 22.04)
After starting with SucceedConfig:
tim@black:~$ grep -i huge /proc/meminfo
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
FileHugePages: 0 kB
HugePages_Total: 8192
HugePages_Free: 4084
HugePages_Rsvd: 61
HugePages_Surp: 0
Hugepagesize: 2048 kB
Hugetlb: 16777216 kB
So, back to FailConfig, but first increase number of pages
try:
tim@black:~$ echo 8500 | sudo tee -a
/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
8500
tim@black:~$ grep -i huge /proc/meminfo
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
FileHugePages: 0 kB
HugePages_Total: 8500
HugePages_Free: 8488
HugePages_Rsvd: 61
HugePages_Surp: 0
Hugepagesize: 2048 kB
Hugetlb: 17408000 kB
and start with "FailConfig"
Result: Boot is successful!
tim@black:~$ grep -i huge /proc/meminfo
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
FileHugePages: 0 kB
HugePages_Total: 8500
HugePages_Free: 296
HugePages_Rsvd: 61
HugePages_Surp: 0
Hugepagesize: 2048 kB
Hugetlb: 17408000 kB
Thank you for helping me. I am new to hugepages and without your excellent
reply I would not have had confidence to keep trying to fix the problem.
Hopefully this can serve as some documentation for other beginners.
On Tue, 16 Jul 2024 at 18:35, Christian Ehrhardt <
2073...@bugs.launchpad.net> wrote:
> Hi Tim,
> you are right that neither of the workaround steps were necessary on 22.04
> and they should neither be on 24.04.
>
>
> ### Topic 1 - changes that should not be needed
>
> Ubuntu 24.04 by default has
>
> 1. hugetlbfs mounted (was that no more the case for you after upgrade?)
> $ mount | grep huge
> hugetlbfs on /dev/hugepages type hugetlbfs
> (rw,nosuid,nodev,relatime,pagesize=2M)
>
> 2. uncommenting hugetlbfs_mount in /etc/libvirt/qemu.conf changes nothing
> default is
> #hugetlbfs_mount = "/dev/hugepages"
> Changing that to
> hugetlbfs_mount = "/dev/hugepages"
> will not change anything.
>
> I assume #1 and #2 have only been things on your try to debug what is
> wrong - which is fine.
> But let us know if it is otherwise.
>
> ### Topic 2 - was there an apparmor denial?
>
> Now back to your actual issue, you mentioned that you set apparmor to
> complain mode.
> Was there an indication that apparmor is the issue, a related denial in
> the syslog when you start your guest? If so could you please pass that
> denial?
>
> ### Topic 3 - repro seems to not trigger this
>
> I picked a guest which before had:
> <memory unit='MiB'>1024</memory>
>
> <currentMemory unit='MiB'>1024</currentMemory>
>
> And changed it to use huge pages by adding:
> <memoryBacking>
>
> <hugepages>
>
> <page size='2048' unit='KiB'/>
>
> </hugepages>
>
> </memoryBacking>
>
> To confirm it tries to use the right thing I started it without allocating
> huge pages and got:
> => qemu-system-x86_64: unable to map backing store for guest RAM: Cannot
> allocate memory
>
> Then I allocated some huge pages in the system and started it again.
> $ echo 512 | sudo tee -a
> /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
> $ virsh start --console test
>
> Worked right away.
>
> I double checked in /proc/meminfo and saw HugePages_Free changing when the
> guest was running.
> So I think, something is configured in an unexpected way.
>
> Could you share
> - your full guest-config in regard to memory
> - hugeadm --list-all-mounts
> - journalctl -f while starting the guest
> - grep -i huge /proc/meminfo
>
> ** Changed in: libvirt (Ubuntu)
> Status: New => Incomplete
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/2073214
>
> Title:
> hugepages causes permissions error
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/2073214/+subscriptions
>
>
--
Tim Richardson
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2073214
Title:
hugepages causes permissions error
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/2073214/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
