Public bug reported:
Firstly, transparent hugepages can be enabled with:
# echo always > /sys/kernel/mm/transparent_hugepage/enabled
they are nice and easy to get working, but do not survive memory
pressure on the host. I/O pressure on the host breaks the 2M pages back
to 4k pages, and therefore renders transparent hugepages useless for
long-lived processes, as nothing will reassemble the 4k pages back to 2M
pages after memory pressure has ceased. So, kvm should be set up using
explicit hugetlbfs support for now.
Firstly, kvm is very irritating in that it tends to silently use
ordinarily backed memory if hugepage support fails to work, so for a
while I thought things were fine despite I had just wasted >8 GB of
memory on unused hugepages. This is, however, something to fix within
kvm itself rather, and it may be that this is how kvm maintainers prefer
it to work. I personally disagree, but it is possible to cat
/proc/meminfo and observe if kvm is using hugepages or not:
HugePages_Free value equals HugePages_Total when nobody is using
hugepages.
I chose to mount hugetlbfs at /hugepages, which makes the directory for
kvm's memory pages /hugepages/libvirt/qemu. libvirt-bin had to be
restarted for it to recognize the hugetlbfs mount location. I need total
of 4400 hugetlb pages for my VMs, so I wrote a
/etc/sysctl.d/60-custom.conf which says "vm.nr_hugepages=4400" and set
that value using sysctl -w. I had to retry that command a bunch of
times, each time getting more hugepages than before, until finally I got
all 4400.
However, apparmor still denies access to the hugepage mount point by
default. I added a line such as "/hugepages/libvirt/qemu/* rw," to every
virtual machine's apparmor profie, which are generated from
/etc/apparmor.d/libvirt/TEMPLATE by libvirt-aa-helper. I did not
discover a way to regenerate the files from the TEMPLATE, so I just
manually modified every one after fixing the TEMPLATE file.
I would suggest that libvirt-aa-helper should detect hugepages usage
from the domain configuration file
(<memoryBacking><hugepages/></memoryBacking>) and then find out where
hugepages are mounted and add that rw grant to that location.
$ lsb_release -rd
Description: Ubuntu 12.04 LTS
Release: 12.04
$ apt-cache policy libvirt-bin
libvirt-bin:
Installed: 0.9.8-2ubuntu17.1
Candidate: 0.9.8-2ubuntu17.1
Version table:
*** 0.9.8-2ubuntu17.1 0
500 http://fi.archive.ubuntu.com/ubuntu/ precise-updates/main amd64
Packages
100 /var/lib/dpkg/status
0.9.8-2ubuntu17 0
500 http://fi.archive.ubuntu.com/ubuntu/ precise/main amd64 Packages
** Affects: libvirt (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1001584
Title:
libvirt-bin support for hugepages: apparmor needs a rule for hugetlbfs
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1001584/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
