Is the QEMU process (after startup) actually running as the QEMU userid ?
/*
* Michael R. Hines
* Platform Engineer, DigitalOcean.
*/
On 02/19/2016 02:43 PM, Roy Shterman wrote:
First off all thank you for your answer,
I couldn't figured how to start virtual machine with increased MEMLOCK,
tried to add into /etc/security/limits.d
qemu soft memlock 3221225
qemu hard memlock 3221225
so max locked-in-memory will be 3G, but it didn't worked.
still has MEMLOCK of 60kb per each VM.
Maybe you can spot what I'm doing wrong?
On Tue, Feb 9, 2016 at 5:16 PM, Michael R. Hines <mich...@hinespot.com
<mailto:mich...@hinespot.com>> wrote:
Hi Roy,
On 02/09/2016 03:57 AM, Roy Shterman wrote:
Hi,
I tried to understand the rdma-migration in qemu code and i
have two questions about it:
1. I'm working with qemu-kvm using libvirt and i'm getting
MEMLOCK max locked-in-memory address space 65536 65536 bytes
in qemu process so I don't understand how can you use
rdma-pin-all with such low MEMLOCK.
I found a solution in libvirt to lock all vm memory in advance
and to enlarge MEMLOCK.
It uses memoryBacking locking and memory tuning hard_limit of
vm memory but I couldn't find a usage of this in
rdma-migration code.
You're absolutey right, the RDMA migration code itself doesn't set
this lock limit explicitly because there are system-wide
restrictions in both appArmour,
/etc/security, as well as SELINUX that restrict applications from
arbitrarily setting their maximum memory lock limits.
The other problem is CGROUPS: If someone sets a cgroup control for
maximum memory and forgets about that mlock() limits, then
there will be a conflict.
So, libvirt must have a policy to deal with all of these
possibilities, not just handle a special case for RDMA migration.
The only way "simple" way (without patching the problems above) to
apply a higher lock limit to QEMU is to set the ulimit for libvirt
(or for QEMU if starting QEMU manually) in your environment or the
command line with $ ulimit # before attempting the migration,
then the RDMA subsystem will be able to lock the memory successfully.
The other option is to use /etc/security/limits.conf and set the
option for a specific libvirt process user and make sure your
libvirt/qemu
are not running as root.
QEMU itself also has a "mlock" option built into the command line,
but it also suffers from the same problem --- you have to find
a way (currently) to increase the limit before using the option.
2. Do you have any comparison of IOPS and bandwidth between
TCP migration and rdma migration?
Yes, lots of comparisons.
http://wiki.qemu.org/Features/RDMALiveMigration
http://www.canturkisci.com/ETC/papers/IBMJRD2011/preprint.pdf
Regards,
Roy
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
