Hello Daniel, thanks for the feedback ! On Tue, Aug 31, 2021 at 10:17 AM Daniel P. Berrangé <berra...@redhat.com> wrote: > > On Tue, Aug 31, 2021 at 08:02:39AM -0300, Leonardo Bras wrote: > > Call qio_channel_set_zerocopy(true) in the start of every multifd thread. > > > > Change the send_write() interface of multifd, allowing it to pass down > > flags for qio_channel_write*(). > > > > Pass down MSG_ZEROCOPY flag for sending memory pages, while keeping the > > other data being sent at the default copying approach. > > > > Signed-off-by: Leonardo Bras <leob...@redhat.com> > > --- > > migration/multifd-zlib.c | 7 ++++--- > > migration/multifd-zstd.c | 7 ++++--- > > migration/multifd.c | 9 ++++++--- > > migration/multifd.h | 3 ++- > > 4 files changed, 16 insertions(+), 10 deletions(-) > > > @@ -675,7 +676,8 @@ static void *multifd_send_thread(void *opaque) > > } > > > > if (used) { > > - ret = multifd_send_state->ops->send_write(p, used, > > &local_err); > > + ret = multifd_send_state->ops->send_write(p, used, > > MSG_ZEROCOPY, > > + &local_err); > > I don't think it is valid to unconditionally enable this feature due to the > resource usage implications > > https://www.kernel.org/doc/html/v5.4/networking/msg_zerocopy.html > > "A zerocopy failure will return -1 with errno ENOBUFS. This happens > if the socket option was not set, the socket exceeds its optmem > limit or the user exceeds its ulimit on locked pages."
You are correct, I unfortunately missed this part in the docs :( > The limit on locked pages is something that looks very likely to be > exceeded unless you happen to be running a QEMU config that already > implies locked memory (eg PCI assignment) Do you mean the limit an user has on locking memory? If so, that makes sense. I remember I needed to set the upper limit of locked memory for the user before using it, or adding a capability to qemu before. Maybe an option would be trying to mlock all guest memory before setting zerocopy=on in qemu code. If it fails, we can print an error message and fall back to not using zerocopy (following the idea of a new io_async_writev() I told you in the previous mail). > > > Regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| > Best regards, Leonardo