[PATCH] MAINTAINERS: Remove myself as reviewer from Live Migration

2023-12-21 Thread Leonardo Bras
I am currently focusing in kernel development, so I will probably not be of much help in reviewing general Live Migration changes. For above reason I am removing my Reviewer status from Migration and RDMA Migration. Signed-off-by: Leonardo Bras --- MAINTAINERS | 2 -- 1 file changed, 2

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-10 Thread Leonardo Bras Soares Passos
On Mon, Jul 10, 2023 at 3:16 PM Michael S. Tsirkin wrote: > > On Mon, Jul 10, 2023 at 02:49:05PM -0300, Leonardo Bras Soares Passos wrote: > > On Thu, Jul 6, 2023 at 5:00 PM Michael S. Tsirkin wrote: > > > > > > On Thu, Jul 06, 2023 at 03:02:07PM -0400, Peter Xu

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-10 Thread Leonardo Bras Soares Passos
n Thu, Jul 06, 2023 at 03:07:40PM -0300, Leonardo Bras Soares Passos > > > > wrote: > > > > > > I asked the same question, and I still keep confused: whether > > > > > > there's a > > > > > > first bad commit? Starting fro

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-06 Thread Leonardo Bras Soares Passos
On Thu, Jul 6, 2023 at 3:24 PM Peter Xu wrote: > > On Thu, Jul 06, 2023 at 03:07:40PM -0300, Leonardo Bras Soares Passos wrote: > > > I asked the same question, and I still keep confused: whether there's a > > > first bad commit? Starting from when it fails? >

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-06 Thread Leonardo Bras Soares Passos
On Thu, Jul 6, 2023 at 3:24 PM Peter Xu wrote: > > On Thu, Jul 06, 2023 at 03:07:40PM -0300, Leonardo Bras Soares Passos wrote: > > > I asked the same question, and I still keep confused: whether there's a > > > first bad commit? Starting from when it fails? >

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-06 Thread Leonardo Bras Soares Passos
On Thu, Jul 6, 2023 at 11:35 AM Peter Xu wrote: > > On Thu, Jul 06, 2023 at 01:55:47AM -0300, Leonardo Bras wrote: > > When trying to migrate a machine type pc-q35-6.0 or lower, with this > > cmdline options, > > > > -device > > driver=pcie-root-port,port=1

Re: [PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-06 Thread Leonardo Bras Soares Passos
On Thu, Jul 6, 2023 at 4:37 AM Juan Quintela wrote: > > Leonardo Bras wrote: > > When trying to migrate a machine type pc-q35-6.0 or lower, with this > > cmdline options, > > > > -device > > driver=pcie-root-port,port=18,chassis=19,id=pcie-root-port18,bus

[PATCH v2 1/1] pcie: Add hotplug detect state register to cmask

2023-07-05 Thread Leonardo Bras
sk field. So, clear PCI_EXP_SLTSTA_PDS bit on cmask, so the fake incompatibility on get_pci_config_device() does not abort the migration. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2215819 Signed-off-by: Leonardo Bras --- hw/pci/pcie.c | 4 1 file changed, 4 insertions(+) diff

Re: [PATCH 1/1] pcie: Add hotplug detect state register to w1cmask

2023-07-05 Thread Leonardo Bras Soares Passos
On Wed, Jul 5, 2023 at 3:40 AM Leonardo Bras Soares Passos wrote: > > On Tue, Jul 4, 2023 at 3:43 AM Michael S. Tsirkin wrote: > > > > On Tue, Jul 04, 2023 at 03:20:36AM -0300, Leonardo Brás wrote: > > > Hello Peter and Michael, I have a few updates on this: > >

Re: [PATCH 1/1] pcie: Add hotplug detect state register to w1cmask

2023-07-04 Thread Leonardo Bras Soares Passos
t; > > > > > Thanks for figuring this out. Let me copy a few more potential > > > > > > > > reviewers > > > > > > > > from commit 17858a1695 ("hw/acpi/ich9: Set ACPI PCI hot-plug as > > > > > > >

[PATCH 1/1] pcie: Add hotplug detect state register to w1cmask

2023-06-29 Thread Leonardo Bras
k, so the fake incompatibility on get_pci_config_device() does not abort the migration. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2215819 Signed-off-by: Leonardo Bras --- hw/pci/pcie.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/pci/pcie.c b/hw/pci/pcie.c index b8c24cf45

Re: [PATCH v2 16/16] migration/multifd: Compute transferred bytes correctly

2023-05-26 Thread Leonardo Bras Soares Passos
eturn 0; > >> } > > > > Humm, those are atomic ops, right? > > > > You think we could have 'multifd_bytes' and 'transferred' in the same > > cacheline, > > to avoid 2 cacheline bounces? > > Don't matter on next series. > > mig_stats.transferred is dropped. > > And transferred becomes: > > qemu_file_transferred + multifd_bytes + rdma_bytes. > > So everytime that we do a write, we only update one counter. That's even better :) Thanks! > > > Well, it's unrelated to this patchset, so: > > > > Reviewed-by: Leonardo Bras >

Re: [PATCH v2 15/16] migration/rdma: Simplify the function that saves a page

2023-05-26 Thread Leonardo Bras Soares Passos
* the current chunk. So, we have to call qemu_update_position() > >> - * later on when the actual write occurs. > >> - */ > >> -if (bytes_sent) { > >> -*bytes_sent = 1; > >> -} > >> - > >> /* > >> * Drain the Completion Queue if possible, but do not block, > >> * just poll. > > > > Oh, so this one complements 13/16. > > Since it doesn't do imaginary transfers anymore, there is no need to use > > bytes_sent pointer to keep track of them anymore. > > > > Other than the pages_sent above that I couldn't understand: > > Reviewed-by: Leonardo Bras > > Dropping that bit. > > Thanks. >

Re: [PATCH v2 09/16] migration: We don't need the field rate_limit_used anymore

2023-05-26 Thread Leonardo Bras Soares Passos
gt; Already in tree. > > See explanation for the split on previous patch. > > The other reason for the split is that this part of the patch is trivial > to review O:-) :-) > > Later, Juan. > > > > > Reviewed-by: Leonardo Bras >

Re: [PATCH v2 08/16] migration: Use migration_transferred_bytes() to calculate rate_limit

2023-05-26 Thread Leonardo Bras Soares Passos
On Fri, May 26, 2023 at 5:17 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Mon, 2023-05-15 at 21:57 +0200, Juan Quintela wrote: > >> Signed-off-by: Juan Quintela > >> Reviewed-by: Cédric Le Goater > >> --- > >> migration/migration-stats.h | 8 +++- > >> migration/migration-stats.

Re: [PATCH v2 04/16] qemu-file: Account for rate_limit usage on qemu_fflush()

2023-05-26 Thread Leonardo Bras Soares Passos
On Fri, May 26, 2023 at 5:09 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Mon, 2023-05-15 at 21:56 +0200, Juan Quintela wrote: > >> That is the moment we know we have transferred something. > >> > >> Signed-off-by: Juan Quintela > >> Reviewed-by: Cédric Le Goater > >> --- > >> migra

Re: [PATCH v2 03/16] migration: Move setup_time to mig_stats

2023-05-26 Thread Leonardo Bras Soares Passos
On Fri, May 26, 2023 at 5:07 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Mon, 2023-05-15 at 21:56 +0200, Juan Quintela wrote: > >> It is a time that needs to be cleaned each time cancel migration. > >> Once there create migration_time_since() to calculate how time since a > >> time in

Re: [PATCH v2 02/16] migration: Correct transferred bytes value

2023-05-26 Thread Leonardo Bras Soares Passos
On Fri, May 26, 2023 at 5:04 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Mon, 2023-05-15 at 21:56 +0200, Juan Quintela wrote: > >> We forget several places to add to trasferred amount of data. With > >> this fixes I get: > >> > >>qemu_file_transferred() + multifd_bytes == transfe

Re: [PATCH v1 1/1] hw/pci: Disable PCI_ERR_UNCOR_MASK register for machine type < 8.0

2023-05-03 Thread Leonardo Bras Soares Passos
On Wed, May 3, 2023 at 6:49 AM Jonathan Cameron wrote: > > On Tue, 2 May 2023 21:27:02 -0300 > Leonardo Bras wrote: > > > Since it's implementation on v8.0.0-rc0, having the PCI_ERR_UNCOR_MASK > > set for machine types < 8.0 will cause migration to fail if the ta

[PATCH v1 1/1] hw/pci: Disable PCI_ERR_UNCOR_MASK register for machine type < 8.0

2023-05-02 Thread Leonardo Bras
e type <= 7.2. Fixes: 010746ae1d ("hw/pci/aer: Implement PCI_ERR_UNCOR_MASK register") Suggested-by: Michael S. Tsirkin Signed-off-by: Leonardo Bras --- include/hw/pci/pci.h | 2 ++ hw/core/machine.c| 1 + hw/pci/pci.c | 2 ++ hw/pci/pcie_aer.c| 11 +++ 4

[PATCH v2 1/1] migration: Disable postcopy + multifd migration

2023-04-24 Thread Leonardo Bras
there is not enough testing/support nor any reported users for this scenario, we should disable this combination before it may cause any problems for users. Suggested-by: Dr. David Alan Gilbert Signed-off-by: Leonardo Bras Acked-by: Peter Xu Reviewed-by: Dr. David Alan Gilbert --- Changes sinc

Re: [PATCH] MAINTAINERS: Add Leonardo and Peter as reviewers

2023-04-24 Thread Leonardo Bras Soares Passos
y they got CC'd on every migration patch. > > > > Signed-off-by: Juan Quintela > > Acked-by: Peter Xu > > -- > Peter Xu > Acked-by: Leonardo Bras Thanks, Leonardo Bras

[RFC PATCH v1 1/1] migration: Disable postcopy + multifd migration

2023-03-27 Thread Leonardo Bras
s not enough testing/support nor any reported users for this scenario, we should disable this combination before it may cause any problems for users. Suggested-by: Dr. David Alan Gilbert Signed-off-by: Leonardo Bras --- migration/migration.c | 5 + 1 file changed, 5 insertions(+) diff --

[PATCH v1 2/4] migration/multifd: Remove unnecessary assignment on multifd_load_cleanup()

2023-02-09 Thread Leonardo Bras
Before assigning "p->quit = true" for every multifd channel, multifd_load_cleanup() will call multifd_recv_terminate_threads() which already does the same assignment, while protected by a mutex. So there is no point doing the same assignment again. Signed-off-by: Leonardo Bras -

[PATCH v1 3/4] migration/multifd: Join all multifd threads in order to avoid leaks

2023-02-09 Thread Leonardo Bras
migration is usually just ran at most a few times, and after it succeeds there is not much to be done before exiting the process. Yet still, it should not hurt performance to join all of them. Signed-off-by: Leonardo Bras --- migration/multifd.c | 3 ++- 1 file changed, 2 insertions(+), 1

[PATCH v1 1/4] migration/multifd: Change multifd_load_cleanup() signature and usage

2023-02-09 Thread Leonardo Bras
n value to decide on setting autostart = false, which will never happen. In order to simplify the codebase, change multifd_load_cleanup() signature to 'void multifd_load_cleanup(void)', and for every usage remove error handling or decision made based on return value != 0. Signed-off-by:

[PATCH v1 4/4] migration/multifd: Move load_cleanup inside incoming_state_destroy

2023-02-09 Thread Leonardo Bras
up() to make sure nothing else is received before dirty_bitmap_mig_before_vm_start(). Signed-off-by: Leonardo Bras --- migration/multifd.h | 1 + migration/migration.c | 4 +++- migration/multifd.c | 7 +++ 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/migration/multifd.h

[PATCH v1 1/4] migration/multifd: Change multifd_load_cleanup() signature and usage

2023-02-09 Thread Leonardo Bras
n value to decide on setting autostart = false, which will never happen. In order to simplify the codebase, change multifd_load_cleanup() signature to 'void multifd_load_cleanup(void)', and for every usage remove error handling or decision made based on return value != 0. Signed-off-by:

[PATCH v1 4/4] migration/multifd: Move load_cleanup inside incoming_state_destroy

2023-02-09 Thread Leonardo Bras
up() to make sure nothing else is received before dirty_bitmap_mig_before_vm_start(). Signed-off-by: Leonardo Bras --- migration/multifd.h | 1 + migration/migration.c | 4 +++- migration/multifd.c | 7 +++ 3 files changed, 11 insertions(+), 1 deletion(-) diff --git a/migration/multifd.h

[PATCH v1 3/4] migration/multifd: Join all multifd threads in order to avoid leaks

2023-02-09 Thread Leonardo Bras
migration is usually just ran at most a few times, and after it succeeds there is not much to be done before exiting the process. Yet still, it should not hurt performance to join all of them. Signed-off-by: Leonardo Bras --- migration/multifd.c | 3 ++- 1 file changed, 2 insertions(+), 1

[PATCH v1 2/4] migration/multifd: Remove unnecessary assignment on multifd_load_cleanup()

2023-02-09 Thread Leonardo Bras
Before assigning "p->quit = true" for every multifd channel, multifd_load_cleanup() will call multifd_recv_terminate_threads() which already does the same assignment, while protected by a mutex. So there is no point doing the same assignment again. Signed-off-by: Leonardo Bras -

Re: [PATCH] migration: Show downtime during postcopy phase

2022-12-20 Thread Leonardo Bras Soares Passos
t; info->has_total_time = true; > info->total_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME) - > s->start_time; > +} > + > +if (migrate_show_downtime(s)) { > +info->has_downtime = true; > + info->downtime = s->downtime; > +} else { > info->has_expected_downtime = true; > info->expected_downtime = s->expected_downtime; > } > -- > 2.37.3 > FWIW: Reviewed-by: Leonardo Bras

Re: [PATCH v1 1/1] migration: Fix yank on postcopy multifd crashing guest after migration

2022-11-29 Thread Leonardo Bras Soares Passos
Hello Peter, On Thu, Nov 24, 2022 at 1:04 PM Peter Xu wrote: > > On Wed, Nov 09, 2022 at 02:56:29AM -0300, Leonardo Bras wrote: > > diff --git a/migration/savevm.c b/migration/savevm.c > > index a0cdb714f7..250caff7f4 100644 > > --- a/migration/savevm.c >

Re: [PATCH v1 1/1] migration: Fix yank on postcopy multifd crashing guest after migration

2022-11-14 Thread Leonardo Bras Soares Passos
On Thu, Nov 10, 2022 at 10:48 AM Juan Quintela wrote: > > Leonardo Bras wrote: > D> When multifd and postcopy-ram capabilities are enabled, if a > > migrate-start-postcopy is attempted, the migration will finish sending the > > memory pages and then crash with the follo

Re: [PATCH v1 1/1] migration: Fix yank on postcopy multifd crashing guest after migration

2022-11-09 Thread Leonardo Bras Soares Passos
On Wed, Nov 9, 2022 at 10:31 AM Dr. David Alan Gilbert wrote: > > * Leonardo Bras (leob...@redhat.com) wrote: > > When multifd and postcopy-ram capabilities are enabled, if a > > migrate-start-postcopy is attempted, the migration will finish sending the > > memory pages

[PATCH v1 1/1] migration: Fix yank on postcopy multifd crashing guest after migration

2022-11-08 Thread Leonardo Bras
ead() before MIGRATION_YANK_INSTANCE is unregistered. Fixes: b5eea99ec2 ("migration: Add yank feature") Reported-by: Li Xiaohui Signed-off-by: Leonardo Bras --- migration/migration.h | 1 + migration/migration.c | 18 +- migration/savevm.c| 2 ++ 3 files changed, 16 inse

[RFC PATCH 0/4] MultiFD zero-copy improvements

2022-10-24 Thread Leonardo Bras
to happen while the most recent ones are ongoing, instead of waiting for everything to finish before sending more. It all works fine in my tests, but maybe I missed some cornercase. Please provide any feedback you find fit. Thank you all! Best regards, Leo Leonardo Bras (4): migration/multifd

[RFC PATCH 1/4] migration/multifd/zero-copy: Create helper function for flushing

2022-10-24 Thread Leonardo Bras
Move flushing code from multifd_send_sync_main() to a new helper, and call it in multifd_send_sync_main(). Signed-off-by: Leonardo Bras --- migration/multifd.c | 30 +++--- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/migration/multifd.c b/migration

[RFC PATCH 4/4] migration/multifd/zero-copy: Flush only the LRU half of the header array

2022-10-24 Thread Leonardo Bras
hput while consuming less cpu time. Signed-off-by: Leonardo Bras --- migration/multifd.c | 16 +++- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index c5d1f911a4..fe9df460f6 100644 --- a/migration/multifd.c +++ b/migration

[RFC PATCH 3/4] QIOChannel: Add max_pending parameter to qio_channel_flush()

2022-10-24 Thread Leonardo Bras
he maximum number of pending writes remaining before the function returns. Also, implement this change in qio_channel_socket_flush(). Change current calls of qio_channel_flush() so (max_pending == 0), and the flush-all behavior is maintained. Signed-off-by: Leonardo Bras --- include/io/cha

[RFC PATCH 2/4] migration/multifd/zero-copy: Merge header & pages send in a single write

2022-10-24 Thread Leonardo Bras
which element of the array will be used as a header. Suggested-by: Juan Quintela Signed-off-by: Leonardo Bras --- migration/multifd.h | 5 - migration/multifd.c | 52 - 2 files changed, 32 insertions(+), 25 deletions(-) diff --git a/migration/multif

Re: [PATCH v7 07/12] multifd: Prepare to send a packet without the mutex held

2022-08-20 Thread Leonardo Bras Soares Passos
On Fri, Aug 19, 2022 at 8:32 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Tue, 2022-08-02 at 08:39 +0200, Juan Quintela wrote: > >> We do the send_prepare() and the fill of the head packet without the > >> mutex held. It will help a lot for compression and later in the > >> series for

Re: [PATCH v7 06/12] multifd: Make flags field thread local

2022-08-20 Thread Leonardo Bras Soares Passos
_state->channels_ready); > > > > IIUC it uses p->sync_needed to keep the sync info, instead of the previous > > flags > > local var, and thus it can set p->flags = 0 earlier. Seems to not change any > > behavior AFAICS. > > The protection of the global flags was being wrong. That is the reason > that I decided to change it to the sync_needed. > > The problem was that at some point we were still sending a packet (that > shouldn't have the SYNC flag enabled), but we received a > multifd_main_sync() and it got enabled anyways. The easier way that I > found te fix it was this way. > > Problem was difficult to detect, that is the reason that I change it > this way. Oh, I see. > > >> -if (flags & MULTIFD_FLAG_SYNC) { > >> +if (sync_needed) { > >> qemu_sem_post(&multifd_recv_state->sem_sync); > >> qemu_sem_wait(&p->sem_sync); > >> } > > > > Ok, IIUC this part should have the same behavior as before, but using a bool > > instead of an u32. > > I changed it to make sure that we only checked the flags at the > beggining of the function, with the lock taken. Thanks for sharing! Best regards, Leo > > > > > FWIW: > > Reviewed-by: Leonardo Bras > > Thanks, Juan. >

Re: [PATCH v7 05/12] migration: Make ram_save_target_page() a pointer

2022-08-20 Thread Leonardo Bras Soares Passos
On Fri, Aug 19, 2022 at 6:52 AM Juan Quintela wrote: > > Leonardo Brás wrote: > > On Tue, 2022-08-02 at 08:39 +0200, Juan Quintela wrote: > >> We are going to create a new function for multifd latest in the series. > >> > >> Signed-off-by: Juan Quintela > >> Reviewed-by: Dr. David Alan Gilbert

Re: [RFC PATCH 2/2] kvm/kvm-all.c: listener should delay kvm_vm_ioctl to the commit phase

2022-08-18 Thread Leonardo Bras Soares Passos
On Thu, Aug 18, 2022 at 5:05 PM Peter Xu wrote: > > On Tue, Aug 16, 2022 at 06:12:50AM -0400, Emanuele Giuseppe Esposito wrote: > > +static void kvm_memory_region_node_add(KVMMemoryListener *kml, > > + struct kvm_userspace_memory_region > > *mem) > > +{ > > +

[PATCH v1 1/1] QIOChannelSocket: Add support for MSG_ZEROCOPY + IPV6

2022-08-04 Thread Leonardo Bras
c58ffc29 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Signed-off-by: Leonardo Bras --- io/channel-socket.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index e6e7a12ae1..4f6bb6fe37 1

Re: [PATCH v1 1/1] migration: add remaining params->has_* = true in migration_instance_init()

2022-07-26 Thread Leonardo Bras Soares Passos
parameter") On Mon, Jul 25, 2022 at 10:02 PM Leonardo Bras wrote: > > Some of params->has_* = true are missing in migration_instance_init, this > causes migrate_params_check() to skip some tests, allowing some > unsupported scenarios. > > Fix this by adding all missing para

[PATCH v1 1/1] migration: add remaining params->has_* = true in migration_instance_init()

2022-07-25 Thread Leonardo Bras
Some of params->has_* = true are missing in migration_instance_init, this causes migrate_params_check() to skip some tests, allowing some unsupported scenarios. Fix this by adding all missing params->has_* = true in migration_instance_init(). Signed-off-by: Leonardo Bras --- mig

[PATCH v1 1/1] migration: Avoid false-positive on non-supported scenarios for zero-copy-send

2022-07-19 Thread Leonardo Bras
ero-copy-send on migrate_params_check(). 3) XBZRLE is also a compression capability, so it makes sense to also add it to the list of capabilities which are not supported with zero-copy-send. Fixes: 1abaec9a1b2c ("migration: Change zero_copy_send from migration parameter to migration

Re: [PATCH v4 3/3] migration/multifd: Report to user when zerocopy not working

2022-07-12 Thread Leonardo Bras Soares Passos
On Tue, Jul 12, 2022 at 7:42 PM Peter Xu wrote: > > On Mon, Jul 11, 2022 at 06:11:13PM -0300, Leonardo Bras wrote: > > Some errors, like the lack of Scatter-Gather support by the network > > interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using > > ze

[PATCH v4 3/3] migration/multifd: Report to user when zerocopy not working

2022-07-11 Thread Leonardo Bras
, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then increment dirty_sync_missed_zero_copy migration stat to let the user know about it. Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé --- migration/ram.h

[PATCH v4 2/3] Add dirty-sync-missed-zero-copy migration stat

2022-07-11 Thread Leonardo Bras
Signed-off-by: Leonardo Bras Acked-by: Markus Armbruster Reviewed-by: Daniel P. Berrangé --- qapi/migration.json | 7 ++- migration/migration.c | 2 ++ monitor/hmp-cmds.c| 5 + 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/qapi/migration.json b/qapi

[PATCH v4 1/3] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-07-11 Thread Leonardo Bras
("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé Acked-by: Daniel P. Berrangé Reviewed-by: Juan Quintela --- io/channel-socket.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-)

[PATCH v4 0/3] Zero copy improvements (QIOChannel + multifd)

2022-07-11 Thread Leonardo Bras
ation release number changed from 7.2 to 7.1 - migration stat renamed from zero-copy-copied to dirty-sync-missed-zero-copy - Updated documentation to make it more user-friendly Changes since v1: - Idea of using a warning replaced by using a migration stat counter Leonardo Br

Re: [PATCH v3 1/3] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-07-11 Thread Leonardo Bras Soares Passos
On Thu, Jul 7, 2022 at 7:18 PM Peter Xu wrote: > > On Thu, Jul 07, 2022 at 06:14:17PM -0300, Leonardo Brás wrote: > > Having 'if(queued == sent)' will cause us to falsely return '1' in two buggy > > cases, while 'if queued == 0) will either skip early or go into 'infinite' > > loop. > > I'm not s

Re: [PATCH v3 3/3] migration/multifd: Warn user when zerocopy not working

2022-07-07 Thread Leonardo Bras Soares Passos
Hello Peter, On Thu, Jul 7, 2022 at 2:56 PM Peter Xu wrote: > > On Mon, Jul 04, 2022 at 05:23:15PM -0300, Leonardo Bras wrote: > > Some errors, like the lack of Scatter-Gather support by the network > > interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on u

Re: [PATCH v3 2/3] Add dirty-sync-missed-zero-copy migration stat

2022-07-07 Thread Leonardo Bras Soares Passos
Hello Peter, On Thu, Jul 7, 2022 at 2:54 PM Peter Xu wrote: > > On Mon, Jul 04, 2022 at 05:23:14PM -0300, Leonardo Bras wrote: > > Signed-off-by: Leonardo Bras > > --- > > qapi/migration.json | 7 ++- > > migration/migration.c | 2 ++ > > monitor

Re: [PATCH v3 1/3] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-07-07 Thread Leonardo Bras Soares Passos
Hello Peter, On Thu, Jul 7, 2022 at 2:47 PM Peter Xu wrote: > > Hi, Leo, > > On Mon, Jul 04, 2022 at 05:23:13PM -0300, Leonardo Bras wrote: > > If flush is called when no buffer was sent with MSG_ZEROCOPY, it currently > > returns 1. This return code should be used only

[PATCH v3 3/3] migration/multifd: Warn user when zerocopy not working

2022-07-04 Thread Leonardo Bras
, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then increment dirty_sync_missed_zero_copy migration stat to let the user know about it. Signed-off-by: Leonardo Bras --- migration/ram.h | 2 ++ migration/multifd.c | 2

[PATCH v3 2/3] Add dirty-sync-missed-zero-copy migration stat

2022-07-04 Thread Leonardo Bras
Signed-off-by: Leonardo Bras --- qapi/migration.json | 7 ++- migration/migration.c | 2 ++ monitor/hmp-cmds.c| 4 3 files changed, 12 insertions(+), 1 deletion(-) diff --git a/qapi/migration.json b/qapi/migration.json index 7102e474a6..fed08b9b88 100644 --- a/qapi/migration.json

[PATCH v3 1/3] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-07-04 Thread Leonardo Bras
("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Signed-off-by: Leonardo Bras --- io/channel-socket.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index 4466bb1cd4..698c086b70 1

[PATCH v3 0/3] Zero copy improvements (QIOChannel + multifd)

2022-07-04 Thread Leonardo Bras
more user-friendly Changes since v1: - Idea of using a warning replaced by using a migration stat counter Leonardo Bras (3): QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent Add dirty-sync-missed-zero-copy migration stat migration/multifd: Warn user when zerocop

[PATCH v2 2/3] Add zero-copy-copied migration stat

2022-07-01 Thread Leonardo Bras
Signed-off-by: Leonardo Bras --- qapi/migration.json | 5 - migration/migration.c | 1 + monitor/hmp-cmds.c| 4 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/qapi/migration.json b/qapi/migration.json index 7102e474a6..925f009868 100644 --- a/qapi/migration.json

[PATCH v2 3/3] migration/multifd: Warn user when zerocopy not working

2022-07-01 Thread Leonardo Bras
, which checks for errors each of the previous calls to sendmsg(...,MSG_ZEROCOPY). If all of them failed to use zero-copy, then increment zero_copy_copied migration stat to let the user know about it. Signed-off-by: Leonardo Bras --- migration/ram.h | 2 ++ migration/multifd.c | 2 ++ migration

[PATCH v2 1/3] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-07-01 Thread Leonardo Bras
("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Signed-off-by: Leonardo Bras --- io/channel-socket.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index 4466bb1cd4..698c086b70 1

[PATCH v2 0/3] Zero copy improvements (QIOChannel + multifd)

2022-07-01 Thread Leonardo Bras
used to keep track of [*]. Honestly I would like some help with this naming, which I don't think is quite good, but I could also not think on anything better. The third patch keeps track of how many zero-copy flushes retured 1 [*] Leonardo Bras (3): QIOChannelSocket: Fix zero-copy flush

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

2022-06-28 Thread Leonardo Bras Soares Passos
On Tue, Jun 28, 2022 at 10:52 AM Dr. David Alan Gilbert wrote: > > * Daniel P. Berrangé (berra...@redhat.com) wrote: > > On Tue, Jun 28, 2022 at 09:32:04AM -0300, Leonardo Bras Soares Passos wrote: > > > On Tue, Jun 28, 2022 at 4:53 AM Daniel P. Berrangé > > > wrot

Re: [PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

2022-06-28 Thread Leonardo Bras Soares Passos
On Tue, Jun 28, 2022 at 4:53 AM Daniel P. Berrangé wrote: > > On Mon, Jun 27, 2022 at 10:09:09PM -0300, Leonardo Bras wrote: > > Some errors, like the lack of Scatter-Gather support by the network > > interface(NETIF_F_SG) may cause sendmsg(...,MSG_ZEROCOPY) to fail on using &g

[PATCH v1 2/2] migration/multifd: Warn user when zerocopy not working

2022-06-27 Thread Leonardo Bras
dirty-bitmap iterations were not able to use zero-copy send. Signed-off-by: Leonardo Bras --- migration/multifd.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/migration/multifd.c b/migration/multifd.c index 684c014c86..9c62aec84e 100644 --- a/migration/multifd.c +++ b/migration

[PATCH v1 0/2] Zero copy improvements (QIOChannel + multifd)

2022-06-27 Thread Leonardo Bras
educes the frequency of multifd syncs, or else it could potentially send 20 warnings per second. ... Leonardo Bras (2): QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent migration/multifd: Warn

[PATCH v1 1/2] QIOChannelSocket: Fix zero-copy flush returning code 1 when nothing sent

2022-06-27 Thread Leonardo Bras
("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Signed-off-by: Leonardo Bras --- io/channel-socket.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index 4466bb1cd4..698c086b70 1

[PATCH v4 4/4] migration: Change zero_copy_send from migration parameter to migration capability

2022-06-19 Thread Leonardo Bras
how recently the change got merged, it was decided that it's still time to make it right, and convert zero_copy_send into a Migration capability. Signed-off-by: Leonardo Bras --- qapi/migration.json | 33 --- migration/migration.c

[PATCH v4 2/4] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-19 Thread Leonardo Bras
corruption in RAM during migration. Fixes: 2bc58ffc2926 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Reported-by: 徐闯 Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé --- io/channel-socket.c | 5 + 1 file changed, 5 insertions(+) d

Re: [PATCH v4 4/4] migration: Change zero_copy_send from migration parameter to migration capability

2022-06-19 Thread Leonardo Bras Soares Passos
CC: Jiri Denemark On Mon, Jun 20, 2022 at 2:40 AM Leonardo Bras wrote: > > When originally implemented, zero_copy_send was designed as a Migration > paramenter. > > But taking into account how is that supposed to work, and how > the difference between a capability and a parame

[PATCH v4 0/4] MSG_ZEROCOPY fixes & improvements

2022-06-19 Thread Leonardo Bras
4 is about making zero-copy-send a migration capability, instead of a migration parameter. Which actually makes more sense and helps the implementation of the libvirt code. Leonardo Bras (4): QIOChannelSocket: Introduce assert and reduce ifdefs to improve readability QIOChannelSocket: Fix

[PATCH v4 3/4] migration: zero-copy flush only at the end of bitmap scanning

2022-06-19 Thread Leonardo Bras
meout. Change so it flushes only after a whole scanning of the dirty bimap, so it never sends a newer version of a page before an older one, while avoiding unnecessary overhead. Signed-off-by: Leonardo Bras --- migration/multifd.h | 1 + migration/multifd.c

[PATCH v4 1/4] QIOChannelSocket: Introduce assert and reduce ifdefs to improve readability

2022-06-19 Thread Leonardo Bras
ned-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé --- io/channel-socket.c | 14 +- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index dc9c165de1..dac9e60c20 100644 --- a/io/channel-socket.c +++ b/io/channel-socket.c

Re: [PATCH v2 1/2] QIOChannelSocket: Reduce ifdefs to improve readability

2022-06-14 Thread Leonardo Bras Soares Passos
On Tue, Jun 14, 2022 at 5:36 AM Daniel P. Berrangé wrote: > > On Mon, Jun 13, 2022 at 06:21:18PM -0300, Leonardo Bras Soares Passos wrote: > > On Fri, Jun 10, 2022 at 5:25 AM Daniel P. Berrangé > > wrote: > > > > > > > [...] > > > > >

[PATCH v3 2/2] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-13 Thread Leonardo Bras
corruption in RAM during migration. Fixes: 2bc58ffc2926 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Reported-by: 徐闯 Signed-off-by: Leonardo Bras --- io/channel-socket.c | 5 + 1 file changed, 5 insertions(+) diff --git a/io/channel-socket.

[PATCH v3 1/2] QIOChannelSocket: Introduce assert and reduce ifdefs to improve readability

2022-06-13 Thread Leonardo Bras
ned-off-by: Leonardo Bras --- io/channel-socket.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/io/channel-socket.c b/io/channel-socket.c index dc9c165de1..cdce7b0b45 100644 --- a/io/channel-socket.c +++ b/io/channel-socket.c @@ -578,11 +578,13 @@ stat

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-13 Thread Leonardo Bras Soares Passos
On Mon, Jun 13, 2022 at 7:53 PM Peter Xu wrote: > > On Mon, Jun 13, 2022 at 05:58:44PM -0300, Leonardo Bras Soares Passos wrote: > > Hello Peter, > > > > On Wed, Jun 8, 2022 at 5:23 PM Peter Xu wrote: > > [...] > > > > In a previous iteration of

Re: [PATCH v2 1/2] QIOChannelSocket: Reduce ifdefs to improve readability

2022-06-13 Thread Leonardo Bras Soares Passos
On Fri, Jun 10, 2022 at 5:25 AM Daniel P. Berrangé wrote: > [...] > Ok, so if it is checked earlier then we merely need an assert. > > if (flags & QIO_CHANNEL_WRITE_FLAG_ZERO_COPY) { > #ifdef QEMU_MSG_ZEROCOPY > sflags = MSG_ZEROCOPY; > zero_copy_enabled = true; > #else

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-13 Thread Leonardo Bras Soares Passos
Hello Peter, On Wed, Jun 8, 2022 at 5:23 PM Peter Xu wrote: [...] > > In a previous iteration of the patchset, it was made clear that it's > > desirable to detect when the kernel falls back to copying mechanism, > > so the user of 'QIOChannelSocket' can switch to copying and avoid the > > overhea

Re: [PATCH v2 1/2] QIOChannelSocket: Reduce ifdefs to improve readability

2022-06-09 Thread Leonardo Bras Soares Passos
Hello Daniel, On Thu, Jun 9, 2022 at 5:10 AM Daniel P. Berrangé wrote: > > On Wed, Jun 08, 2022 at 06:04:02PM -0300, Leonardo Bras wrote: > > During implementation of MSG_ZEROCOPY feature, a lot of #ifdefs were > > introduced, particularly at qio_channel_socket_writev(). >

[PATCH v2 2/2] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-08 Thread Leonardo Bras
corruption in RAM during migration. Fixes: 2bc58ffc2926 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Reported-by: 徐闯 Signed-off-by: Leonardo Bras --- io/channel-socket.c | 5 + 1 file changed, 5 insertions(+) diff --git a/io/channel-socket.

[PATCH v2 1/2] QIOChannelSocket: Reduce ifdefs to improve readability

2022-06-08 Thread Leonardo Bras
During implementation of MSG_ZEROCOPY feature, a lot of #ifdefs were introduced, particularly at qio_channel_socket_writev(). Rewrite some of those changes so it's easier to read. ... Signed-off-by: Leonardo Bras --

Re: [PATCH v1 1/1] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-08 Thread Leonardo Bras Soares Passos
On Wed, Jun 8, 2022 at 5:55 PM Peter Xu wrote: > > On Wed, Jun 08, 2022 at 04:26:10PM -0400, Peter Xu wrote: > > On Wed, Jun 08, 2022 at 03:18:09PM -0300, Leonardo Bras wrote: > > > Somewhere between v6 and v7 the of the zero-copy-send patchset a crucial > > > part

Re: [PATCH v1 1/1] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-08 Thread Leonardo Bras Soares Passos
Hello Daniel, On Wed, Jun 8, 2022 at 3:46 PM Daniel P. Berrangé wrote: > > On Wed, Jun 08, 2022 at 03:18:09PM -0300, Leonardo Bras wrote: > > Somewhere between v6 and v7 the of the zero-copy-send patchset a crucial > > part of the flushing mechanism got missing: incrementin

[PATCH v1 1/1] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-08 Thread Leonardo Bras
in RAM during migration. Fixes: 2bc58ffc2926 ("QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX") Reported-by: 徐闯 Signed-off-by: Leonardo Bras --- io/channel-socket.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --gi

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-08 Thread Leonardo Bras Soares Passos
On Wed, Jun 8, 2022 at 8:41 AM Peter Xu wrote: > > On Wed, Jun 08, 2022 at 02:37:28AM -0300, Leonardo Bras Soares Passos wrote: > > (1) is not an option, as the interface currently uses ret=1 to make > > sure MSG_ZEROCOPY is getting used, > > I added that so the user of qi

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-07 Thread Leonardo Bras Soares Passos
Hello Peter, On Wed, Jun 1, 2022 at 10:58 AM Peter Xu wrote: > [...] > > Hi, Leonardo. I'm also paying attention to the application of MSG_ZEROCOPY > > in live migration recently. I noticed that you defined a member > > `zero_copy_queued` in the struct QIOChannelSocket, but I can't find out > > w

Re: [External] [PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-06-07 Thread Leonardo Bras Soares Passos
Hello 徐闯, Thanks for reviewing! On Wed, Jun 1, 2022 at 6:37 AM 徐闯 wrote: [...] > Hi, Leonardo. I'm also paying attention to the application of > MSG_ZEROCOPY in live migration recently. I noticed that you defined a > member `zero_copy_queued` in the struct QIOChannelSocket, but I can't > find ou

[PATCH v13 8/8] multifd: Implement zero copy write in multifd migration (multifd-zero-copy)

2022-05-13 Thread Leonardo Bras
tion with zero-copy enabled, so disabling the feature should be necessary for low-privileged users trying to perform multifd migrations. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé --- migration/multifd.h | 2 ++ migration/migration.c

[PATCH v13 6/8] multifd: multifd_send_sync_main now returns negative on error

2022-05-13 Thread Leonardo Bras
(This change is important to next patch on multifd zero copy implementation, to make it sure an error in zero-copy flush does not go unnoticed. Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé Reviewed-by: Peter Xu --- migration/multifd.h | 2 +- migration/multifd.c | 10 ++

[PATCH v13 4/8] migration: Add zero-copy-send parameter for QMP/HMP for Linux

2022-05-12 Thread Leonardo Bras
. On non-Linux builds this parameter is compiled-out. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Acked-by: Markus Armbruster --- qapi/migration.json | 24 migration/migration.h | 5 + migr

[PATCH v13 3/8] QIOChannelSocket: Implement io_writev zero copy flag & io_flush for CONFIG_LINUX

2022-05-12 Thread Leonardo Bras
nism to disable it, so it can still be accessible to less privileged users. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé Reviewed-by: Juan Quintela --- include/io/channel-socket.h | 2 + io/channel-socket.c | 116 +

[PATCH v13 7/8] multifd: Send header packet without flags if zero-copy-send is enabled

2022-05-12 Thread Leonardo Bras
happening before. This only changes how a migration with zero-copy-send=true works, not changing any current behavior for migrations with zero-copy-send=false. Signed-off-by: Leonardo Bras Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé --- migration/multifd.c | 22 +++-

[PATCH v13 2/8] QIOChannel: Add flags on io_writev and introduce io_flush callback

2022-05-12 Thread Leonardo Bras
flags. Signed-off-by: Leonardo Bras Reviewed-by: Daniel P. Berrangé Reviewed-by: Peter Xu Reviewed-by: Juan Quintela --- include/io/channel.h| 38 +- chardev/char-io.c | 2 +- hw/remote/mpqemu-link.c | 2 +- io/channel-buf

[PATCH v13 1/8] meson.build: Fix docker-test-build@alpine when including linux/errqueue.h

2022-05-12 Thread Leonardo Bras
mmand failed === As above error message suggests, 'struct __kernel_timespec' was already defined by liburing/compat.h. Fix alpine CI by adding test to disable liburing in configure step if a redefinition happens between linux/errqueue.h and liburing/compat.h. Signed-off-by: Leonar

[PATCH v13 5/8] migration: Add migrate_use_tls() helper

2022-05-12 Thread Leonardo Bras
A lot of places check parameters.tls_creds in order to evaluate if TLS is in use, and sometimes call migrate_get_current() just for that test. Add new helper function migrate_use_tls() in order to simplify testing for TLS usage. Signed-off-by: Leonardo Bras Reviewed-by: Juan Quintela Reviewed

  1   2   3   4   >