Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling

2024-06-05 Thread Peter Xu
dea you mention), if > the destination can issue an RDMA read itself, it doesn't need to send > messages > to the source to ask for a page fetch; it just goes and grabs it itself, > that's got to be good for latency. Oh, that's pretty internal stuff of rdma to me and beyond my knowledge.. but from what I can tell it sounds very reasonable indeed! Thanks! -- Peter Xu

Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling

2024-06-05 Thread Peter Xu
On Wed, Jun 05, 2024 at 10:10:57AM -0400, Peter Xu wrote: > > e) Someone made a good suggestion (sorry can't remember who) - that the > > RDMA migration structure was the wrong way around - it should be the > > destination which initiates an RDMA read, rath

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-05 Thread Peter Xu
On Wed, Jun 05, 2024 at 10:09:43AM +, Gonglei (Arei) wrote: > Hi Peter, > > > -Original Message- > > From: Peter Xu [mailto:pet...@redhat.com] > > Sent: Wednesday, June 5, 2024 3:32 AM > > To: Gonglei (Arei) > > Cc: qemu-devel@nongnu.org; yu

Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling

2024-06-05 Thread Peter Xu
icely for postcopy. I'm not sure whether it'll still be a problem if rdma recv side is based on zero-copy. It would be a matter of whether atomicity can be guaranteed so that we don't want the guest vcpus to see a partially copied page during on-flight DMAs. UFFDIO_COPY (or friend) is currently the only solution for that. Thanks, -- Peter Xu

Re: [PATCH 0/7] Live migration acceleration with UADK

2024-06-04 Thread Peter Xu
ich can  free up CPU computing power and improve computing performance. > > This series is on top of Intel IAA accelerator live migration support > series[1] from Yuan Liu. Many thanks for doing this. Just looked at the IAA series too, I didn't read multifd-*.[ch] much on both sides but bot

Re: [PATCH v2 18/18] migration/ram: Add direct-io support to precopy file migration

2024-06-04 Thread Peter Xu
direct-io during that moment. > > Signed-off-by: Fabiano Rosas Is anyone going to consume this? How's the performance? It doesn't look super fast to me if we need to enable/disable dio in each loop.. then it's a matter of whether we should bother, or would it be easier that we simply require multifd when direct-io=on. Thanks, -- Peter Xu

Re: [PATCH v2 15/18] tests/qtest/migration: Add a test for mapped-ram with passing of fds

2024-06-04 Thread Peter Xu
need > to match the flags QEMU uses to open the file. Currently O_WRONLY > for src and O_RDONLY for dst. > > Note that fdset code goes under _WIN32 because fd passing is not > supported on Windows. > > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v2 14/18] migration: Add documentation for fdset with multifd + file

2024-06-04 Thread Peter Xu
ensured to be written respecting O_DIRECT alignment > -restrictions (direct-io support not yet implemented). > +restrictions. > > Usage > - > @@ -35,6 +35,10 @@ Use a ``file:`` URL for migration: > Mapped-ram migration is best done non-live, i.e. by stopping the VM on > the source side before migrating. > > +For best performance enable the ``direct-io`` capability as well: Parameter? > + > +``migrate_set_capability direct-io on`` migrate_set_parameters? If with that fixed: Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v7 1/7] docs/migration: add qpl compression feature

2024-06-04 Thread Peter Xu
On Mon, Jun 03, 2024 at 11:41:00PM +0800, Yuan Liu wrote: > add Intel Query Processing Library (QPL) compression method > introduction > > Signed-off-by: Yuan Liu > Reviewed-by: Nanhai Zou Acked-by: Peter Xu -- Peter Xu

Re: [PATCH 0/6] refactor RDMA live migration based on rsocket API

2024-06-04 Thread Peter Xu
ead first to keep the dest QEMU main loop intact, I think, but I hope we will reach that irrelevant of rdma, IOW it'll be nice to happen even earlier if possible. Another nitpick is that qio_channel_rdma_listen_async() doesn't look used and may prone to removal. Thanks, -- Peter Xu

Re: [PATCH V1 17/26] machine: memfd-alloc option

2024-06-04 Thread Peter Xu
On Tue, Jun 04, 2024 at 06:14:08PM +0200, David Hildenbrand wrote: > On 04.06.24 17:58, Peter Xu wrote: > > On Tue, Jun 04, 2024 at 08:13:26AM +0100, Daniel P. Berrangé wrote: > > > On Mon, Jun 03, 2024 at 05:48:32PM -0400, Peter Xu wrote: > > > > That property, i

Re: [PATCH V1 17/26] machine: memfd-alloc option

2024-06-04 Thread Peter Xu
On Tue, Jun 04, 2024 at 08:13:26AM +0100, Daniel P. Berrangé wrote: > On Mon, Jun 03, 2024 at 05:48:32PM -0400, Peter Xu wrote: > > That property, irrelevant of what it is called (and I doubt whether Dan's > > suggestion on "shared-ram" is good, e.g. mmap(MAP_SHARED) does

Re: [PATCH V1 19/26] physmem: preserve ram blocks for cpr

2024-06-03 Thread Peter Xu
On Fri, May 31, 2024 at 03:32:11PM -0400, Steven Sistare wrote: > On 5/30/2024 2:39 PM, Peter Xu wrote: > > On Thu, May 30, 2024 at 01:12:40PM -0400, Steven Sistare wrote: > > > On 5/29/2024 3:25 PM, Peter Xu wrote: > > > > On Wed, May 29, 2024 at 01:31:53

Re: [PATCH V1 17/26] machine: memfd-alloc option

2024-06-03 Thread Peter Xu
On Fri, May 31, 2024 at 03:32:05PM -0400, Steven Sistare wrote: > On 5/30/2024 2:14 PM, Peter Xu wrote: > > On Thu, May 30, 2024 at 01:11:09PM -0400, Steven Sistare wrote: > > > On 5/29/2024 3:14 PM, Peter Xu wrote: > > > > On Wed, May 29, 2024 at 01:31:38

Re: [PATCH v2 4/4] tests/qtest/migration-test: Add a postcopy memfile test

2024-06-03 Thread Peter Xu
On Mon, Jun 03, 2024 at 04:02:42PM +1000, Nicholas Piggin wrote: > On Fri May 31, 2024 at 11:34 PM AEST, Peter Xu wrote: > > On Thu, May 30, 2024 at 07:54:07PM +1000, Nicholas Piggin wrote: > > > Postcopy requires userfaultfd support, which requires tmpfs if a memory

Re: [PATCH v2 07/18] monitor: Simplify fdset and fd removal

2024-05-31 Thread Peter Xu
On Thu, May 23, 2024 at 04:05:37PM -0300, Fabiano Rosas wrote: > Remove fds right away instead of setting the ->removed flag. We don't > need the extra complexity of having a cleanup function reap the > removed entries at a later time. > > Signed-off-by: Fabiano Rosas Rev

Re: [PATCH v2 13/18] monitor: fdset: Match against O_DIRECT

2024-05-31 Thread Peter Xu
On Fri, May 31, 2024 at 12:42:05PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Thu, May 23, 2024 at 04:05:43PM -0300, Fabiano Rosas wrote: > >> We're about to enable the use of O_DIRECT in the migration code and > >> due to the alignment restrictions

Re: [PATCH v2 06/18] monitor: Stop removing non-duplicated fds

2024-05-31 Thread Peter Xu
On Fri, May 31, 2024 at 12:25:52PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Thu, May 23, 2024 at 04:05:36PM -0300, Fabiano Rosas wrote: > >> We've been up until now cleaning up any file descriptors that have > >> been passed into QEMU and never dupl

Re: [PATCH v2 4/4] tests/qtest/migration-test: Add a postcopy memfile test

2024-05-31 Thread Peter Xu
e don't need to care about uffd yet? As what we already do with test_postcopy(). I'm guessing the test just doesn't run on !linux, while compilation always works with/without that. Thanks, > +MigrateCommon args = { > +.start.use_shm_memfile = true, > +}; > + > +test_postcopy_common(); > +} > + > static void test_postcopy_suspend(void) > { > MigrateCommon args = { > @@ -3441,6 +3501,7 @@ int main(int argc, char **argv) > > if (has_uffd) { > migration_test_add("/migration/postcopy/plain", test_postcopy); > +migration_test_add("/migration/postcopy/memfile", > test_postcopy_memfile); > migration_test_add("/migration/postcopy/recovery/plain", > test_postcopy_recovery); > migration_test_add("/migration/postcopy/preempt/plain", > -- > 2.43.0 > -- Peter Xu

Re: [PATCH v2 13/18] monitor: fdset: Match against O_DIRECT

2024-05-30 Thread Peter Xu
by code that > needs to alternate between those flags when doing IO. > > Extend the fdset matching to behave the same with the O_DIRECT flag. > > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu One thing I am confused but totally irrelevant to this specific change. I

Re: [PATCH v2 11/18] migration/multifd: Add direct-io support

2024-05-30 Thread Peter Xu
_TYPE_FILE; > +} > + > static bool > migration_channels_and_transport_compatible(MigrationAddress *addr, > Error **errp) > @@ -180,6 +196,13 @@ > migration_channels_and_transport_compatible(MigrationAddress *addr, > return false; > } > > +if (migration_needs_extra_fds() && > +!transport_supports_extra_fds(addr)) { > +error_setg(errp, > + "Migration requires a transport that allows for extra fds > (e.g. file)"); > +return false; > +} > + > return true; > } > > -- > 2.35.3 > > -- Peter Xu

Re: [PATCH v2 10/18] migration: Add direct-io parameter

2024-05-30 Thread Peter Xu
t; clear window guaranteed to perform aligned writes. > > Acked-by: Markus Armbruster > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v2 09/18] io: Stop using qemu_open_old in channel-file

2024-05-30 Thread Peter Xu
API instead. > > Reviewed-by: Daniel P. Berrangé > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v2 08/18] monitor: Report errors from monitor_fdset_dup_fd_add

2024-05-30 Thread Peter Xu
On Thu, May 23, 2024 at 04:05:38PM -0300, Fabiano Rosas wrote: > I'm keeping the EACCES because callers expect to be able to look at > errno. > > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v2 06/18] monitor: Stop removing non-duplicated fds

2024-05-30 Thread Peter Xu
data = qmp_greeting(mon); > qmp_send_response(mon, data); > qobject_unref(data); > -mon_refcount++; > break; > case CHR_EVENT_CLOSED: > /* > @@ -479,7 +478,6 @@ static void monitor_qmp_event(void *opaque, QEMUChrEvent > event) > json_message_parser_destroy(>parser); > json_message_parser_init(>parser, handle_qmp_command, > mon, NULL); > -mon_refcount--; > monitor_fdsets_cleanup(); > break; > case CHR_EVENT_BREAK: I like this too when mon_refcount can be dropped. It turns out I like this patch and the next a lot, and I hope nothing will break. Aside, you seem to have forgot removal of the "int mon_refcount" in monitor.c. -- Peter Xu

Re: [PATCH v2 05/18] monitor: Introduce monitor_fdset_*free

2024-05-30 Thread Peter Xu
emove_fd(). > > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu One nitpick below. > --- > monitor/fds.c | 26 ++ > 1 file changed, 18 insertions(+), 8 deletions(-) > > diff --git a/monitor/fds.c b/monitor/fds.c > index fb9f58c056..101e21720d

Re: [PATCH V1 00/26] Live update: cpr-exec

2024-05-30 Thread Peter Xu
On Thu, May 30, 2024 at 01:17:05PM -0400, Steven Sistare wrote: > On 5/28/2024 12:42 PM, Peter Xu wrote: > > On Tue, May 28, 2024 at 11:10:27AM -0400, Steven Sistare wrote: > > > On 5/27/2024 1:45 PM, Peter Xu wrote: > > > > On Tue, May 21, 2024 at 07:46:12

Re: [PATCH V1 19/26] physmem: preserve ram blocks for cpr

2024-05-30 Thread Peter Xu
On Thu, May 30, 2024 at 01:12:40PM -0400, Steven Sistare wrote: > On 5/29/2024 3:25 PM, Peter Xu wrote: > > On Wed, May 29, 2024 at 01:31:53PM -0400, Steven Sistare wrote: > > > On 5/28/2024 5:44 PM, Peter Xu wrote: > > > > On Mon, Apr 29, 2024 at 08:55:2

Re: [PATCH V1 17/26] machine: memfd-alloc option

2024-05-30 Thread Peter Xu
On Thu, May 30, 2024 at 01:11:09PM -0400, Steven Sistare wrote: > On 5/29/2024 3:14 PM, Peter Xu wrote: > > On Wed, May 29, 2024 at 01:31:38PM -0400, Steven Sistare wrote: > > > > > diff --git a/system/memory.c b/system/memory.c > > > > > index 49f1cb2

Re: [PATCH V1 07/26] migration: VMStateId

2024-05-30 Thread Peter Xu
On Thu, May 30, 2024 at 01:11:26PM -0400, Steven Sistare wrote: > On 5/29/2024 2:53 PM, Peter Xu wrote: > > On Wed, May 29, 2024 at 01:30:18PM -0400, Steven Sistare wrote: > > > How about a more general name for the type: > > > > > > migration/misc.h >

Re: [PATCH v2 03/18] tests/qtest/migration: Add a precopy file test with fdset

2024-05-30 Thread Peter Xu
cted because the fdset interface is the one used by the > management layer when providing a non empty migration file. > > Note that fd passing is not available on Windows, so anything that > uses add-fd needs to exclude that platform. > > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v2 02/18] tests/qtest/migration: Fix file migration offset check

2024-05-30 Thread Peter Xu
Fix the tests to write some data to the offset region and check that > it's actually there after the migration. > > While here, switch to using g_get_file_contents() which is more > portable than mmap(). > > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v2 01/18] migration: Fix file migration with fdset

2024-05-30 Thread Peter Xu
> > Fixes: 385f510df5 ("migration: file URI offset") > Suggested-by: Daniel P. Berrangé > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu Right below the change, did we forget to free the ioc if qio_channel_io_seek() failed? > --- > migration/file.c | 11 +-

Re: [PATCH V1 19/26] physmem: preserve ram blocks for cpr

2024-05-29 Thread Peter Xu
On Wed, May 29, 2024 at 01:31:53PM -0400, Steven Sistare wrote: > On 5/28/2024 5:44 PM, Peter Xu wrote: > > On Mon, Apr 29, 2024 at 08:55:28AM -0700, Steve Sistare wrote: > > > Preserve fields of RAMBlocks that allocate their host memory during CPR so > > > the RAM

Re: [PATCH V1 17/26] machine: memfd-alloc option

2024-05-29 Thread Peter Xu
igration) maintainers will start to be soon scared and running away, if such proposal was not shot down. In short, I hope when we introduce new knobs for cpr, we shouldn't always keep cpr-* modes in mind, but consider whenever the user can use it without cpr-*. I'm not sure whether it'll be always possible, but we should try. Thanks, -- Peter Xu

Re: [PATCH V1 07/26] migration: VMStateId

2024-05-29 Thread Peter Xu
Meanwhile, s/MigrationID/IDString/? > > exec/ramblock.h > struct RAMBlock { > MigrationId idstr; > > migration/savevm.c > typedef struct CompatEntry { > MigrationId idstr; > > typedef struct SaveStateEntry { > MigrationId idstr; > > > - Steve > -- Peter Xu

Re: [PATCH V1 05/26] migration: precreate vmstate

2024-05-29 Thread Peter Xu
On Tue, May 28, 2024 at 11:09:49AM -0400, Steven Sistare wrote: > On 5/27/2024 2:16 PM, Peter Xu wrote: > > On Mon, Apr 29, 2024 at 08:55:14AM -0700, Steve Sistare wrote: > > > Provide the VMStateDescription precreate field to mark objects that must > > > be loaded

Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling

2024-05-29 Thread Peter Xu
ut I didn't further check either. I've put that issue aside just to see whether this may or may not make sense. Thanks, -- Peter Xu

Re: [PATCH] tests/qtest/migrate-test: Add a postcopy memfile test

2024-05-29 Thread Peter Xu
ithout uffd. Nicolas, please feel free to move ahead with your arch enablement series with /tmp if you want to separate the shmem issue. Thanks, -- Peter Xu

Re: [PATCH] tests/qtest/migrate-test: Use regular file file for shared-memory tests

2024-05-28 Thread Peter Xu
tially change any future use_shmem to not test shmem anymore.. instead we test something we don't suggest users to use.. The only concern is a small /dev/shm mount, am I right? Would it work if switch to memory-backend-memfd,shared=on? Thanks, -- Peter Xu

Re: [PATCH V1 19/26] physmem: preserve ram blocks for cpr

2024-05-28 Thread Peter Xu
ny kernel header to be referenced, we should only merge it together with the full iommufd support comes later (and that'll be after iommufd kernel supports land). Thanks, -- Peter Xu

Re: [PATCH V1 17/26] machine: memfd-alloc option

2024-05-28 Thread Peter Xu
oc(rb->max_length, >align, > qemu_ram_is_shared(rb), > @@ -2106,8 +2120,10 @@ RAMBlock *qemu_ram_alloc_resizeable(ram_addr_t size, > ram_addr_t maxsz, > void *host), > MemoryRegion *mr, Error **errp) > { > +uint32_t flags = current_machine->memfd_alloc ? RAM_SHARED : 0; > +flags |= RAM_RESIZEABLE; > return qemu_ram_alloc_internal(size, maxsz, resized, NULL, > - RAM_RESIZEABLE, mr, errp); > + flags, mr, errp); > } > > static void reclaim_ramblock(RAMBlock *block) > diff --git a/system/trace-events b/system/trace-events > index f0a80ba..0092734 100644 > --- a/system/trace-events > +++ b/system/trace-events > @@ -41,3 +41,4 @@ dirtylimit_vcpu_execute(int cpu_index, int64_t > sleep_time_us) "CPU[%d] sleep %"P > > # physmem.c > ram_block_create(const char *name, uint32_t flags, int fd, size_t > used_length, size_t max_length, size_t align) "%s, flags %u, fd %d, len %lu, > maxlen %lu, align %lu" > +qemu_anon_memfd_alloc(const char *name, size_t size, int fd, void *ptr) "%s > size %zu fd %d -> %p" > -- > 1.8.3.1 > -- Peter Xu

Re: [PATCH v2 2/6] tests/qtest/migration-test: Fix and enable test_ignore_shared

2024-05-28 Thread Peter Xu
igration_test_add("/migration/ignore_shared", test_ignore_shared); */ > +migration_test_add("/migration/ignore_shared", test_ignore_shared); > + > #ifndef _WIN32 > migration_test_add("/migration/precopy/fd/tcp", > test_migrate_precopy_fd_socket); > -- > 2.43.0 > -- Peter Xu

Re: [RFC PATCH 4/4] ci: Add the new migration device tests

2024-05-28 Thread Peter Xu
On Tue, May 28, 2024 at 03:10:48PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Mon, May 27, 2024 at 08:59:00PM -0300, Fabiano Rosas wrote: > >> Peter Xu writes: > >> > >> > On Thu, May 23, 2024 at 05:19:22PM -0300, Fabiano Rosas wrot

Re: [PATCH V1 08/26] migration: vmstate_info_void_ptr

2024-05-28 Thread Peter Xu
On Tue, May 28, 2024 at 11:10:16AM -0400, Steven Sistare wrote: > On 5/27/2024 2:31 PM, Peter Xu wrote: > > On Mon, Apr 29, 2024 at 08:55:17AM -0700, Steve Sistare wrote: > > > Define VMSTATE_VOID_PTR so the value of a pointer (but not its target) > > > can be sa

Re: [PATCH V1 07/26] migration: VMStateId

2024-05-28 Thread Peter Xu
On Tue, May 28, 2024 at 11:10:03AM -0400, Steven Sistare via wrote: > On 5/27/2024 2:20 PM, Peter Xu wrote: > > On Mon, Apr 29, 2024 at 08:55:16AM -0700, Steve Sistare wrote: > > > Define a type for the 256 byte id string to guarantee the same length is > > > u

Re: [PATCH V1 00/26] Live update: cpr-exec

2024-05-28 Thread Peter Xu
On Tue, May 28, 2024 at 11:10:27AM -0400, Steven Sistare wrote: > On 5/27/2024 1:45 PM, Peter Xu wrote: > > On Tue, May 21, 2024 at 07:46:12AM -0400, Steven Sistare wrote: > > > I understand, thanks. If I can help with any of your todo list, > > > just ask - steve &

Re: [PATCH] tests/qtest/migrate-test: Use regular file file for shared-memory tests

2024-05-28 Thread Peter Xu
On Tue, May 28, 2024 at 09:35:22AM -0400, Peter Xu wrote: > On Tue, May 28, 2024 at 02:27:57PM +1000, Nicholas Piggin wrote: > > There is no need to use /dev/shm for file-backed memory devices, and > > it is too small to be usable in gitlab CI. Switch to using a regular > >

Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling

2024-05-28 Thread Peter Xu
On Tue, May 28, 2024 at 09:06:04AM +, Gonglei (Arei) wrote: > Hi Peter, > > > -Original Message- > > From: Peter Xu [mailto:pet...@redhat.com] > > Sent: Wednesday, May 22, 2024 6:15 AM > > To: Yu Zhang > > Cc: Michael Galaxy ; Jinpu Wang > >

Re: [RFC PATCH 2/4] tests/qtest/migration: Add a test that runs vmstate-static-checker

2024-05-28 Thread Peter Xu
On Mon, May 27, 2024 at 07:52:28PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Thu, May 23, 2024 at 05:19:20PM -0300, Fabiano Rosas wrote: > >> We have the vmstate-static-checker script that takes the output of: > >> '$QEMU -M $machine -dump-vmstate' f

Re: [RFC PATCH 4/4] ci: Add the new migration device tests

2024-05-28 Thread Peter Xu
On Mon, May 27, 2024 at 08:59:00PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Thu, May 23, 2024 at 05:19:22PM -0300, Fabiano Rosas wrote: > >> We have two new migration tests that check cross version > >> compatibility. One uses the vmstate-static-che

Re: [RFC PATCH 3/4] tests/qtest/migration: Add support for simple device tests

2024-05-28 Thread Peter Xu
On Mon, May 27, 2024 at 07:59:50PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Thu, May 23, 2024 at 05:19:21PM -0300, Fabiano Rosas wrote: > >> The current migration-tests are almost entirely focused on catching > >> bugs on the migration code itsel

Re: [PATCH v6 2/7] migration/multifd: put IOV initialization into compression method

2024-05-28 Thread Peter Xu
On Tue, May 28, 2024 at 01:36:38PM +, Liu, Yuan1 wrote: > > -Original Message- > > From: Peter Xu > > Sent: Tuesday, May 28, 2024 4:51 AM > > To: Liu, Yuan1 > > Cc: faro...@suse.de; qemu-devel@nongnu.org; Zou, Nanhai > > > > Subject: Re:

Re: [PATCH] tests/qtest/migrate-test: Use regular file file for shared-memory tests

2024-05-28 Thread Peter Xu
gt; return 0; > @@ -1875,7 +1860,7 @@ static void test_ignore_shared(void) > g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs); > QTestState *from, *to; > MigrateStart args = { > -.use_shmem = true, > +.use_memfile = true, > }; > > if (test_migrate_start(, , uri, )) { > @@ -2033,7 +2018,7 @@ static void test_mode_reboot(void) > g_autofree char *uri = g_strdup_printf("file:%s/%s", tmpfs, > FILE_TEST_FILENAME); > MigrateCommon args = { > -.start.use_shmem = true, > +.start.use_memfile = true, > .connect_uri = uri, > .listen_uri = "defer", > .start_hook = test_mode_reboot_start > -- > 2.43.0 > -- Peter Xu

Re: [RFC PATCH 4/4] ci: Add the new migration device tests

2024-05-27 Thread Peter Xu
on core tests: Use the migration-tests from the older > +# QEMU tree. This avoids testing an old QEMU against new > +# features/tests that it is not compatible with. > +- cd ../build-previous > # old to new > - QTEST_QEMU_BINARY_SRC=./qemu-system-${TARGET} >QTEST_QEMU_BINARY=../build/qemu-system-${TARGET} > ./tests/qtest/migration-test > -- > 2.35.3 > -- Peter Xu

Re: [RFC PATCH 3/4] tests/qtest/migration: Add support for simple device tests

2024-05-27 Thread Peter Xu
ionState dst_state; > #define QEMU_ENV_SRC "QTEST_QEMU_BINARY_SRC" > #define QEMU_ENV_DST "QTEST_QEMU_BINARY_DST" > > +/* > + * The tests using DEFAULT_DEVICES need a special invocation and > + * cannot be reached from make check, so don't bother with the > + * --without-default-devices build. What's this "--without-default-devices"? Other than this it looks all good.. thanks. -- Peter Xu

Re: [RFC PATCH 2/4] tests/qtest/migration: Add a test that runs vmstate-static-checker

2024-05-27 Thread Peter Xu
XITSTATUS(wstatus) != 0) { > +g_test_message("Failed to run vmstate-static-checker.py"); > +g_test_fail(); > +} > + > +cleanup("vmstate-src"); > +cleanup("vmstate-dst"); > +} Did I ask before on whether this can be written without C? I think this and also the analyze-script are more suitable to be written in other ways, e.g., bash or python, no? > #endif > > static void test_precopy_common(MigrateCommon *args) > @@ -3495,6 +3575,8 @@ int main(int argc, char **argv) > #ifndef _WIN32 > if (!g_str_equal(arch, "s390x")) { > migration_test_add("/migration/analyze-script", test_analyze_script); > +migration_test_add("/migration/vmstate-checker-script", > + test_vmstate_checker_script); > } > #endif > migration_test_add("/migration/precopy/unix/plain", > -- > 2.35.3 > -- Peter Xu

Re: [PATCH v6 7/7] tests/migration-test: add qpl compression test

2024-05-27 Thread Peter Xu
is unavailable, it will > automatically fallback to the software path for testing. > > Signed-off-by: Yuan Liu > Reviewed-by: Nanhai Zou Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v6 2/7] migration/multifd: put IOV initialization into compression method

2024-05-27 Thread Peter Xu
; > @@ -1353,8 +1358,6 @@ static void > multifd_recv_cleanup_channel(MultiFDRecvParams *p) > p->packet_len = 0; > g_free(p->packet); > p->packet = NULL; > -g_free(p->iov); > -p->iov = NULL; > g_free(p->normal); > p->normal = NULL; > g_free(p->zero); > @@ -1602,7 +1605,6 @@ int multifd_recv_setup(Error **errp) > p->packet = g_malloc0(p->packet_len); > } > p->name = g_strdup_printf("multifdrecv_%d", i); > -p->iov = g_new0(struct iovec, page_count); > p->normal = g_new0(ram_addr_t, page_count); > p->zero = g_new0(ram_addr_t, page_count); > p->page_count = page_count; > -- > 2.39.3 > -- Peter Xu

Re: [PATCH V1 08/26] migration: vmstate_info_void_ptr

2024-05-27 Thread Peter Xu
d why. Do we need to poison that for generic VMSD use (perhaps with prefixed underscores)? I think I'll need to read on the rest to tell.. Thanks, -- Peter Xu

Re: [PATCH V1 07/26] migration: VMStateId

2024-05-27 Thread Peter Xu
QLIST_ENTRY(RAMBlock) next; > QLIST_HEAD(, RAMBlockNotifier) ramblock_notifiers; Hmm.. Don't look like a good idea to include a migration header in ramblock.h? Is this ramblock change needed for this work? -- Peter Xu

Re: [PATCH V1 05/26] migration: precreate vmstate

2024-05-27 Thread Peter Xu
(QEMUFile *f, > SaveStateEntry *se, > } > } > > +static bool send_section_footer(SaveStateEntry *se) > +{ > +return (se->vmsd && se->vmsd->precreate) || > + migrate_get_current()->send_section_footer; > +} Does the precreate vmsd "require" the footer? Or it should also work? IMHO it's less optimal to bind features without good reasons. Thanks, -- Peter Xu

Re: [PATCH V1 03/26] migration: SAVEVM_FOREACH

2024-05-27 Thread Peter Xu
ose with > SAVEVM_FOREACH_NOT_PRECREATE. I can go either way, but we need > Peter's opinion also. I don't have a strong opinion yet on the name, I think it'll be clearer indeed when the _ALL() (or whatever other name) is used only when with the real users. OTOH, besides the name (which is much more trivial..) the precreated idea in general is a bit scary to me.. if that was for a "workaround" to some new ordering issue due to newly added dependencies on exec() support. Maybe I'll understand better when I get to know better on the whole design. -- Peter Xu

Re: [PATCH V1 00/26] Live update: cpr-exec

2024-05-27 Thread Peter Xu
stmem-memfd > * cpr-exec migration test Another request besides the questions I threw already.. could you push to a tree where it has everything? Maybe that'll help to review also the min set. Thanks, -- Peter Xu

Re: [PATCH V1 04/26] migration: delete unused parameter mis

2024-05-27 Thread Peter Xu
On Mon, Apr 29, 2024 at 08:55:13AM -0700, Steve Sistare wrote: > Signed-off-by: Steve Sistare Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH V1 00/26] Live update: cpr-exec

2024-05-27 Thread Peter Xu
great to mention how you chose this way. Considering this work contains too many things, it'll be nice if such discussion can start with the fundamentals, e.g. on why exec() is a must. Thanks, -- Peter Xu

Re: [RFC PATCH 3/3] tests/qtest/migration-test: Enable test_ignore_shared

2024-05-27 Thread Peter Xu
On Mon, May 27, 2024 at 12:11:45PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Mon, May 27, 2024 at 09:42:28AM -0300, Fabiano Rosas wrote: > >> However, there is an issue here still on all archs - which might very > >> well have been the orig

Re: [RFC PATCH 3/3] tests/qtest/migration-test: Enable test_ignore_shared

2024-05-27 Thread Peter Xu
ITE) To be run at the entry of migration-test, and skip all use_shmem=true tests if ret != 0, or any step failed above. Thanks, -- Peter Xu

Re: [PATCH 5/6] migration: Rephrase message on failure to save / load Xen device state

2024-05-27 Thread Peter Xu
On Mon, May 27, 2024 at 12:53:22PM +0200, Markus Armbruster wrote: > Peter Xu writes: > > > On Mon, May 13, 2024 at 04:17:02PM +0200, Markus Armbruster wrote: > >> Functions that use an Error **errp parameter to return errors should > >> not also report them

Re: [PATCH 5/6] migration: Rephrase message on failure to save / load Xen device state

2024-05-23 Thread Peter Xu
QMP core from > > An IO error has occurred > > to > saving Xen device state failed > > and > > loading Xen device state failed > > respectively. > > Signed-off-by: Markus Armbruster Acked-by: Peter Xu -- Peter Xu

Re: [PATCH] tests/qtest/migration-test: Run some basic tests on s390x and ppc64 with TCG, too

2024-05-22 Thread Peter Xu
accident in a previous refactoring of the code). > > Signed-off-by: Thomas Huth Reviewed-by: Peter Xu Irrelevant of this patch, when I was looking at cleaning of bootfile it looks to me that we're leaking bootpath in the test loops.. maybe we need a bootfile_delete() at the entry of boot

Re: [PATCH] tests/qtest/migration-test: Fix the check for a successful run of analyze-migration.py

2024-05-22 Thread Peter Xu
nalyze-script > ... > > Let's better fail the test in case the child did not exit properly, too. > > Signed-off-by: Thomas Huth Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH] hw/core/machine: move compatibility flags for VirtIO-net USO to machine 8.1

2024-05-21 Thread Peter Xu
u stable process (I'd say that's because we have a good stable tree maintainer..). So we need this in qemu 8.1, 8.2 and 9.0? > Fixes: 53da8b5a99 ("virtio-net: Add support for USO features") > Signed-off-by: Fiona Ebner Reviewed-by: Peter Xu > --- > hw/core/machine.c | 6 +++--

Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling

2024-05-21 Thread Peter Xu
erful now, but again as I mentioned I don't think it's a reason we need to deprecate rdma, especially if QEMU's rdma migration has the chance to be refactored using rsocket. Is there anyone who started looking into that direction? Would it make sense we start some PoC now? Thanks, -- Peter Xu

Re: [PATCH V1 00/26] Live update: cpr-exec

2024-05-20 Thread Peter Xu
Conference back then pto until today, so tomorrow will be my first working day after those. Sorry Steve, will try my best to read it before next week. I didn't dare to read too much my inbox yet. A bit scared but need to face it tomorrow. On Mon, May 20, 2024, 6:28 p.m. Fabiano Rosas wrote: >

Re: [PATCH v4 0/3] Fix "virtio-gpu: fix scanout migration post-load"

2024-05-16 Thread Peter Xu
Looks good here, thanks. On Thu, May 16, 2024, 2:40 a.m. wrote: > From: Marc-André Lureau > > Hi, > > The aforementioned patch breaks virtio-gpu device migrations for versions > pre-9.0/9.0, both forwards and backwards. Versioning of `VMS_STRUCT` is > more > complex than it may initially

Re: [PATCH v3 5/5] virtio-gpu: fix v2 migration

2024-05-15 Thread Peter Xu
On Wed, May 15, 2024 at 06:15:48PM +0100, Daniel P. Berrangé wrote: > On Wed, May 15, 2024 at 11:03:27AM -0600, Peter Xu wrote: > > On Wed, May 15, 2024 at 05:03:44PM +0100, Daniel P. Berrangé wrote: > > > Above all, I'm failing to see why there's a compelling reason &g

Re: [PATCH v3 5/5] virtio-gpu: fix v2 migration

2024-05-15 Thread Peter Xu
e majority will still be that the VMSD change is caused by a new feature, and exporting that property might in most cases be wanted. In all cases, for now I agree it's at least easier to stick with the simple way. Thanks, -- Peter Xu

Re: [PATCH v3 5/5] virtio-gpu: fix v2 migration

2024-05-15 Thread Peter Xu
r saving/loading based on the machine version. > > The VMSD.version is irrelevant now. > > > > Fixes: dfcf74fa ("virtio-gpu: fix scanout migration post-load") > > Suggested-by: Peter Xu > > Signed-off-by: Marc-André Lureau > > I don't get it. Our s

Re: [PATCH v3 0/5] Fix "virtio-gpu: fix scanout migration post-load"

2024-05-15 Thread Peter Xu
ssue first, then whenever we have that new helper later we simply use the new helper to replace the old, alongside we can drop the new field / property too as long as it is declared with "x-". Might be easier to backport too in this case. Marc-Andre, what do you think? Thanks, -- Peter Xu

Re: [PATCH v2 3/4] virtio-gpu: add x-vmstate-version

2024-05-14 Thread Peter Xu
On Tue, May 14, 2024 at 11:25:26AM +0400, Marc-André Lureau wrote: > Hi > > On Tue, May 14, 2024 at 8:35 AM Peter Xu wrote: > > > > Hey, Marc-Andre, > > > > On Mon, May 13, 2024 at 11:19:04AM +0400, marcandre.lur...@redhat.com wrote: > > > diff --git a/h

Re: [PATCH v2 3/4] virtio-gpu: add x-vmstate-version

2024-05-13 Thread Peter Xu
PROP_SIZE("hostmem", VirtIOGPU, parent_obj.conf.hostmem, 0), > +DEFINE_PROP_UINT8("x-vmstate-version", VirtIOGPU, vmstate_version, 1), > DEFINE_PROP_END_OF_LIST(), > }; > > -- > 2.41.0.28.gd7d8841f67 > -- Peter Xu

Re: [PATCH v2 2/4] migration: fix a typo

2024-05-13 Thread Peter Xu
On Mon, May 13, 2024 at 11:19:03AM +0400, marcandre.lur...@redhat.com wrote: > From: Marc-André Lureau > > Signed-off-by: Marc-André Lureau Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH 3/4] virtio-gpu: use a VMState variant for the scanout field

2024-05-10 Thread Peter Xu
topic too.. so we can address the immediate breakage first. Thanks, ==8<== >From a24ef99670fa7102da461d795aed4a957bad86b1 Mon Sep 17 00:00:00 2001 From: Peter Xu Date: Fri, 10 May 2024 12:33:34 -0400 Subject: [PATCH] fix gpu Signed-off-by: Peter Xu --- include/hw/virtio/virtio-

Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling

2024-05-09 Thread Peter Xu
please check the whole thread discussion, it may help to understand what we are looking for on rdma migrations [1]. Meanwhile please feel free to sync with Jinpu's team and see how to move forward with such a project. [1] https://lore.kernel.org/qemu-devel/87frwatp7n@suse.de/ Thanks, -- Peter Xu

Re: [PATCH 3/3] migration/colo: Tidy up bql_unlock() around bdrv_activate_all()

2024-05-09 Thread Peter Xu
On Thu, May 09, 2024 at 11:31:06AM +0800, Li Zhijian via wrote: > Make the code more tight. > > Cc: Michael Tokarev > Signed-off-by: Li Zhijian Reviewed-by: Peter Xu > --- > This change/comment suggested by "Michael Tokarev " came > a bit late at that

Re: [PATCH 2/3] migration/colo: make colo_incoming_co() return void

2024-05-09 Thread Peter Xu
COLO process will > take over the remaining processes until COLO exits. > > Signed-off-by: Li Zhijian Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH 1/3] migration/colo: Minor fix for colo error message

2024-05-09 Thread Peter Xu
On Thu, May 09, 2024 at 11:31:04AM +0800, Li Zhijian wrote: > - Explicitly show the missing module name: replication > - Fix capability name to x-colo > > Signed-off-by: Li Zhijian Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v2] target/loongarch/kvm: Fix VM recovery from disk failures

2024-05-09 Thread Peter Xu
On Wed, May 08, 2024 at 10:47:32AM +0800, Song Gao wrote: > vmstate does not save kvm_state_conter, > which can cause VM recovery from disk to fail. > > Signed-off-by: Song Gao Acked-by: Peter Xu -- Peter Xu

Re: [PATCH 8/9] migration: Add support for fdset with multifd + file

2024-05-08 Thread Peter Xu
> > Related to my thoughts in an earlier patch, where I say that use of fdsets > ought to be transparent to QEMU code, I'm not a fan of having this logic > in migration code. > > IIUC, the migration code will call qio_channel_file_new_path twice, > once with O_DIRECT and once without. This should trigger two calls > into monitor_fdset_dup_fd_add with different flags. If we're matching > flags in that monitor_fdset_dup_fd_add(), then if only 1 FD was > provided, are we not able to report an error there ? Right, this sounds working. For a real sanity check, we may want to somehow check the two fds returned from qio_channel_file_new_path() to point to the same file underneath. What mentioned in the other thread (kcmp with KCMP_FILE) might not work, as the whole purpose of having two fds is to make sure they have different struct file to back the fd (and only one of them has O_DIRECT). fstat() might work in this case over the st_ino field, etc. maybe fstatfs() too but perhaps that's over cautious. Just a pain to use two fds as a start.. Thanks, -- Peter Xu

Re: [PATCH 2/9] migration: Fix file migration with fdset

2024-05-08 Thread Peter Xu
On Wed, May 08, 2024 at 09:02:16AM +0100, Daniel P. Berrangé wrote: > On Fri, May 03, 2024 at 12:23:51PM -0400, Peter Xu wrote: > > On Fri, Apr 26, 2024 at 11:20:35AM -0300, Fabiano Rosas wrote: > > > When the migration using the "file:" URI was implemented, I don't &

Re: [PATCH 0/4] Fix "virtio-gpu: fix scanout migration post-load"

2024-05-07 Thread Peter Xu
and the list of devices can start with a minumum; an extreme case is we add one device only if something broke first with a stable ABI. Then if the ABI is stable, we need to go through the deprecation procedure, and that takes two releases. We can drop the device in migration-test in the release of when deprecation starts. Thanks, -- Peter Xu

Re: [PATCH v10 0/7] Support message-based DMA in vfio-user server

2024-05-07 Thread Peter Xu
ry > and > discussion, I am leaving this to be addressed in a separate patch. I assume Jag will pick this up then. Thanks, -- Peter Xu

Re: [PATCH v10 1/7] system/physmem: Replace qemu_mutex_lock() calls with QEMU_LOCK_GUARD

2024-05-07 Thread Peter Xu
mutex_unlock calls by > the WITH_QEMU_LOCK_GUARD() macro. > > Signed-off-by: Philippe Mathieu-Daudé > Signed-off-by: Mattias Nissler > Reviewed-by: Mattias Nissler Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v10 4/7] softmmu: Support concurrent bounce buffers

2024-05-07 Thread Peter Xu
. The default limit is 4096 bytes, matching the previous > maximum buffer size. A new x-max-bounce-buffer-size parameter is > provided to configure the limit for PCI devices. > > Signed-off-by: Mattias Nissler Acked-by: Peter Xu -- Peter Xu

Re: [PATCH 3/4] virtio-gpu: use a VMState variant for the scanout field

2024-05-07 Thread Peter Xu
pp, struct virtio_gpu_scanout, 2), VMSTATE_UINT32_V(fb.width, struct virtio_gpu_scanout, 2), VMSTATE_UINT32_V(fb.height, struct virtio_gpu_scanout, 2), VMSTATE_UINT32_V(fb.stride, struct virtio_gpu_scanout, 2), VMSTATE_UINT32_V(fb.offset, struct virtio_gpu_scanout, 2), Thanks, -- Peter Xu

Re: [PATCH-for-9.1 v2 2/3] migration: Remove RDMA protocol handling

2024-05-07 Thread Peter Xu
On Tue, May 07, 2024 at 01:50:43AM +, Gonglei (Arei) wrote: > Hello, > > > -Original Message- > > From: Peter Xu [mailto:pet...@redhat.com] > > Sent: Monday, May 6, 2024 11:18 PM > > To: Gonglei (Arei) > > Cc: Daniel P. Berrangé ; Markus Armbru

Re: [PATCH RFC 00/26] Multifd  device state transfer support with VFIO consumer

2024-05-07 Thread Peter Xu
al more or less on allowing concurrent vfio migrations, so it will be greatly helpful to have your input / reviews, also making sure the ultimate solution will work for all the use cases. Thanks, -- Peter Xu

Re: [PATCH 2/3] vfio/migration: Emit VFIO device migration state change QAPI event

2024-05-07 Thread Peter Xu
as-is if maintainers are happy. Thanks, -- Peter Xu

Re: [PATCH] target/loongarch/kvm: Fix VM recovery from disk failures

2024-05-07 Thread Peter Xu
On Tue, May 07, 2024 at 04:12:34PM +0800, gaosong wrote: > Just remove CONIFG_KVM  would be OK? You're the loongarch maintainer so I'd say your call. :) If you're not yet sure, IMHO we should go with the simplest, which is the original oneliner patch. Thanks, -- Peter Xu

  1   2   3   4   5   6   7   8   9   10   >