[Qemu-devel] [PULL 08/16] qemu-option.hx: Update missed parameter for colo-compare

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Zhang Chen We missed the iothread related args in this file. This patch is used to fix this issue. Signed-off-by: Zhang Chen Message-Id: <20190426090730.2691-4-chen.zh...@intel.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- qemu-options.hx | 9 ++

[Qemu-devel] [PULL 12/16] migration/savevm: load_header before load_setup

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Wei Yang In migration_thread() and qemu_savevm_state(), we savevm_state in following sequence: qemu_savevm_state_header(f); qemu_savevm_state_setup(f); Then it would be more proper to loadvm_state in the save sequence. Signed-off-by: Wei Yang Message-Id: <20190424004700.12766-4-

[Qemu-devel] [PULL 09/16] migration/ram.c: start of migration_bitmap_sync_range is always 0

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Wei Yang We can eliminate to pass 0. Signed-off-by: Wei Yang Message-Id: <20190430034412.12935-2-richardw.y...@linux.intel.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) d

[Qemu-devel] [PULL 05/16] migration: savevm: fix error code with migration blockers

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Cole Robinson The only caller that checks the error code is looking for != 0, so returning false is incorrect. Fixes: 5aaac467938 "migration: savevm: consult migration blockers" Signed-off-by: Cole Robinson Message-Id: Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Juan Quintela Si

[Qemu-devel] [PULL 10/16] migration: update comments of migration bitmap

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Yi Wang Since the ram bitmap and the unsent bitmap are split by RAMBlock in commit 6b6712e, it's better to update the comments about them. Signed-off-by: Yi Wang Message-Id: <1555311089-18610-1-git-send-email-wang.y...@zte.com.cn> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. Da

[Qemu-devel] [PULL 15/16] migration/ram.c: fix typos in comments

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Wei Yang Signed-off-by: Wei Yang Message-Id: <20190510233729.15554-1-richardw.y...@linux.intel.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/migration/ram.

[Qemu-devel] [PULL 06/16] migration/colo.c: Remove redundant input parameter

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Zhang Chen The colo_do_failover no need the input parameter. Signed-off-by: Zhang Chen Message-Id: <20190426090730.2691-2-chen.zh...@intel.com> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert --- include/migration/colo.h | 2 +- migration/colo-failover.c | 2

[Qemu-devel] [PULL 13/16] migration/savevm: wrap into qemu_loadvm_state_header()

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Wei Yang On source side, we have qemu_savevm_state_header() to send related data, while on the receiving side those steps are scattered in qemu_loadvm_state(). This patch wrap those related steps into qemu_loadvm_state_header() to make it friendly to read. Signed-off-by: Wei Yang Message

[Qemu-devel] [PULL 11/16] migration/savevm: remove duplicate check of migration_is_blocked

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Wei Yang Current call flow of save_snapshot is: save_snapshot migration_is_blocked qemu_savevm_state migration_is_blocked Since qemu_savevm_state is only called in save_snapshot, this means migration_is_blocked has been already checked. Signed-off-by: Wei Yang Messag

[Qemu-devel] [PULL 16/16] monitor: Call mon_get_cpu() only once at hmp_gva2gpa()

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Eduardo Habkost hmp_gva2gpa() calls mon_get_cpu() twice, which is unnecessary. Not an actual bug, but this is reported as a defect by Coverity Scan (CID 1401346). Signed-off-by: Eduardo Habkost Message-Id: <20190510185620.15757-1-ehabk...@redhat.com> Reviewed-by: Richard Henderson Review

[Qemu-devel] [PULL 14/16] migration: Fix use-after-free during process exit

2019-05-14 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov It fixes heap-use-after-free which was found by clang's ASAN. Control flow of this use-after-free: main_thread: * Got SIGTERM and completes main loop * Calls migration_shutdown - migrate_fd_cancel (so, migration_thread begins to complete) - object_unref(OBJEC

[Qemu-devel] [PULL 00/22] migration queue

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit b5b6b2b912bbcd3953407da938a8f969577ad3a1: Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-mar-05-2019' into staging (2019-03-05 21:07:29 +) are available in the Git repository at: git://github.com/dagrh/

[Qemu-devel] [PULL 01/22] migration: Fix cancel state

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" During a cancelled migration there's a race where the fd can go into an error state before we get back around the migration loop and migration_detect_error transitions from cancelling->failed. Check for cancelled/cancelling and don't change the state. Red Hat bug:

[Qemu-devel] [PULL 02/22] migration/rdma: Fix qemu_rdma_cleanup null check

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" If the migration fails before the channel is open (e.g. a bad address) we end up in the cleanup with rdma->channel==NULL. Spotted by Coverity: CID 1398634 Fixes: fbbaacab2758cb3f32a0 Signed-off-by: Dr. David Alan Gilbert Message-Id: <20190214185351.5927-1-dgilb...

[Qemu-devel] [PULL 15/22] migration: use bitmap_mutex in migration_bitmap_clear_dirty

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Wei Wang The bitmap mutex is used to synchronize threads to update the dirty bitmap and the migration_dirty_pages counter. For example, the free page optimization clears bits of free pages from the bitmap in an iothread context. This patch makes migration_bitmap_clear_dirty update the bitma

[Qemu-devel] [PULL 03/22] migration: Cleanup during exit

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Currently we cleanup the migration object as we exit main after the main_loop finishes; however if there's a migration running things get messy and we can end up with the migration thread still trying to access freed structures. We now take a ref to the object arou

[Qemu-devel] [PULL 13/22] bitmap: fix bitmap_count_one

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Wei Wang BITMAP_LAST_WORD_MASK(nbits) returns 0x when "nbits=0", which makes bitmap_count_one fail to handle the "nbits=0" case. It appears to be preferred to remain BITMAP_LAST_WORD_MASK identical to the kernel implementation that it is ported from. So this patch fixes bitmap_coun

[Qemu-devel] [PULL 04/22] migration/rdma: clang compilation fix

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Marcel Apfelbaum Configuring QEMU with: ../configure --cc=clang --enable-rdma Leads to compilation error: CC migration/rdma.o CC migration/block.o qemu/migration/rdma.c:3615:58: error: taking address of packed member 'rkey' of class or structure 'RDMARegiste

[Qemu-devel] [PULL 18/22] migration/ram.c: add the free page optimization enable flag

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Wei Wang This patch adds the free page optimization enable flag, and a function to set this flag. When the free page optimization is enabled, not all the pages are needed to be sent in the bulk stage. Why using a new flag, instead of directly disabling ram_bulk_stage when the optimization

[Qemu-devel] [PULL 11/22] migration: Create socket-address parameter

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Juan Quintela It will be used to store the uri parameters. We want this only for tcp, so we don't set it for other uris. We need it to know what port is migration running. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert dgilbert:

[Qemu-devel] [PULL 16/22] migration: API to clear bits of guest free pages from the dirty bitmap

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Wei Wang This patch adds an API to clear bits corresponding to guest free pages from the dirty bitmap. Spilt the free page block if it crosses the QEMU RAMBlock boundary. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu Reviewed-

[Qemu-devel] [PULL 05/22] exec: Change RAMBlockIterFunc definition

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov Currently, qemu_ram_foreach_* calls RAMBlockIterFunc with many block-specific arguments. But often iter func needs RAMBlock*. This refactoring is needed for fast access to RAMBlock flags from qemu_ram_foreach_block's callback. The only way to achieve this now is to call qemu_ram_

[Qemu-devel] [PULL 21/22] Migration/colo.c: Make COLO node running after failover

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Zhang Chen Delay to close COLO for auto start VM after failover. Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert Message-Id: <20190303145021.2962-4-chen.zh...@intel.com> Signed-off-by: Dr. David Alan Gilbert --- migration/colo.c | 1 - migration/migration.c | 3 +++

[Qemu-devel] [PULL 06/22] migration: Introduce ignore-shared capability

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov We want to use local migration to update QEMU for running guests. In this case we don't need to migrate shared (file backed) RAM. So, add a capability to ignore such blocks during live migration. Signed-off-by: Yury Kotov Message-Id: <20190215174548.2630-3-yury-ko...@yandex-tea

[Qemu-devel] [PULL 14/22] bitmap: bitmap_count_one_with_offset

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Wei Wang Count the number of 1s in a bitmap starting from an offset. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin Reviewed-by: Dr. David Alan Gilbert Message-Id: <1544516693-5395-3-git-send-email-wei.w.w...@intel.com> Signed-off-by: Dr. D

[Qemu-devel] [PULL 12/22] tests: Add basic migration precopy tcp test

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Juan Quintela Not sharing code from precopy/unix because we have to read back the tcp parameter. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu Reviewed-by: Thomas Huth Message-Id: <20190227105128.1655-4-quint...@redhat.com> Signed-off-by: Dr. D

[Qemu-devel] [PULL 07/22] migration: Add an ability to ignore shared RAM blocks

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov If ignore-shared capability is set then skip shared RAMBlocks during the RAM migration. Also, move qemu_ram_foreach_migratable_block (and rename) to the migration code, because it requires access to the migration capabilities. Signed-off-by: Yury Kotov Message-Id: <201902151745

[Qemu-devel] [PULL 19/22] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Wei Wang The new feature enables the virtio-balloon device to receive hints of guest free pages from the free page vq. A notifier is registered to the migration precopy notifier chain. The notifier calls free_page_start after the migration thread syncs the dirty bitmap, so that the free pa

[Qemu-devel] [PULL 17/22] migration/ram.c: add a notifier chain for precopy

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Wei Wang This patch adds a notifier chain for the memory precopy. This enables various precopy optimizations to be invoked at specific places. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu Reviewed-by: Peter Xu Message-Id: <1

[Qemu-devel] [PULL 08/22] tests/migration-test: Add a test for ignore-shared capability

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov Signed-off-by: Yury Kotov Message-Id: <20190215174548.2630-5-yury-ko...@yandex-team.ru> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert dgilbert: Disabled the test for now, not happy on aarch64 --- tests/migration-test.c | 134 +

[Qemu-devel] [PULL 20/22] Migration/colo.c: Fix double close bug when occur COLO failover

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Zhang Chen In migration_incoming_state_destroy(void) will check the mis->to_src_file to double close the mis->to_src_file when occur COLO failover. Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert Message-Id: <20190303145021.2962-2-chen.zh...@intel.com> Signed-off-by: Dr. Da

[Qemu-devel] [PULL 09/22] migration: Add capabilities validation

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov Currently we don't check which capabilities set in the source QEMU. We just expect that the target QEMU has the same enabled capabilities. Add explicit validation for capabilities to make sure that the target VM has them too. This is enabled for only new capabilities to keep com

[Qemu-devel] [PULL 22/22] qapi/migration.json: Remove a variable that doesn't exist in example

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Zhang Chen Remove the "active" variable in example for query-colo-status. It is a doc bug from commit f56c0065 Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Message-Id: <20190303145021.2962-6-chen.zh...@intel.com> Signed-off-by: Dr. David Alan Gilbert --- qapi/migration.json | 2 +-

[Qemu-devel] [PULL 10/22] tests: Add migration xbzrle test

2019-03-06 Thread Dr. David Alan Gilbert (git)
From: Juan Quintela Reviewed-by: Peter Xu Signed-off-by: Juan Quintela Message-Id: <20190227105128.1655-2-quint...@redhat.com> Signed-off-by: Dr. David Alan Gilbert dgilbert: Fixup for class with Yury's series --- tests/migration-test.c | 64 ++ 1 fil

[Qemu-devel] [PATCH] migration/postcopy: Update the bandwidth during postcopy

2019-03-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The recently added max-postcopy-bandwidth parameter is only read at the transition from precopy->postcopy where as the older max-bandwidth parameter updates the migration bandwidth when changed even if the migration is already running. Fix this discrepency so that:

[Qemu-devel] [PATCH] migration/rdma: Check qemu_rdma_init_one_block

2019-03-08 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Actually it can't fail at the moment, but Coverity moans that it's the only place it's not checked, and it's an easy check. Reported-by: Coverity (CID 1399413) Signed-off-by: Dr. David Alan Gilbert --- migration/rdma.c | 6 +- 1 file changed, 5 insertions(+),

[Qemu-devel] [PATCH v2 1/2] net/announce: Allow optional list of interfaces

2019-05-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Allow the caller to restrict the set of interfaces that announces are sent on. The default is still to send on all interfaces. e.g. { "execute": "announce-self", "arguments": { "initial": 50, "max": 550, "rounds": 5, "step": 50, "interfaces": ["vn2","vn1"] } }

[Qemu-devel] [PATCH v2 0/2] network announce; interface selection

2019-05-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Laine asked for some extra features on the network announce support; this is the first one of them. It allows you to send an announce on a subset of the interfaces. Note since we've still only got one timer, if you start one announce on an interface and then you s

[Qemu-devel] [PATCH v2 2/2] net/announce: Add HMP optional interface list

2019-05-29 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add the optional interface list to the HMP command. i.e. All interfaces announce_self Just the named interfaces: announce_self vn1,vn2 Signed-off-by: Dr. David Alan Gilbert --- hmp-commands.hx | 6 -- hmp.c | 38 +++

[Qemu-devel] [PULL 0/2] migration queue

2018-11-21 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 47c1cc30e440860aa695358f7c2dd0b9d7b53d16: Update version for v3.1.0-rc2 release (2018-11-20 18:10:26 +) are available in the Git repository at: git://github.com/dagrh/qemu.git tags/pull-migration-20181121a for you to fe

[Qemu-devel] [PULL 1/2] migration/colo.c: Fix compilation issue when disable replication

2018-11-21 Thread Dr. David Alan Gilbert (git)
From: Zhang Chen This compilation issue will occur when user use --disable-replication to config Qemu. Reported-by: Thomas Huth Signed-off-by: Zhang Chen Message-Id: <20181101021226.6353-1-zhangc...@gmail.com> Tested-by: Thomas Huth Signed-off-by: Dr. David Alan Gilbert --- migration/colo.c

[Qemu-devel] [PULL 2/2] migration/migration.c: Add COLO dependency checks

2018-11-21 Thread Dr. David Alan Gilbert (git)
From: Zhang Chen Current COLO mode(independent disk mode) need replication module work together. Suggested by Dr. David Alan Gilbert . Signed-off-by: Zhang Chen Message-Id: <20181114190912.7242-1-chen.zh...@intel.com> Reviewed-by: Peter Xu Signed-off-by: Dr. David Alan Gilbert --- migration/

[Qemu-devel] [PATCH v5 0/9] Network announce changes

2019-02-26 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Firstly, it's parameterised, so that you can change the number of packets and the gap between them; the number can be set to 0 to disable announce completely. Secondly, you can force an announce by a qmp or hmp command at any time. This is useful if you need

[Qemu-devel] [PATCH v5 4/9] migration: Switch to using announce timer

2019-02-26 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Switch the announcements to using the new announce timer. Move the code that does it to announce.c rather than savevm because it really has nothing to do with the actual migration. Migration starts the announce from bh's and so they're all in the main thread/bql, a

[Qemu-devel] [PATCH v5 8/9] hmp: Add hmp_announce_self

2019-02-26 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add an HMP command to trigger self annocements. Unlike the QMP command (which takes a set of parameters), the HMP command reuses the set of parameters used for migration. Signend-off-by: Vladislav Yasevich Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michae

[Qemu-devel] [PATCH v5 1/9] net: Introduce announce timer

2019-02-26 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The 'announce timer' will be used by migration, and explicit requests for qemu to perform network announces. Based on the work by Germano Veit Michel and Vlad Yasevich Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michael S. Tsirkin --- include/net/anno

[Qemu-devel] [PATCH v5 3/9] virtio-net: Switch to using announce timer

2019-02-26 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Switch virtio's self announcement to use the AnnounceTimer. It keeps it's own AnnounceTimer (per device), and starts running it using a migration post-load and a virtual clock; that way the announce happens once the guest is actually running. The timer uses the migr

[Qemu-devel] [PATCH v5 5/9] net: Add a network device specific self-announcement ability

2019-02-26 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Some network devices have a capability to do self announcements (ex: virtio-net). Add infrastructure that would allow devices to expose this ability. Signed-off-by: Vladislav Yasevich Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michael S. Tsirkin --- in

[Qemu-devel] [PATCH v5 9/9] tests: Add a test for qemu self announcements

2019-02-26 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" We now expose qemu_announce_self through QMP and HMP. Add a test with some very basic packet validation (make sure we get a RARP). Signed-off-by: Vlad Yasevich Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michael S. Tsirkin --- tests/Makefile.include

[Qemu-devel] [PATCH v5 6/9] virtio-net: Allow qemu_announce_self to trigger virtio announcements

2019-02-26 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Expose the virtio-net self announcement capability and allow qemu_announce_self() to call it. These announces are caused by something external (i.e. the announce-self command); they won't trigger if the migration counter is triggering announces at the same time. S

[Qemu-devel] [PATCH v5 2/9] migration: Add announce parameters

2019-02-26 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add migration parameters that control RARP/GARP announcement timeouts. Based on earlier patches by myself and Vladislav Yasevich Signed-off-by: Dr. David Alan Gilbert Acked-by: Markus Armbruster Reviewed-by: Michael S. Tsirkin --- hmp.c|

[Qemu-devel] [PATCH v5 7/9] qmp: Add announce-self command

2019-02-26 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add a qmp command that can trigger guest announcements. It uses its own announce-timer instance, and parameters passed to it explicitly in the command. Like most qmp commands, it's in the main thread/bql, so there's no racing with any outstanding timer. Based on

[Qemu-devel] [PATCH v6 8/9] hmp: Add hmp_announce_self

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add an HMP command to trigger self annocements. Unlike the QMP command (which takes a set of parameters), the HMP command reuses the set of parameters used for migration. Signend-off-by: Vladislav Yasevich Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michae

[Qemu-devel] [PATCH v6 2/9] migration: Add announce parameters

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add migration parameters that control RARP/GARP announcement timeouts. Based on earlier patches by myself and Vladislav Yasevich Signed-off-by: Dr. David Alan Gilbert Acked-by: Markus Armbruster Reviewed-by: Michael S. Tsirkin --- hmp.c|

[Qemu-devel] [PATCH v6 1/9] net: Introduce announce timer

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The 'announce timer' will be used by migration, and explicit requests for qemu to perform network announces. Based on the work by Germano Veit Michel and Vlad Yasevich Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michael S. Tsirkin --- include/net/anno

[Qemu-devel] [PATCH v6 7/9] qmp: Add announce-self command

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add a qmp command that can trigger guest announcements. It uses its own announce-timer instance, and parameters passed to it explicitly in the command. Like most qmp commands, it's in the main thread/bql, so there's no racing with any outstanding timer. Based on

[Qemu-devel] [PATCH v6 0/9] Network announce changes

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Hi, This is a reworking of a few sets of patches from 2017 that were put together by myself, Germano and Vlad that make the network announce system more flexible. Firstly, it's parameterised, so that you can change the number of packets and the gap between them

[Qemu-devel] [PATCH v6 5/9] net: Add a network device specific self-announcement ability

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Some network devices have a capability to do self announcements (ex: virtio-net). Add infrastructure that would allow devices to expose this ability. Signed-off-by: Vladislav Yasevich Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michael S. Tsirkin --- in

[Qemu-devel] [PATCH v6 3/9] virtio-net: Switch to using announce timer

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Switch virtio's self announcement to use the AnnounceTimer. It keeps it's own AnnounceTimer (per device), and starts running it using a migration post-load and a virtual clock; that way the announce happens once the guest is actually running. The timer uses the migr

[Qemu-devel] [PATCH v6 6/9] virtio-net: Allow qemu_announce_self to trigger virtio announcements

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Expose the virtio-net self announcement capability and allow qemu_announce_self() to call it. These announces are caused by something external (i.e. the announce-self command); they won't trigger if the migration counter is triggering announces at the same time. S

[Qemu-devel] [PATCH v6 9/9] tests: Add a test for qemu self announcements

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" We now expose qemu_announce_self through QMP and HMP. Add a test with some very basic packet validation (make sure we get a RARP). Signed-off-by: Vlad Yasevich Signed-off-by: Dr. David Alan Gilbert Reviewed-by: Michael S. Tsirkin --- tests/Makefile.include

[Qemu-devel] [PATCH v6 4/9] migration: Switch to using announce timer

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Switch the announcements to using the new announce timer. Move the code that does it to announce.c rather than savevm because it really has nothing to do with the actual migration. Migration starts the announce from bh's and so they're all in the main thread/bql, a

[Qemu-devel] [PATCH] migration: Cleanup during exit

2019-02-27 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Currently we cleanup the migration object as we exit main after the main_loop finishes; however if there's a migration running things get messy and we can end up with the migration thread still trying to access freed structures. We now take a ref to the object arou

[Qemu-devel] [PULL 01/21] migration: Fix cancel state

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" During a cancelled migration there's a race where the fd can go into an error state before we get back around the migration loop and migration_detect_error transitions from cancelling->failed. Check for cancelled/cancelling and don't change the state. Red Hat bug:

[Qemu-devel] [PULL 03/21] migration/rdma: clang compilation fix

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Marcel Apfelbaum Configuring QEMU with: ../configure --cc=clang --enable-rdma Leads to compilation error: CC migration/rdma.o CC migration/block.o qemu/migration/rdma.c:3615:58: error: taking address of packed member 'rkey' of class or structure 'RDMARegiste

[Qemu-devel] [PULL 00/21] migration queue

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 0984a157c1c053394adbf64ed7de97f1aebe6a2d: Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging (2019-03-05 09:33:20 +) are available in the Git repository at: git://github.com/dagrh/qemu.gi

[Qemu-devel] [PULL 07/21] tests/migration-test: Add a test for ignore-shared capability

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov Signed-off-by: Yury Kotov Message-Id: <20190215174548.2630-5-yury-ko...@yandex-team.ru> Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan Gilbert dgilbert: Disabled the test for now, not happy on aarch64 --- tests/migration-test.c | 134 +

[Qemu-devel] [PULL 04/21] exec: Change RAMBlockIterFunc definition

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov Currently, qemu_ram_foreach_* calls RAMBlockIterFunc with many block-specific arguments. But often iter func needs RAMBlock*. This refactoring is needed for fast access to RAMBlock flags from qemu_ram_foreach_block's callback. The only way to achieve this now is to call qemu_ram_

[Qemu-devel] [PULL 12/21] bitmap: fix bitmap_count_one

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Wei Wang BITMAP_LAST_WORD_MASK(nbits) returns 0x when "nbits=0", which makes bitmap_count_one fail to handle the "nbits=0" case. It appears to be preferred to remain BITMAP_LAST_WORD_MASK identical to the kernel implementation that it is ported from. So this patch fixes bitmap_coun

[Qemu-devel] [PULL 14/21] migration: use bitmap_mutex in migration_bitmap_clear_dirty

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Wei Wang The bitmap mutex is used to synchronize threads to update the dirty bitmap and the migration_dirty_pages counter. For example, the free page optimization clears bits of free pages from the bitmap in an iothread context. This patch makes migration_bitmap_clear_dirty update the bitma

[Qemu-devel] [PULL 08/21] migration: Add capabilities validation

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov Currently we don't check which capabilities set in the source QEMU. We just expect that the target QEMU has the same enabled capabilities. Add explicit validation for capabilities to make sure that the target VM has them too. This is enabled for only new capabilities to keep com

[Qemu-devel] [PULL 15/21] migration: API to clear bits of guest free pages from the dirty bitmap

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Wei Wang This patch adds an API to clear bits corresponding to guest free pages from the dirty bitmap. Spilt the free page block if it crosses the QEMU RAMBlock boundary. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu Reviewed-

[Qemu-devel] [PULL 11/21] tests: Add basic migration precopy tcp test

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Juan Quintela Not sharing code from precopy/unix because we have to read back the tcp parameter. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Peter Xu Reviewed-by: Thomas Huth Message-Id: <20190227105128.1655-4-quint...@redhat.com> Signed-off-by: Dr. D

[Qemu-devel] [PULL 21/21] qapi/migration.json: Remove a variable that doesn't exist in example

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Zhang Chen Remove the "active" variable in example for query-colo-status. It is a doc bug from commit f56c0065 Signed-off-by: Zhang Chen Reviewed-by: Eric Blake Message-Id: <20190303145021.2962-6-chen.zh...@intel.com> Signed-off-by: Dr. David Alan Gilbert --- qapi/migration.json | 2 +-

[Qemu-devel] [PULL 09/21] tests: Add migration xbzrle test

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Juan Quintela Reviewed-by: Peter Xu Signed-off-by: Juan Quintela Message-Id: <20190227105128.1655-2-quint...@redhat.com> Signed-off-by: Dr. David Alan Gilbert dgilbert: Fixup for class with Yury's series --- tests/migration-test.c | 64 ++ 1 fil

[Qemu-devel] [PULL 02/21] migration/rdma: Fix qemu_rdma_cleanup null check

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" If the migration fails before the channel is open (e.g. a bad address) we end up in the cleanup with rdma->channel==NULL. Spotted by Coverity: CID 1398634 Fixes: fbbaacab2758cb3f32a0 Signed-off-by: Dr. David Alan Gilbert Message-Id: <20190214185351.5927-1-dgilb...

[Qemu-devel] [PULL 10/21] migration: Create socket-address parameter

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Juan Quintela It will be used to store the uri parameters. We want this only for tcp, so we don't set it for other uris. We need it to know what port is migration running. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Juan Quintela -- This used to be uri parameter, but it has so

[Qemu-devel] [PULL 05/21] migration: Introduce ignore-shared capability

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov We want to use local migration to update QEMU for running guests. In this case we don't need to migrate shared (file backed) RAM. So, add a capability to ignore such blocks during live migration. Signed-off-by: Yury Kotov Message-Id: <20190215174548.2630-3-yury-ko...@yandex-tea

[Qemu-devel] [PULL 17/21] migration/ram.c: add the free page optimization enable flag

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Wei Wang This patch adds the free page optimization enable flag, and a function to set this flag. When the free page optimization is enabled, not all the pages are needed to be sent in the bulk stage. Why using a new flag, instead of directly disabling ram_bulk_stage when the optimization

[Qemu-devel] [PULL 06/21] migration: Add an ability to ignore shared RAM blocks

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Yury Kotov If ignore-shared capability is set then skip shared RAMBlocks during the RAM migration. Also, move qemu_ram_foreach_migratable_block (and rename) to the migration code, because it requires access to the migration capabilities. Signed-off-by: Yury Kotov Message-Id: <201902151745

[Qemu-devel] [PULL 18/21] virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Wei Wang The new feature enables the virtio-balloon device to receive hints of guest free pages from the free page vq. A notifier is registered to the migration precopy notifier chain. The notifier calls free_page_start after the migration thread syncs the dirty bitmap, so that the free pa

[Qemu-devel] [PULL 13/21] bitmap: bitmap_count_one_with_offset

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Wei Wang Count the number of 1s in a bitmap starting from an offset. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin Reviewed-by: Dr. David Alan Gilbert Message-Id: <1544516693-5395-3-git-send-email-wei.w.w...@intel.com> Signed-off-by: Dr. D

[Qemu-devel] [PULL 20/21] Migration/colo.c: Make COLO node running after failover

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Zhang Chen Delay to close COLO for auto start VM after failover. Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert Message-Id: <20190303145021.2962-4-chen.zh...@intel.com> Signed-off-by: Dr. David Alan Gilbert --- migration/colo.c | 1 - migration/migration.c | 3 +++

[Qemu-devel] [PULL 16/21] migration/ram.c: add a notifier chain for precopy

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Wei Wang This patch adds a notifier chain for the memory precopy. This enables various precopy optimizations to be invoked at specific places. Signed-off-by: Wei Wang CC: Dr. David Alan Gilbert CC: Juan Quintela CC: Michael S. Tsirkin CC: Peter Xu Reviewed-by: Peter Xu Message-Id: <1

[Qemu-devel] [PULL 19/21] Migration/colo.c: Fix double close bug when occur COLO failover

2019-03-05 Thread Dr. David Alan Gilbert (git)
From: Zhang Chen In migration_incoming_state_destroy(void) will check the mis->to_src_file to double close the mis->to_src_file when occur COLO failover. Signed-off-by: Zhang Chen Reviewed-by: Dr. David Alan Gilbert Message-Id: <20190303145021.2962-2-chen.zh...@intel.com> Signed-off-by: Dr. Da

[Qemu-devel] [PATCH v2] slirp: Gcc 9 -O3 fix

2019-04-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Gcc 9 needs some convincing that sopreprbuf really is going to fill in iov in the call from soreadbuf, even though the failure case shouldn't happen. Signed-off-by: Dr. David Alan Gilbert --- slirp/src/socket.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion

[Qemu-devel] [PATCH v3] slirp: Gcc 9 -O3 fix

2019-04-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Gcc 9 needs some convincing that sopreprbuf really is going to fill in iov in the call from soreadbuf, even though the failure case shouldn't happen. Signed-off-by: Dr. David Alan Gilbert --- slirp/src/socket.c | 5 - 1 file changed, 4 insertions(+), 1 deleti

[Qemu-devel] [PULL 2/2] hmp: gva2gpa debug command

2019-05-01 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Add a gva2gpa command purely for debug which performs address translation on the gva, the existing gpa2hva command can then also be used to find it in the qemu userspace; e.g. (qemu) info registers RSP=81c03e98 (qemu) gva2gpa 0x81c03e98 gp

[Qemu-devel] [PULL 0/2] hmp queue

2019-05-01 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit f75d15231e56cb0f2bafe19faf1229c459a60731: Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging (2019-04-30 17:06:57 +0100) are available in the Git repository at: git://github.com/dagrh/qemu.git tags/

[Qemu-devel] [PULL 1/2] hmp: delvm: use hmp_handle_error

2019-05-01 Thread Dr. David Alan Gilbert (git)
From: Cole Robinson This gives us the consistent 'Error:' prefix added in 66363e9a43f, which helps users like libvirt who still need to scrape hmp error messages to detect failure. Reviewed-by: Eric Blake Signed-off-by: Cole Robinson Message-Id: <984634226fa14316641f31f84e7dd0bb770bd94c.15561

[PULL 0/5] migration queue

2022-08-02 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 0399521e53336bd2cdc15482bca0ffd3493fdff6: Merge tag 'for-upstream' of git://repo.or.cz/qemu/kevin into staging (2022-08-02 06:52:05 -0700) are available in the Git repository at: https://gitlab.com/dagrh/qemu.git tags/pull-

[PULL 3/5] migration: Assert that migrate_multifd_compression() returns an in-range value

2022-08-02 Thread Dr. David Alan Gilbert (git)
From: Peter Maydell Coverity complains that when we use the return value from migrate_multifd_compression() as an array index: multifd_recv_state->ops = multifd_ops[migrate_multifd_compression()]; that this might overrun the array (which is declared to have size MULTIFD_COMPRESSION__MAX). Thi

[PULL 2/5] Revert "migration: Simplify unqueue_page()"

2022-08-02 Thread Dr. David Alan Gilbert (git)
From: Thomas Huth This reverts commit cfd66f30fb0f735df06ff4220e5000290a43dad3. The simplification of unqueue_page() introduced a bug that sometimes breaks migration on s390x hosts. The problem is not fully understood yet, but since we are already in the freeze for QEMU 7.1 and we need somethin

[PULL 1/5] migration: add remaining params->has_* = true in migration_instance_init()

2022-08-02 Thread Dr. David Alan Gilbert (git)
From: Leonardo Bras Some of params->has_* = true are missing in migration_instance_init, this causes migrate_params_check() to skip some tests, allowing some unsupported scenarios. Fix this by adding all missing params->has_* = true in migration_instance_init(). Fixes: 69ef1f36b0 ("migration: d

[PULL 4/5] migration: Define BLK_MIG_BLOCK_SIZE as unsigned long long

2022-08-02 Thread Dr. David Alan Gilbert (git)
From: Peter Maydell When we use BLK_MIG_BLOCK_SIZE in expressions like block_mig_state.submitted * BLK_MIG_BLOCK_SIZE, this multiplication is done as 32 bits, because both operands are 32 bits. Coverity complains about possible overflows because we then accumulate that into a 64 bit variable. D

[PULL 5/5] virtiofsd: Disable killpriv_v2 by default

2022-08-02 Thread Dr. David Alan Gilbert (git)
From: Vivek Goyal We are having bunch of issues with killpriv_v2 enabled by default. First of all it relies on clearing suid/sgid bits as needed by dropping capability CAP_FSETID. This does not work for remote filesystems like NFS (and possibly others). Secondly, we are noticing other issues rel

[PATCH] virtio-rng-pci: fix transitional migration compat for vectors

2023-02-07 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" In bad9c5a5166fd5e3a892b7b0477cf2f4bd3a959a I fixed the virito-rng-pci migration compatibility, but it was discovered that we also need to fix the other aliases of the device for the transitional cases. Fixes: 9ea02e8f1 ('virtio-rng-pci: Allow setting nvectors, so

[PATCH v2 4/4] virtiofsd: Swing deprecated message to removed-features

2023-02-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Move the deprecation message, since it's now gone. Signed-off-by: Dr. David Alan Gilbert --- docs/about/deprecated.rst | 18 -- docs/about/removed-features.rst | 13 + 2 files changed, 13 insertions(+), 18 deletions(-) diff --gi

[PATCH v2 2/4] virtiofsd: Remove build and docs glue

2023-02-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Remove all the virtiofsd build and docs infrastructure. Signed-off-by: Dr. David Alan Gilbert --- MAINTAINERS| 2 -- docs/conf.py | 4 docs/meson.build | 1

[PATCH v2 0/4] Remove C virtiofsd

2023-02-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" We deprecated the C virtiofsd in commit 34deee7b6a1418f3d62a in v7.0 in favour of the Rust implementation at https://gitlab.com/virtio-fs/virtiofsd since then, the Rust version has had more development and has held up well. It's time to say goodbye to the C v

<    7   8   9   10   11   12   13   14   15   16   >