[PATCH v2 1/4] virtiofsd: Remove test

2023-02-15 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Rmove the avocado test for virtiofsd, since we're about to remove the C implementation. Signed-off-by: Dr. David Alan Gilbert --- .../org.centos/stream/8/x86_64/test-avocado | 7 - tests/avocado/virtiofs_submounts.py | 217 -- 2 file

[PULL 0/4] virtiofs queue

2023-02-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 6dffbe36af79e26a4d23f94a9a1c1201de99c261: Merge tag 'migration-20230215-pull-request' of https://gitlab.com/juan.quintela/qemu into staging (2023-02-16 13:09:51 +) are available in the Git repository at: https://gitlab.

[PULL 4/4] virtiofsd: Swing deprecated message to removed-features

2023-02-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Move the deprecation message, since it's now gone. Signed-off-by: Dr. David Alan Gilbert Acked-by: Stefan Hajnoczi --- docs/about/deprecated.rst | 18 -- docs/about/removed-features.rst | 13 + 2 files changed, 13 insertions(+),

[PULL 1/4] virtiofsd: Remove test

2023-02-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Rmove the avocado test for virtiofsd, since we're about to remove the C implementation. Signed-off-by: Dr. David Alan Gilbert Acked-by: Stefan Hajnoczi --- .../org.centos/stream/8/x86_64/test-avocado | 7 - tests/avocado/virtiofs_submounts.py | 217

[PULL 2/4] virtiofsd: Remove build and docs glue

2023-02-16 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Remove all the virtiofsd build and docs infrastructure. Signed-off-by: Dr. David Alan Gilbert Acked-by: Stefan Hajnoczi --- MAINTAINERS| 2 -- docs/conf.py | 4 docs/meson.build

[PATCH 3/3] virtiofsd: Swing deprecated message to removed-features

2023-01-18 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Move the deprecation message, since it's now gone. Signed-off-by: Dr. David Alan Gilbert --- docs/about/deprecated.rst | 18 -- docs/about/removed-features.rst | 13 + 2 files changed, 13 insertions(+), 18 deletions(-) diff --gi

[PATCH 1/3] virtiofsd: Remove build and docs glue

2023-01-18 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Remove all the virtiofsd build and docs infrastructure. Signed-off-by: Dr. David Alan Gilbert --- MAINTAINERS| 2 -- docs/conf.py | 4 docs/meson.build | 1

[PATCH 0/3] Remove C virtiofsd

2023-01-18 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" We deprecated the C virtiofsd in commit 34deee7b6a1418f3d62a in v7.0 in favour of the Rust implementation at https://gitlab.com/virtio-fs/virtiofsd since then, the Rust version has had more development and has held up well. It's time to say goodbye to the C ver

[PATCH] MAINTAINERS: Remove and change David Gilbert maintainer entries

2023-03-30 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" I'm leaving Red Hat next week, so clean up the maintainer entries. 'virtiofs' is just the device code now, so is pretty small, and Stefan is still a maintainer there. 'migration' still has Juan. For 'HMP' I'll swing that over to my personal email. Signed-off-by:

[PATCH 1/2] Trivial: 3 char repeat typos

2022-06-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" Inspired by Julia Lawall's fixing of Linux kernel comments, I looked at qemu, although I did it manually. Signed-off-by: Dr. David Alan Gilbert --- hw/intc/openpic.c| 2 +- hw/net/imx_fec.c | 2 +- hw/pci/pcie_aer.c

[PATCH 2/2] trivial typos: namesapce

2022-06-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" 'namespace' is misspelled in a bunch of places. Signed-off-by: Dr. David Alan Gilbert --- hw/9pfs/9p-xattr-user.c | 8 hw/acpi/nvdimm.c| 2 +- hw/nvme/ctrl.c | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/hw/9pfs/

[PATCH 0/2] Two sets of trivials

2022-06-14 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" I've sent the 3 char set last month, but have updated it a little; I cleaned up a comment style that was already broken so checkpatch is happy. The 'namesapce' is a new patch; it's amazing how many places make the same typo! Dave Dr. David Alan Gilbert (2): Tri

[PULL 02/33] QIOChannelSocket: Introduce assert and reduce ifdefs to improve readability

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Leonardo Bras During implementation of MSG_ZEROCOPY feature, a lot of #ifdefs were introduced, particularly at qio_channel_socket_writev(). Rewrite some of those changes so it's easier to read. Also, introduce an assert to help detect incorrect zero-copy usage is when it's disabled on bui

[PULL 01/33] migration: Remove RDMA_UNREGISTRATION_EXAMPLE

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Juan Quintela Nobody has ever showed up to unregister individual pages, and another set of patches written by Daniel P. Berrangé just remove qemu_rdma_signal_unregister() function needed here. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan

[PULL 03/33] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Leonardo Bras Somewhere between v6 and v7 the of the zero-copy-send patchset a crucial part of the flushing mechanism got missing: incrementing zero_copy_queued. Without that, the flushing interface becomes a no-op, and there is no guarantee the buffer is really sent. This can go as bad a

[PULL 13/33] migration: introduce a QIOChannel impl for BlockDriverState VMState

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Introduce a QIOChannelBlock class that exposes the BlockDriverState VMState region for I/O. This is kept in the migration/ directory rather than io/, to avoid a mutual dependancy between block/ <-> io/ directories. Also the VMState should only be used by the migration co

[PULL 10/33] migration: rename qemu_ftell to qemu_file_total_transferred

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The name 'ftell' gives the misleading impression that the QEMUFile objects are seekable. This is not the case, as in general we just have an opaque stream. The users of this method are only interested in the total bytes processed. This switches to a new name that reflects

[PULL 04/33] migration: Change zero_copy_send from migration parameter to migration capability

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Leonardo Bras When originally implemented, zero_copy_send was designed as a Migration paramenter. But taking into account how is that supposed to work, and how the difference between a capability and a parameter, it only makes sense that zero-copy-send would work better as a capability. T

[PULL 00/33] migration queue

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 2b049d2c8dc01de750410f8f1a4eac498c04c723: Merge tag 'pull-aspeed-20220622' of https://github.com/legoater/qemu into staging (2022-06-22 07:27:06 -0700) are available in the Git repository at: https://gitlab.com/dagrh/qemu.g

[PULL 12/33] migration: rename qemu_file_update_transfer to qemu_file_acct_rate_limit

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The qemu_file_update_transfer name doesn't give a clear guide on what its purpose is, and how it differs from the qemu_file_credit_transfer method. The latter is specifically for accumulating for total migration traffic, while the former is specifically for accounting in

[PULL 05/33] io: add a QIOChannelNull equivalent to /dev/null

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This is for code which needs a portable equivalent to a QIOChannelFile connected to /dev/null. Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- include/io/channel-null.h | 55

[PULL 08/33] migration: rename rate limiting fields in QEMUFile

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This renames the following QEMUFile fields * bytes_xfer -> rate_limit_used * xfer_limit -> rate_limit_max The intent is to make it clear that 'bytes_xfer' is specifically related to rate limiting of data and applies to data queued, which need not have been transferred

[PULL 06/33] migration: switch to use QIOChannelNull for dummy channel

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This removes one further custom impl of QEMUFile, in favour of a QIOChannel based impl. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 7 +

[PULL 26/33] accel/kvm/kvm-all: Refactor per-vcpu dirty ring reaping

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Add a non-required argument 'CPUState' to kvm_dirty_ring_reap so that it can cover single vcpu dirty-ring-reaping scenario. Signed-off-by: Hyman Huang(黄勇) Reviewed-by: Peter Xu Message-Id: Signed-off-by: Dr. David Alan Gilbert --- accel/kvm/kvm-all.c | 23

[PULL 07/33] migration: remove unreachble RDMA code in save_hook impl

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The QEMUFile 'save_hook' callback has a 'size_t size' parameter. The RDMA impl of this has logic that takes different actions depending on whether the value is zero or non-zero. It has commented out logic that would have taken further actions if the value was negative.

[PULL 16/33] migration: hardcode assumption that QEMUFile is backed with QIOChannel

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The only callers of qemu_fopen_ops pass 'true' for the 'has_ioc' parameter, so hardcode this assumption in QEMUFile, by passing in the QIOChannel object as a non-opaque parameter. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Q

[PULL 14/33] migration: convert savevm to use QIOChannelBlock for VMState

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé With this change, all QEMUFile usage is backed by QIOChannel at last. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert dgilbert: Wrap long lines ---

[PULL 28/33] migration/dirtyrate: Refactor dirty page rate calculation

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) abstract out dirty log change logic into function global_dirty_log_change. abstract out dirty page rate calculation logic via dirty-ring into function vcpu_calculate_dirtyrate. abstract out mathematical dirty page rate calculation into do_calculate_dirtyrate, decouple it f

[PULL 09/33] migration: rename 'pos' field in QEMUFile to 'bytes_processed'

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The field name 'pos' gives the misleading impression that the QEMUFile objects are seekable. This is not the case, as in general we just have an opaque stream. The users of this method are only interested in the total bytes processed. This switches to a new name that refl

[PULL 11/33] migration: rename qemu_update_position to qemu_file_credit_transfer

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The qemu_update_position method name gives the misleading impression that it is changing the current file offset. Most of the files are just streams, however, so there's no concept of a file offset in the general case. What this method is actually used for is to report o

[PULL 19/33] migration: remove the QEMUFileOps 'shut_down' callback

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the shutdown logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-channel.c

[PULL 15/33] migration: stop passing 'opaque' parameter to QEMUFile hooks

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The only user of the hooks is RDMA which provides a QIOChannel backed impl of QEMUFile. It can thus use the qemu_file_get_ioc() method. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signe

[PULL 17/33] migration: introduce new constructors for QEMUFile

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Prepare for the elimination of QEMUFileOps by introducing a pair of new constructors. This lets us distinguish between an input and output file object explicitly rather than via the existance of specific callbacks. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dani

[PULL 25/33] migration: remove the QEMUFileOps abstraction

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Now that all QEMUFile callbacks are removed, the entire concept can be deleted. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/channel

[PULL 18/33] migration: remove unused QEMUFileGetFD typedef / qemu_get_fd method

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file.h | 5 - 1 file changed, 5 deletions(-) diff --git a/migration/qemu-file.

[PULL 30/33] accel/kvm/kvm-all: Introduce kvm_dirty_ring_size function

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Introduce kvm_dirty_ring_size util function to help calculate dirty ring ful time. Signed-off-by: Hyman Huang(黄勇) Acked-by: Peter Xu Message-Id: Signed-off-by: Dr. David Alan Gilbert dgilbert: Fixed up excess #endif --- accel/kvm/kvm-all.c| 5 + accel/stubs/

[PULL 24/33] migration: remove the QEMUFileOps 'get_return_path' callback

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the get_return_path logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-cha

[PULL 22/33] migration: remove the QEMUFileOps 'get_buffer' callback

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the get_buffer logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-channel.

[PULL 29/33] softmmu/dirtylimit: Implement vCPU dirtyrate calculation periodically

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Introduce the third method GLOBAL_DIRTY_LIMIT of dirty tracking for calculate dirtyrate periodly for dirty page rate limit. Add dirtylimit.c to implement dirtyrate calculation periodly, which will be used for dirty page rate limit. Add dirtylimit.h to export util functions

[PULL 20/33] migration: remove the QEMUFileOps 'set_blocking' callback

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the set_blocking logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-channe

[PULL 23/33] migration: remove the QEMUFileOps 'writev_buffer' callback

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the writev_buffer logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-chann

[PULL 31/33] softmmu/dirtylimit: Implement virtual CPU throttle

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Setup a negative feedback system when vCPU thread handling KVM_EXIT_DIRTY_RING_FULL exit by introducing throttle_us_per_full field in struct CPUState. Sleep throttle_us_per_full microseconds to throttle vCPU if dirtylimit is in service. Signed-off-by: Hyman Huang(黄勇) Revie

[PULL 21/33] migration: remove the QEMUFileOps 'close' callback

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the close logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-channel.c | 1

[PULL 32/33] softmmu/dirtylimit: Implement dirty page rate limit

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Implement dirtyrate calculation periodically basing on dirty-ring and throttle virtual CPU until it reachs the quota dirty page rate given by user. Introduce qmp commands "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit", "query-vcpu-dirty-limit" to enable, disable, query d

[PULL 33/33] tests: Add dirty page rate limit test

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Add dirty page rate limit test if kernel support dirty ring. The following qmp commands are covered by this test case: "calc-dirty-rate", "query-dirty-rate", "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit" and "query-vcpu-dirty-limit". Signed-off-by: Hyman Huang(黄勇) Ack

[PULL 27/33] cpus: Introduce cpu_list_generation_id

2022-06-22 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Introduce cpu_list_generation_id to track cpu list generation so that cpu hotplug/unplug can be detected during measurement of dirty page rate. cpu_list_generation_id could be used to detect changes of cpu list, which is prepared for dirty page rate measurement. Signed-off

[PULL 05/33] io: add a QIOChannelNull equivalent to /dev/null

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This is for code which needs a portable equivalent to a QIOChannelFile connected to /dev/null. Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- include/io/channel-null.h | 55

[PULL 00/33] migration queue

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 2b049d2c8dc01de750410f8f1a4eac498c04c723: Merge tag 'pull-aspeed-20220622' of https://github.com/legoater/qemu into staging (2022-06-22 07:27:06 -0700) are available in the Git repository at: https://gitlab.com/dagrh/qemu.g

[PULL 03/33] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Leonardo Bras Somewhere between v6 and v7 the of the zero-copy-send patchset a crucial part of the flushing mechanism got missing: incrementing zero_copy_queued. Without that, the flushing interface becomes a no-op, and there is no guarantee the buffer is really sent. This can go as bad a

[PULL 06/33] migration: switch to use QIOChannelNull for dummy channel

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This removes one further custom impl of QEMUFile, in favour of a QIOChannel based impl. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 7 +

[PULL 07/33] migration: remove unreachble RDMA code in save_hook impl

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The QEMUFile 'save_hook' callback has a 'size_t size' parameter. The RDMA impl of this has logic that takes different actions depending on whether the value is zero or non-zero. It has commented out logic that would have taken further actions if the value was negative.

[PULL 14/33] migration: convert savevm to use QIOChannelBlock for VMState

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé With this change, all QEMUFile usage is backed by QIOChannel at last. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert dgilbert: Wrap long lines ---

[PULL 10/33] migration: rename qemu_ftell to qemu_file_total_transferred

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The name 'ftell' gives the misleading impression that the QEMUFile objects are seekable. This is not the case, as in general we just have an opaque stream. The users of this method are only interested in the total bytes processed. This switches to a new name that reflects

[PULL 01/33] migration: Remove RDMA_UNREGISTRATION_EXAMPLE

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Juan Quintela Nobody has ever showed up to unregister individual pages, and another set of patches written by Daniel P. Berrangé just remove qemu_rdma_signal_unregister() function needed here. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan

[PULL 17/33] migration: introduce new constructors for QEMUFile

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Prepare for the elimination of QEMUFileOps by introducing a pair of new constructors. This lets us distinguish between an input and output file object explicitly rather than via the existance of specific callbacks. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dani

[PULL 02/33] QIOChannelSocket: Introduce assert and reduce ifdefs to improve readability

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Leonardo Bras During implementation of MSG_ZEROCOPY feature, a lot of #ifdefs were introduced, particularly at qio_channel_socket_writev(). Rewrite some of those changes so it's easier to read. Also, introduce an assert to help detect incorrect zero-copy usage is when it's disabled on bui

[PULL 15/33] migration: stop passing 'opaque' parameter to QEMUFile hooks

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The only user of the hooks is RDMA which provides a QIOChannel backed impl of QEMUFile. It can thus use the qemu_file_get_ioc() method. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signe

[PULL 04/33] migration: Change zero_copy_send from migration parameter to migration capability

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Leonardo Bras When originally implemented, zero_copy_send was designed as a Migration paramenter. But taking into account how is that supposed to work, and how the difference between a capability and a parameter, it only makes sense that zero-copy-send would work better as a capability. T

[PULL 08/33] migration: rename rate limiting fields in QEMUFile

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This renames the following QEMUFile fields * bytes_xfer -> rate_limit_used * xfer_limit -> rate_limit_max The intent is to make it clear that 'bytes_xfer' is specifically related to rate limiting of data and applies to data queued, which need not have been transferred

[PULL 18/33] migration: remove unused QEMUFileGetFD typedef / qemu_get_fd method

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file.h | 5 - 1 file changed, 5 deletions(-) diff --git a/migration/qemu-file.

[PULL 19/33] migration: remove the QEMUFileOps 'shut_down' callback

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the shutdown logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-channel.c

[PULL 20/33] migration: remove the QEMUFileOps 'set_blocking' callback

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the set_blocking logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-channe

[PULL 11/33] migration: rename qemu_update_position to qemu_file_credit_transfer

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The qemu_update_position method name gives the misleading impression that it is changing the current file offset. Most of the files are just streams, however, so there's no concept of a file offset in the general case. What this method is actually used for is to report o

[PULL 21/33] migration: remove the QEMUFileOps 'close' callback

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the close logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-channel.c | 1

[PULL 32/33] softmmu/dirtylimit: Implement dirty page rate limit

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Implement dirtyrate calculation periodically basing on dirty-ring and throttle virtual CPU until it reachs the quota dirty page rate given by user. Introduce qmp commands "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit", "query-vcpu-dirty-limit" to enable, disable, query d

[PULL 25/33] migration: remove the QEMUFileOps abstraction

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Now that all QEMUFile callbacks are removed, the entire concept can be deleted. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/channel

[PULL 13/33] migration: introduce a QIOChannel impl for BlockDriverState VMState

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Introduce a QIOChannelBlock class that exposes the BlockDriverState VMState region for I/O. This is kept in the migration/ directory rather than io/, to avoid a mutual dependancy between block/ <-> io/ directories. Also the VMState should only be used by the migration co

[PULL 23/33] migration: remove the QEMUFileOps 'writev_buffer' callback

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the writev_buffer logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-chann

[PULL 26/33] accel/kvm/kvm-all: Refactor per-vcpu dirty ring reaping

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Add a non-required argument 'CPUState' to kvm_dirty_ring_reap so that it can cover single vcpu dirty-ring-reaping scenario. Signed-off-by: Hyman Huang(黄勇) Reviewed-by: Peter Xu Message-Id: Signed-off-by: Dr. David Alan Gilbert --- accel/kvm/kvm-all.c | 23

[PULL 27/33] cpus: Introduce cpu_list_generation_id

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Introduce cpu_list_generation_id to track cpu list generation so that cpu hotplug/unplug can be detected during measurement of dirty page rate. cpu_list_generation_id could be used to detect changes of cpu list, which is prepared for dirty page rate measurement. Signed-off

[PULL 09/33] migration: rename 'pos' field in QEMUFile to 'bytes_processed'

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The field name 'pos' gives the misleading impression that the QEMUFile objects are seekable. This is not the case, as in general we just have an opaque stream. The users of this method are only interested in the total bytes processed. This switches to a new name that refl

[PULL 22/33] migration: remove the QEMUFileOps 'get_buffer' callback

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the get_buffer logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert dgilbert: Fixup len = *-*EIO as

[PULL 30/33] accel/kvm/kvm-all: Introduce kvm_dirty_ring_size function

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Introduce kvm_dirty_ring_size util function to help calculate dirty ring ful time. Signed-off-by: Hyman Huang(黄勇) Acked-by: Peter Xu Message-Id: Signed-off-by: Dr. David Alan Gilbert dgilbert: Fixed up excess #endif --- accel/kvm/kvm-all.c| 5 + accel/stubs/

[PULL 16/33] migration: hardcode assumption that QEMUFile is backed with QIOChannel

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The only callers of qemu_fopen_ops pass 'true' for the 'has_ioc' parameter, so hardcode this assumption in QEMUFile, by passing in the QIOChannel object as a non-opaque parameter. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Q

[PULL 29/33] softmmu/dirtylimit: Implement vCPU dirtyrate calculation periodically

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Introduce the third method GLOBAL_DIRTY_LIMIT of dirty tracking for calculate dirtyrate periodly for dirty page rate limit. Add dirtylimit.c to implement dirtyrate calculation periodly, which will be used for dirty page rate limit. Add dirtylimit.h to export util functions

[PULL 12/33] migration: rename qemu_file_update_transfer to qemu_file_acct_rate_limit

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The qemu_file_update_transfer name doesn't give a clear guide on what its purpose is, and how it differs from the qemu_file_credit_transfer method. The latter is specifically for accumulating for total migration traffic, while the former is specifically for accounting in

[PULL 24/33] migration: remove the QEMUFileOps 'get_return_path' callback

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the get_return_path logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-cha

[PULL 33/33] tests: Add dirty page rate limit test

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Add dirty page rate limit test if kernel support dirty ring. The following qmp commands are covered by this test case: "calc-dirty-rate", "query-dirty-rate", "set-vcpu-dirty-limit", "cancel-vcpu-dirty-limit" and "query-vcpu-dirty-limit". Signed-off-by: Hyman Huang(黄勇) Ack

[PULL 28/33] migration/dirtyrate: Refactor dirty page rate calculation

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) abstract out dirty log change logic into function global_dirty_log_change. abstract out dirty page rate calculation logic via dirty-ring into function vcpu_calculate_dirtyrate. abstract out mathematical dirty page rate calculation into do_calculate_dirtyrate, decouple it f

[PULL 31/33] softmmu/dirtylimit: Implement virtual CPU throttle

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Hyman Huang(黄勇) Setup a negative feedback system when vCPU thread handling KVM_EXIT_DIRTY_RING_FULL exit by introducing throttle_us_per_full field in struct CPUState. Sleep throttle_us_per_full microseconds to throttle vCPU if dirtylimit is in service. Signed-off-by: Hyman Huang(黄勇) Revie

[PULL 03/25] QIOChannelSocket: Fix zero-copy send so socket flush works

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Leonardo Bras Somewhere between v6 and v7 the of the zero-copy-send patchset a crucial part of the flushing mechanism got missing: incrementing zero_copy_queued. Without that, the flushing interface becomes a no-op, and there is no guarantee the buffer is really sent. This can go as bad a

[PULL 02/25] QIOChannelSocket: Introduce assert and reduce ifdefs to improve readability

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Leonardo Bras During implementation of MSG_ZEROCOPY feature, a lot of #ifdefs were introduced, particularly at qio_channel_socket_writev(). Rewrite some of those changes so it's easier to read. Also, introduce an assert to help detect incorrect zero-copy usage is when it's disabled on bui

[PULL 00/25] migration queue

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: "Dr. David Alan Gilbert" The following changes since commit 2b049d2c8dc01de750410f8f1a4eac498c04c723: Merge tag 'pull-aspeed-20220622' of https://github.com/legoater/qemu into staging (2022-06-22 07:27:06 -0700) are available in the Git repository at: https://gitlab.com/dagrh/qemu.g

[PULL 04/25] migration: Change zero_copy_send from migration parameter to migration capability

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Leonardo Bras When originally implemented, zero_copy_send was designed as a Migration paramenter. But taking into account how is that supposed to work, and how the difference between a capability and a parameter, it only makes sense that zero-copy-send would work better as a capability. T

[PULL 01/25] migration: Remove RDMA_UNREGISTRATION_EXAMPLE

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Juan Quintela Nobody has ever showed up to unregister individual pages, and another set of patches written by Daniel P. Berrangé just remove qemu_rdma_signal_unregister() function needed here. Signed-off-by: Juan Quintela Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dr. David Alan

[PULL 07/25] migration: remove unreachble RDMA code in save_hook impl

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The QEMUFile 'save_hook' callback has a 'size_t size' parameter. The RDMA impl of this has logic that takes different actions depending on whether the value is zero or non-zero. It has commented out logic that would have taken further actions if the value was negative.

[PULL 10/25] migration: rename qemu_ftell to qemu_file_total_transferred

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The name 'ftell' gives the misleading impression that the QEMUFile objects are seekable. This is not the case, as in general we just have an opaque stream. The users of this method are only interested in the total bytes processed. This switches to a new name that reflects

[PULL 06/25] migration: switch to use QIOChannelNull for dummy channel

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This removes one further custom impl of QEMUFile, in favour of a QIOChannel based impl. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/ram.c | 7 +

[PULL 05/25] io: add a QIOChannelNull equivalent to /dev/null

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This is for code which needs a portable equivalent to a QIOChannelFile connected to /dev/null. Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- include/io/channel-null.h | 55

[PULL 09/25] migration: rename 'pos' field in QEMUFile to 'bytes_processed'

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The field name 'pos' gives the misleading impression that the QEMUFile objects are seekable. This is not the case, as in general we just have an opaque stream. The users of this method are only interested in the total bytes processed. This switches to a new name that refl

[PULL 14/25] migration: convert savevm to use QIOChannelBlock for VMState

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé With this change, all QEMUFile usage is backed by QIOChannel at last. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert dgilbert: Wrap long lines ---

[PULL 08/25] migration: rename rate limiting fields in QEMUFile

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This renames the following QEMUFile fields * bytes_xfer -> rate_limit_used * xfer_limit -> rate_limit_max The intent is to make it clear that 'bytes_xfer' is specifically related to rate limiting of data and applies to data queued, which need not have been transferred

[PULL 12/25] migration: rename qemu_file_update_transfer to qemu_file_acct_rate_limit

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The qemu_file_update_transfer name doesn't give a clear guide on what its purpose is, and how it differs from the qemu_file_credit_transfer method. The latter is specifically for accumulating for total migration traffic, while the former is specifically for accounting in

[PULL 17/25] migration: introduce new constructors for QEMUFile

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Prepare for the elimination of QEMUFileOps by introducing a pair of new constructors. This lets us distinguish between an input and output file object explicitly rather than via the existance of specific callbacks. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Dani

[PULL 15/25] migration: stop passing 'opaque' parameter to QEMUFile hooks

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The only user of the hooks is RDMA which provides a QIOChannel backed impl of QEMUFile. It can thus use the qemu_file_get_ioc() method. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signe

[PULL 13/25] migration: introduce a QIOChannel impl for BlockDriverState VMState

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Introduce a QIOChannelBlock class that exposes the BlockDriverState VMState region for I/O. This is kept in the migration/ directory rather than io/, to avoid a mutual dependancy between block/ <-> io/ directories. Also the VMState should only be used by the migration co

[PULL 18/25] migration: remove unused QEMUFileGetFD typedef / qemu_get_fd method

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file.h | 5 - 1 file changed, 5 deletions(-) diff --git a/migration/qemu-file.

[PULL 16/25] migration: hardcode assumption that QEMUFile is backed with QIOChannel

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The only callers of qemu_fopen_ops pass 'true' for the 'has_ioc' parameter, so hardcode this assumption in QEMUFile, by passing in the QIOChannel object as a non-opaque parameter. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Q

[PULL 11/25] migration: rename qemu_update_position to qemu_file_credit_transfer

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé The qemu_update_position method name gives the misleading impression that it is changing the current file offset. Most of the files are just streams, however, so there's no concept of a file offset in the general case. What this method is actually used for is to report o

[PULL 24/25] migration: remove the QEMUFileOps 'get_return_path' callback

2022-06-23 Thread Dr. David Alan Gilbert (git)
From: Daniel P. Berrangé This directly implements the get_return_path logic using QIOChannel APIs. Reviewed-by: Dr. David Alan Gilbert Signed-off-by: Daniel P. Berrangé Reviewed-by: Juan Quintela Signed-off-by: Juan Quintela Signed-off-by: Dr. David Alan Gilbert --- migration/qemu-file-cha

<    8   9   10   11   12   13   14   15   16   17   >