Re: [PATCH v3 3/5] mirror: allow specifying working bitmap

2024-05-12 Thread Markus Armbruster
Fiona Ebner writes: > From: John Snow > > for the mirror job. The bitmap's granularity is used as the job's > granularity. > > The new @bitmap parameter is marked unstable in the QAPI and can > currently only be used for @sync=full mode. > > Clusters initially dirty in the bitmap as well as new

[PATCH v2 00/11] qcow2: make subclusters discardable

2024-05-12 Thread Andrey Drobyshev
v1: https://lists.nongnu.org/archive/html/qemu-devel/2023-10/msg07223.html Andrey Drobyshev (11): qcow2: make function update_refcount_discard() global qcow2: simplify L2 entries accounting for discard-no-unref qcow2: put discard requests in the common queue when discard-no-unref enabled

[PATCH v2 03/11] qcow2: put discard requests in the common queue when discard-no-unref enabled

2024-05-12 Thread Andrey Drobyshev
Normally discard requests are stored in the queue attached to BDRVQcow2State to be processed later at once. Currently discard-no-unref option handling causes these requests to be processed straight away. Let's fix that. Note that when doing regular discards qcow2_free_any_cluster() would check f

[PATCH v2 04/11] block/file-posix: add trace event for fallocate() calls

2024-05-12 Thread Andrey Drobyshev
This would ease debugging of write zeroes and discard operations. Signed-off-by: Andrey Drobyshev --- block/file-posix.c | 1 + block/trace-events | 1 + 2 files changed, 2 insertions(+) diff --git a/block/file-posix.c b/block/file-posix.c index 35684f7e21..45134f0eef 100644 --- a/block/file-po

[PATCH v2 10/11] qcow2: zero_l2_subclusters: fall through to discard operation when requested

2024-05-12 Thread Andrey Drobyshev
When zeroizing subclusters within single cluster, detect usage of the BDRV_REQ_MAY_UNMAP flag and fall through to the subcluster-based discard operation, much like it's done with the cluster-based discards. That way subcluster-aligned operations "qemu-io -c 'write -z -u ...'" will lead to actual u

[PATCH v2 07/11] qcow2: add get_sc_range_info() helper for working with subcluster ranges

2024-05-12 Thread Andrey Drobyshev
This helper simply obtains the l2 table parameters of the cluster which contains the given subclusters range. Right now this info is being obtained and used by zero_l2_subclusters(). As we're about to introduce the subclusters discard operation, this helper would let us avoid code duplication. A

[PATCH v2 05/11] iotests/common.rc: add disk_usage function

2024-05-12 Thread Andrey Drobyshev
Move the definition from iotests/250 to common.rc. This is used to detect real disk usage of sparse files. In particular, we want to use it for checking subclusters-based discards. Signed-off-by: Andrey Drobyshev --- tests/qemu-iotests/250 | 5 - tests/qemu-iotests/common.rc | 6

[PATCH v2 11/11] iotests/271: add test cases for subcluster-based discard/unmap

2024-05-12 Thread Andrey Drobyshev
Add a bunch of test cases covering new subclusters behaviour: unmap of last allocated subclusters; unmap of subclusters within unallocated cluster; discard of unallocated subclusters within a cluster; regular discard of subclusters within a cluster; discard of last allocated subclusters. Also make

[PATCH v2 01/11] qcow2: make function update_refcount_discard() global

2024-05-12 Thread Andrey Drobyshev
We are going to need it for discarding separate subclusters. The function itself doesn't do anything with the refcount tables, it simply adds a discard request to the queue, so rename it to qcow2_queue_discard(). Signed-off-by: Andrey Drobyshev Reviewed-by: Hanna Czenczek --- block/qcow2-refco

[PATCH v2 09/11] qcow2: make subclusters discardable

2024-05-12 Thread Andrey Drobyshev
This commit makes the discard operation work on the subcluster level rather than cluster level. It introduces discard_l2_subclusters() function and makes use of it in qcow2 discard implementation, much like it's done with zero_in_l2_slice() / zero_l2_subclusters(). It also changes the qcow2 drive

[PATCH v2 02/11] qcow2: simplify L2 entries accounting for discard-no-unref

2024-05-12 Thread Andrey Drobyshev
Commits 42a2890a and b2b10904 introduce handling of discard-no-unref option in discard_in_l2_slice() and zero_in_l2_slice(). They add even more if's when chosing the right l2 entry. What we really need for this option is the new entry simply to contain the same host cluster offset, no matter whet

[PATCH v2 06/11] iotests/290: add test case to check 'discard-no-unref' option behavior

2024-05-12 Thread Andrey Drobyshev
We basically fill 2 images with identical data and perform discard operations with and without 'discard-no-unref' enabled. Then we check that images still read identically, that their disk usage is the same (i.e. fallocate(FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE) is called for both) and that with

[PATCH v2 08/11] qcow2: zeroize the entire cluster when there're no non-zero subclusters

2024-05-12 Thread Andrey Drobyshev
When zeroizing the last non-zero subclusters within single cluster, it makes sense to go zeroize the entire cluster and go down zero_in_l2_slice() path right away. That way we'd also update the corresponding refcount table. Signed-off-by: Andrey Drobyshev Reviewed-by: Hanna Czenczek --- block/

[PATCH v2 02/11] block/raw: add persistent reservation in/out driver

2024-05-12 Thread Changqi Lu
Add persistent reservation in/out operations for raw driver. The following methods are implemented: bdrv_co_pr_read_keys, bdrv_co_pr_read_reservation, bdrv_co_pr_register, bdrv_co_pr_reserve, bdrv_co_pr_release, bdrv_co_pr_clear and bdrv_co_pr_preempt. Reviewed-by: Stefan Hajnoczi Signed-off-by:

[PATCH v2 01/11] block: add persistent reservation in/out api

2024-05-12 Thread Changqi Lu
Add persistent reservation in/out operations at the block level. The following operations are included: - read_keys:retrieves the list of registered keys. - read_reservation: retrieves the current reservation status. - register: registers a new reservation key. - reserve:

[PATCH v2 05/11] hw/scsi: add persistent reservation in/out api for scsi device

2024-05-12 Thread Changqi Lu
Add persistent reservation in/out operations in the SCSI device layer. By introducing the persistent reservation in/out api, this enables the SCSI device to perform reservation-related tasks, including querying keys, querying reservation status, registering reservation keys, initiating and releasin

[PATCH v2 10/11] hw/nvme: add reservation protocal command

2024-05-12 Thread Changqi Lu
Add reservation acquire, reservation register, reservation release and reservation report commands in the nvme device layer. By introducing these commands, this enables the nvme device to perform reservation-related tasks, including querying keys, querying reservation status, registering reservati

[PATCH v2 09/11] hw/nvme: enable namespace rescap function

2024-05-12 Thread Changqi Lu
This commit enables the rescap function in the namespace by detecting the supported reservation function in the backend driver. Signed-off-by: Changqi Lu Signed-off-by: zhenwei pi --- hw/nvme/ns.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/hw/nvme/ns.c b/hw/nvme/ns.c index ea8db

[PATCH v2 04/11] scsi/util: add helper functions for persistent reservation types conversion

2024-05-12 Thread Changqi Lu
This commit introduces two helper functions that facilitate the conversion between the persistent reservation types used in the SCSI protocol and those used in the block layer. Signed-off-by: Changqi Lu Signed-off-by: zhenwei pi --- include/scsi/utils.h | 8 + scsi/utils.c | 77 +++

[PATCH v2 11/11] block/iscsi: add persistent reservation in/out driver

2024-05-12 Thread Changqi Lu
Add persistent reservation in/out operations for iscsi driver. The following methods are implemented: bdrv_co_pr_read_keys, bdrv_co_pr_read_reservation, bdrv_co_pr_register, bdrv_co_pr_reserve, bdrv_co_pr_release, bdrv_co_pr_clear and bdrv_co_pr_preempt. Signed-off-by: Changqi Lu Signed-off-by: z

[PATCH v2 00/11] Support persistent reservation operations

2024-05-12 Thread Changqi Lu
Hi Stefan; Thanks for your advices! The existing PRManager only works with local scsi devices. This series will completely decouple devices and drivers. The device can not only be scsi, but also other devices such as nvme. The same is true for the driver, which is completely unrestricted. And bloc

[PATCH v2 08/11] hw/nvme: enable ONCS reservations

2024-05-12 Thread Changqi Lu
This commit enables ONCS to support the reservation function at the controller level. It also lays the groundwork for detecting and enabling the reservation function on a per-namespace basis in RESCAP. Signed-off-by: Changqi Lu Signed-off-by: zhenwei pi --- hw/nvme/ctrl.c | 3 ++- 1 file change

[PATCH v2 07/11] hw/nvme: add helper functions for converting reservation types

2024-05-12 Thread Changqi Lu
This commit introduces two helper functions that facilitate the conversion between the reservation types used in the NVME protocol and those used in the block layer. Reviewed-by: Stefan Hajnoczi Signed-off-by: Changqi Lu Signed-off-by: zhenwei pi --- hw/nvme/nvme.h | 40 +++

[PATCH v2 06/11] block/nvme: add reservation command protocol constants

2024-05-12 Thread Changqi Lu
Add constants for the NVMe persistent command protocol. The constants include the reservation command opcode and reservation type values defined in section 7 of the NVMe 2.0 specification. Signed-off-by: Changqi Lu Signed-off-by: zhenwei pi --- include/block/nvme.h | 55

[PATCH v2 03/11] scsi/constant: add persistent reservation in/out protocol constants

2024-05-12 Thread Changqi Lu
Add constants for the persistent reservation in/out protocol in the scsi/constant module. The constants include the persistent reservation command, type, and scope values defined in sections 6.13 and 6.14 of the SCSI Primary Commands-4 (SPC-4) specification. Signed-off-by: Changqi Lu Signed-off-b