Re: [Qemu-block] [PATCH v2] nbd/server: attach client channel to the export's AioContext

2019-09-20 Thread Eric Blake
On 9/20/19 2:11 PM, Eric Blake wrote: > On 9/20/19 1:49 PM, John Snow wrote: >> > To fix this, as soon we have the export corresponding to the client, we call qio_channel_attach_aio_context() to attach the QIOChannel context to the export's AioContext. This matches with the logic

[PATCH 2/1] tests: Use iothreads during iotest 223

2019-09-20 Thread Eric Blake
Doing so catches the bugs we just fixed with NBD not properly using correct contexts. Signed-off-by: Eric Blake --- This is https://bugzilla.redhat.com/show_bug.cgi?id=1741094, distinct from Sergio's patch also related to aiocontext in NBD, which is

Re: [Qemu-devel] [PATCH v2 00/11] RFC crypto/luks: encryption key managment using amend interface

2019-09-20 Thread John Snow
On 9/12/19 6:30 PM, Maxim Levitsky wrote: > This patch series is continuation of my work to add encryption > key managment to luks/qcow2 with luks. > > This is second version of this patch set. > The changes are mostly addressing the review feedback, > plus I tested (and fixed sadly) the

Re: [Qemu-block] [PATCH] xen-block: treat XenbusStateUnknown the same as XenbusStateClosed

2019-09-20 Thread John Snow
On 9/18/19 7:57 AM, Paul Durrant wrote: > When a frontend gracefully disconnects from an offline backend, it will > set its own state to XenbusStateClosed. The code in xen-block.c correctly > deals with this and sets the backend into XenbusStateClosed. Unfortunately > it is possible for

Re: [RFC 4/4] ahci media error reporting

2019-09-20 Thread John Snow
On 9/20/19 3:25 PM, Tony Asleson wrote: > On 9/20/19 11:18 AM, John Snow wrote: >> For sure -- I handle the normative cases, but I don't test what happens >> if you issue an unsupported NCQ command. (I don't know what real >> hardware does right now, either. I'm sure I could read the spec and

Re: [RFC 4/4] ahci media error reporting

2019-09-20 Thread Tony Asleson
On 9/20/19 11:18 AM, John Snow wrote: > For sure -- I handle the normative cases, but I don't test what happens > if you issue an unsupported NCQ command. (I don't know what real > hardware does right now, either. I'm sure I could read the spec and find > out, but don't have a testing setup that

Re: [Qemu-block] [PATCH v2] nbd/server: attach client channel to the export's AioContext

2019-09-20 Thread Eric Blake
On 9/20/19 1:49 PM, John Snow wrote: > >>> To fix this, as soon we have the export corresponding to the client, >>> we call qio_channel_attach_aio_context() to attach the QIOChannel >>> context to the export's AioContext. This matches with the logic at >>> blk_aio_attached(). >>> >> >> I assume

Re: [Qemu-block] [PATCH v2] nbd/server: attach client channel to the export's AioContext

2019-09-20 Thread John Snow
On 9/20/19 2:12 PM, John Snow wrote: > > > On 9/12/19 7:00 AM, Sergio Lopez wrote: >> On creation, the export's AioContext is set to the same one as the >> BlockBackend, while the AioContext in the client QIOChannel is left >> untouched. >> >> As a result, when using data-plane,

Re: [Qemu-block] [PATCH v2] nbd/server: attach client channel to the export's AioContext

2019-09-20 Thread John Snow
On 9/12/19 7:00 AM, Sergio Lopez wrote: > On creation, the export's AioContext is set to the same one as the > BlockBackend, while the AioContext in the client QIOChannel is left > untouched. > > As a result, when using data-plane, nbd_client_receive_next_request() > schedules coroutines in

Re: [Qemu-block] [Ping] [PATCH v6 0/3] qcow2: add zstd cluster compression

2019-09-20 Thread John Snow
On 9/18/19 12:29 PM, Denis Plotnikov wrote: > On 12.09.2019 11:46, Denis Plotnikov wrote: >> ping! >> https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg00943.html I was under the impression you were addressing feedback from Vladimir. --js

Re: [RFC 4/4] ahci media error reporting

2019-09-20 Thread John Snow
On 9/19/19 5:49 PM, Tony Asleson wrote: > On 9/19/19 3:43 PM, John Snow wrote: >> >> >> On 9/19/19 3:48 PM, Tony Asleson wrote: >>> Initial attempt at returning a media error for ahci. This is certainly >>> wrong and needs serious improvement. >>> >> >> Hi; I have the unfortunate distinction

[PULL 4/4] iotests: Remove Python 2 compatibility code

2019-09-20 Thread Kevin Wolf
Some scripts check the Python version number and have two code paths to accomodate both Python 2 and 3. Remove the code specific to Python 2 and assert the minimum version of 3.6 instead (check skips Python tests in this case, so the assertion would only ever trigger if a Python script is executed

[PULL 1/4] block/snapshot: Restrict set of snapshot nodes

2019-09-20 Thread Kevin Wolf
Nodes involved in internal snapshots were those that were returned by bdrv_next(), inserted and not read-only. bdrv_next() in turn returns all nodes that are either the root node of a BlockBackend or monitor-owned nodes. With the typical -drive use, this worked well enough. However, in the

[PULL 0/4] Block layer patches

2019-09-20 Thread Kevin Wolf
The following changes since commit 521db80318d6c749a6f6c5a65a68397af9e3ef16: Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-09-16' into staging (2019-09-16 15:25:55 +0100) are available in the Git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream for

[PULL 3/4] iotests: Require Python 3.6 or later

2019-09-20 Thread Kevin Wolf
Running iotests is not required to build QEMU, so we can have stricter version requirements for Python here and can make use of new features and drop compatibility code earlier. This makes qemu-iotests skip all Python tests if a Python version before 3.6 is used for the build. Suggested-by:

Re: [RFC 4/4] ahci media error reporting

2019-09-20 Thread John Snow
On 9/20/19 4:43 AM, Kevin Wolf wrote: > Am 19.09.2019 um 22:43 hat John Snow geschrieben: >> I'd have to check -- because I can't say the AHCI emulator was designed >> so much as congealed -- but you might need calls to ncq_finish. >> >> usually, ncq_cb handles the return from any NCQ command

Re: [PATCH 1/2] block/snapshot: Restrict set of snapshot nodes

2019-09-20 Thread Max Reitz
On 17.09.19 13:04, Kevin Wolf wrote: > Nodes involved in internal snapshots were those that were returned by > bdrv_next(), inserted and not read-only. bdrv_next() in turn returns all > nodes that are either the root node of a BlockBackend or monitor-owned > nodes. > > With the typical -drive

[PULL 2/4] iotests: Test internal snapshots with -blockdev

2019-09-20 Thread Kevin Wolf
Signed-off-by: Kevin Wolf Reviewed-by: Peter Krempa Tested-by: Peter Krempa --- tests/qemu-iotests/267 | 168 tests/qemu-iotests/267.out | 182 +++ tests/qemu-iotests/common.filter | 5 +- tests/qemu-iotests/group

[PATCH 22/22] iotests: Mirror must not attempt to create loops

2019-09-20 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/041 | 152 + tests/qemu-iotests/041.out | 4 +- 2 files changed, 154 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041 index e4cc829fe2..6ea4764ae8 100755 ---

Re: [PATCH v13 00/15] backup-top filter driver for backup

2019-09-20 Thread Max Reitz
On 20.09.19 16:20, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > These series introduce backup-top driver. It's a filter-node, which > do copy-before-write operation. Mirror uses filter-node for handling > guest writes, let's move to filter-node (from write-notifiers) for > backup too. > >

Re: [PATCH v13 00/15] backup-top filter driver for backup

2019-09-20 Thread John Snow
On 9/20/19 12:01 PM, Vladimir Sementsov-Ogievskiy wrote: > 20.09.2019 18:55, Max Reitz wrote: >> On 20.09.19 16:20, Vladimir Sementsov-Ogievskiy wrote: >>> Hi all! >>> >>> These series introduce backup-top driver. It's a filter-node, which >>> do copy-before-write operation. Mirror uses

Re: [PATCH v13 00/15] backup-top filter driver for backup

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
20.09.2019 18:55, Max Reitz wrote: > On 20.09.19 16:20, Vladimir Sementsov-Ogievskiy wrote: >> Hi all! >> >> These series introduce backup-top driver. It's a filter-node, which >> do copy-before-write operation. Mirror uses filter-node for handling >> guest writes, let's move to filter-node (from

[PATCH 20/22] iotests: Add tests for invalid Quorum @replaces

2019-09-20 Thread Max Reitz
Add two tests to see that you cannot replace a Quorum child with the mirror job while the child is in use by a different parent. Signed-off-by: Max Reitz --- tests/qemu-iotests/041 | 57 +- tests/qemu-iotests/041.out | 4 +-- 2 files changed, 58

[PATCH 18/22] iotests: Resolve TODOs in 041

2019-09-20 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/041 | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041 index d91f538276..ca126de3ff 100755 --- a/tests/qemu-iotests/041 +++ b/tests/qemu-iotests/041 @@ -921,8 +921,7 @@ class

[PATCH 16/22] iotests: Use complete_and_wait() in 155

2019-09-20 Thread Max Reitz
This way, we get to see errors during the completion phase. Signed-off-by: Max Reitz --- tests/qemu-iotests/155 | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/tests/qemu-iotests/155 b/tests/qemu-iotests/155 index e19485911c..d7ef2579d3 100755 ---

[PATCH 17/22] iotests: Add VM.assert_block_path()

2019-09-20 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 48 +++ 1 file changed, 48 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index daed4ee013..e6fb46287d 100644 --- a/tests/qemu-iotests/iotests.py +++

[PATCH 15/22] mirror: Prevent loops

2019-09-20 Thread Max Reitz
While bdrv_replace_node() will not follow through with it, a specific @replaces asks the mirror job to create a loop. For example, say both the source and the target share a child where the source is a filter; by letting @replaces point to the common child, you ask for a loop. Or if you use

[PATCH 12/22] block: Remove bdrv_recurse_is_first_non_filter()

2019-09-20 Thread Max Reitz
It no longer has any users. Signed-off-by: Max Reitz --- include/block/block.h | 4 include/block/block_int.h | 8 block.c | 33 - block/blkverify.c | 15 --- block/copy-on-read.c | 9 -

[PATCH 13/22] mirror: Double-check immediately before replacing

2019-09-20 Thread Max Reitz
There is no guarantee that we can still replace the node we want to replace at the end of the mirror job. Double-check by calling bdrv_recurse_can_replace(). Signed-off-by: Max Reitz --- block/mirror.c | 14 +- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git

[PATCH 14/22] quorum: Stop marking it as a filter

2019-09-20 Thread Max Reitz
Quorum is not a filter, for example because it cannot guarantee which of its children will serve the next request. Thus, any of its children may differ from the data visible to quorum's parents. We have other filters with multiple children, but they differ in this aspect: - blkverify quits the

[PATCH 09/22] quorum: Add QuorumChild.to_be_replaced

2019-09-20 Thread Max Reitz
We will need this to verify that Quorum can let one of its children be replaced without breaking anything else. Signed-off-by: Max Reitz --- block/quorum.c | 23 +++ 1 file changed, 23 insertions(+) diff --git a/block/quorum.c b/block/quorum.c index cf2171cc74..207054a64e

[PATCH 19/22] iotests: Use self.image_len in TestRepairQuorum

2019-09-20 Thread Max Reitz
041's TestRepairQuorum has its own image_len, no need to refer to TestSingleDrive. (This patch allows uncommenting TestSingleDrive to speed up 041 during test testing.) Signed-off-by: Max Reitz --- tests/qemu-iotests/041 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

[PATCH 08/22] quorum: Store children in own structure

2019-09-20 Thread Max Reitz
This will be useful when we want to store additional attributes for each child. Signed-off-by: Max Reitz --- block/quorum.c | 58 -- 1 file changed, 33 insertions(+), 25 deletions(-) diff --git a/block/quorum.c b/block/quorum.c index

[PATCH 21/22] iotests: Check that @replaces can replace filters

2019-09-20 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/041 | 45 ++ tests/qemu-iotests/041.out | 4 ++-- 2 files changed, 47 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041 index 148dc47ce4..e4cc829fe2 100755 ---

[PATCH 11/22] block: Use bdrv_recurse_can_replace()

2019-09-20 Thread Max Reitz
Let check_to_replace_node() use the more specialized bdrv_recurse_can_replace() instead of bdrv_recurse_is_first_non_filter(), which is too restrictive. Signed-off-by: Max Reitz --- block.c | 18 -- 1 file changed, 16 insertions(+), 2 deletions(-) diff --git a/block.c b/block.c

[PATCH 05/22] quorum: Fix child permissions

2019-09-20 Thread Max Reitz
Quorum is not actually a filter. It cannot share WRITE or RESIZE on its children. Signed-off-by: Max Reitz --- block/quorum.c | 19 ++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/block/quorum.c b/block/quorum.c index df68adcfaa..17b439056f 100644 ---

[PATCH 03/22] block: Drop bdrv_is_first_non_filter()

2019-09-20 Thread Max Reitz
It is unused now. (And it was ugly because it needed to explore all BDS chains from the top.) Signed-off-by: Max Reitz --- include/block/block.h | 1 - block.c | 26 -- 2 files changed, 27 deletions(-) diff --git a/include/block/block.h

[PATCH 06/22] block: Add bdrv_recurse_can_replace()

2019-09-20 Thread Max Reitz
After a couple of follow-up patches, this function will replace bdrv_recurse_is_first_non_filter() in check_to_replace_node(). bdrv_recurse_is_first_non_filter() is both not sufficiently specific for check_to_replace_node() (it allows cases that should not be allowed, like replacing child nodes

[PATCH 10/22] quorum: Implement .bdrv_recurse_can_replace()

2019-09-20 Thread Max Reitz
Signed-off-by: Max Reitz --- block/quorum.c | 62 ++ 1 file changed, 62 insertions(+) diff --git a/block/quorum.c b/block/quorum.c index 207054a64e..81b57dbae2 100644 --- a/block/quorum.c +++ b/block/quorum.c @@ -825,6 +825,67 @@ static bool

[PATCH 00/22] block: Fix check_to_replace_node()

2019-09-20 Thread Max Reitz
Based-on: <20190912135632.13925-1-mre...@redhat.com> (“mirror: Do not dereference invalid pointers”) Hi, We have this function bdrv_is_first_non_filter(), and I don’t know what we have it for exactly. It’s used in three places: 1. To allow snapshots only on such nodes, 2. To allow resizing

[PATCH 04/22] iotests: Let 041 use -blockdev for quorum children

2019-09-20 Thread Max Reitz
Using -drive with default options means that a virtio-blk drive will be created that has write access to the to-be quorum children. Quorum should have exclusive write access to them, so we should use -blockdev instead. Signed-off-by: Max Reitz --- tests/qemu-iotests/041 | 5 - 1 file

[PATCH 07/22] blkverify: Implement .bdrv_recurse_can_replace()

2019-09-20 Thread Max Reitz
Signed-off-by: Max Reitz --- block/blkverify.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/block/blkverify.c b/block/blkverify.c index 304b0a1368..0add3ab483 100644 --- a/block/blkverify.c +++ b/block/blkverify.c @@ -282,6 +282,20 @@ static bool

[PATCH 01/22] blockdev: Allow external snapshots everywhere

2019-09-20 Thread Max Reitz
There is no good reason why we would allow external snapshots only on the first non-filter node in a chain. Parent BDSs should not care whether their child is replaced by a snapshot. (If they do care, they should announce that via freezing the chain, which is checked in bdrv_append() through

Re: COMPARE_AND_WRITE support for rbd driver

2019-09-20 Thread Yaowei Bai
On Fri, Sep 20, 2019 at 01:22:02PM +0200, Paolo Bonzini wrote: > On 19/09/19 07:36, Yaowei Bai wrote: > > > > Hey guys, > > > > I noticed that COMPARE_AND_WRITE had been supported by CEPH/librbd since > > v12.1.1. And in my company, we use this COMPARE_AND_WRITE support in > > CEPH with the

Re: [PATCH v5 0/5] qcow2: async handling of fragmented io

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
20.09.2019 17:17, Eric Blake wrote: > On 9/20/19 7:53 AM, Vladimir Sementsov-Ogievskiy wrote: > >>> >>> It doesn’t matter much to me, I diff all patches anyway. :-) >>> >> >> then a bit offtopic: >> >> Which tools are you use? >> >> I've some scripts to compare different versions of one serie (or

[PATCH v13 11/15] iotests: 257: drop device_add

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
SCSI devices are unused in test, drop them. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/257 | 8 --- tests/qemu-iotests/257.out | 44 -- 2 files changed, 52 deletions(-) diff --git

[PATCH v13 10/15] iotests: 257: drop unused Drive.device field

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
After previous commit Drive.device is actually unused. Drop it together with .name property. While being here reuse .node in qmp commands instead of writing 'drive0' twice. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/257 | 37

[PATCH v13 15/15] block/backup: use backup-top instead of write notifiers

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Drop write notifiers and use filter node instead. = Changes = 1. Add filter-node-name argument for backup qmp api. We have to do it in this commit, as 257 needs to be fixed. 2. There are no more write notifiers here, so is_write_notifier parameter is dropped from block-copy paths. 3.

[PATCH v13 09/15] iotests: prepare 124 and 257 bitmap querying for backup-top filter

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
After backup-top filter appearing it's not possible to see dirty bitmaps in top node, so use node-name instead. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/124| 83 tests/qemu-iotests/257| 49 ++---

[PATCH v13 12/15] block/io: refactor wait_serialising_requests

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Split out do_wait_serialising_requests with additional possibility to not actually wait but just check, that there is something to wait for. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/io.c | 24 1 file changed, 16 insertions(+), 8

[PATCH v13 07/15] block: move block_copy from block/backup.c to separate file

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Split block_copy to separate file, to be cleanly shared with backup-top filter driver in further commits. It's a clean movement, the only change is drop "static" from interface functions. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- include/block/block-copy.h | 76

[PATCH v13 05/15] block/backup: introduce BlockCopyState

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Split copying code part from backup to "block-copy", including separate state structure and function renaming. This is needed to share it with backup-top filter driver in further commits. Notes: 1. As BlockCopyState keeps own BlockBackend objects, remaining job->common.blk users only use it to

[PATCH v13 06/15] block/backup: fix block-comment style

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
We need to fix comment style around block-copy functions before further moving them to separate file to satisfy checkpatch. But do more: fix all comments style. Also, seems like doubled first asterisk is not forbidden, but drop it too for consistency. Signed-off-by: Vladimir Sementsov-Ogievskiy

[PATCH v13 13/15] block: add lock/unlock range functions

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Introduce lock/unlock range functionality, based on serialized requests. This is needed to refactor backup, dropping local tracked-request-like synchronization. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- include/block/block_int.h | 4 block/io.c

[PATCH v13 14/15] block: introduce backup-top filter driver

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Backup-top filter caches write operations and does copy-before-write operations. The driver will be used in backup instead of write-notifiers. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup-top.h | 37 +++ block/backup-top.c | 244

[PATCH v13 08/15] block: teach bdrv_debug_breakpoint skip filters with backing

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Teach bdrv_debug_breakpoint and bdrv_debug_remove_breakpoint skip filters with backing. This is needed to implement and use in backup job it's own backup_top filter driver (like mirror already has one), and without this improvement, breakpoint removal will fail at least in 55 iotest.

[PATCH v13 00/15] backup-top filter driver for backup

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Hi all! These series introduce backup-top driver. It's a filter-node, which do copy-before-write operation. Mirror uses filter-node for handling guest writes, let's move to filter-node (from write-notifiers) for backup too. v11,v12 -> v13 changes: [v12 was two fixes in separate: [PATCH v12 0/2]

[PATCH v13 03/15] block/backup: split shareable copying part from backup_do_cow

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Split copying logic which will be shared with backup-top filter. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup.c | 47 --- 1 file changed, 32 insertions(+), 15 deletions(-) diff --git a/block/backup.c

[PATCH v13 02/15] block/backup: fix backup_cow_with_offload for last cluster

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
We shouldn't try to copy bytes beyond EOF. Fix it. Fixes: 9ded4a0114968e Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/backup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/backup.c b/block/backup.c index

[PATCH v13 01/15] block/backup: fix max_transfer handling for copy_range

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Of course, QEMU_ALIGN_UP is a typo, it should be QEMU_ALIGN_DOWN, as we are trying to find aligned size which satisfy both source and target. Also, don't ignore too small max_transfer. In this case seems safer to disable copy_range. Fixes: 9ded4a0114968e Signed-off-by: Vladimir

[PATCH v13 04/15] block/backup: improve comment about image fleecing

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/backup.c | 15 --- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/block/backup.c b/block/backup.c index 98d7f7a905..ae28849660 100644 --- a/block/backup.c +++ b/block/backup.c @@ -747,9

Re: [PATCH v5 0/5] qcow2: async handling of fragmented io

2019-09-20 Thread Eric Blake
On 9/20/19 7:53 AM, Vladimir Sementsov-Ogievskiy wrote: >> >> It doesn’t matter much to me, I diff all patches anyway. :-) >> > > then a bit offtopic: > > Which tools are you use? > > I've some scripts to compare different versions of one serie (or to check, > what > was changed in patches

Re: [PATCH v11 04/14] block/backup: introduce BlockCopyState

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
20.09.2019 16:26, Max Reitz wrote: > On 20.09.19 14:56, Vladimir Sementsov-Ogievskiy wrote: >> 20.09.2019 15:56, Vladimir Sementsov-Ogievskiy wrote: >>> 20.09.2019 15:46, Max Reitz wrote: On 13.09.19 20:25, Vladimir Sementsov-Ogievskiy wrote: > 10.09.2019 13:23, Vladimir

Re: [PATCH v5 0/5] qcow2: async handling of fragmented io

2019-09-20 Thread Max Reitz
On 20.09.19 15:26, Vladimir Sementsov-Ogievskiy wrote: > 20.09.2019 16:10, Max Reitz wrote: >> On 20.09.19 14:53, Vladimir Sementsov-Ogievskiy wrote: >>> 20.09.2019 15:40, Max Reitz wrote: On 20.09.19 13:53, Vladimir Sementsov-Ogievskiy wrote: > 20.09.2019 14:10, Max Reitz wrote: >>

Re: [PATCH v11 04/14] block/backup: introduce BlockCopyState

2019-09-20 Thread Max Reitz
On 20.09.19 14:56, Vladimir Sementsov-Ogievskiy wrote: > 20.09.2019 15:56, Vladimir Sementsov-Ogievskiy wrote: >> 20.09.2019 15:46, Max Reitz wrote: >>> On 13.09.19 20:25, Vladimir Sementsov-Ogievskiy wrote: 10.09.2019 13:23, Vladimir Sementsov-Ogievskiy wrote: > Split copying code part

Re: [PATCH v5 0/5] qcow2: async handling of fragmented io

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
20.09.2019 16:10, Max Reitz wrote: > On 20.09.19 14:53, Vladimir Sementsov-Ogievskiy wrote: >> 20.09.2019 15:40, Max Reitz wrote: >>> On 20.09.19 13:53, Vladimir Sementsov-Ogievskiy wrote: 20.09.2019 14:10, Max Reitz wrote: > On 16.09.19 19:53, Vladimir Sementsov-Ogievskiy wrote: >>

Re: [PATCH v5 0/5] qcow2: async handling of fragmented io

2019-09-20 Thread Max Reitz
On 20.09.19 14:53, Vladimir Sementsov-Ogievskiy wrote: > 20.09.2019 15:40, Max Reitz wrote: >> On 20.09.19 13:53, Vladimir Sementsov-Ogievskiy wrote: >>> 20.09.2019 14:10, Max Reitz wrote: On 16.09.19 19:53, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > Here is an asynchronous

Re: [RFC] error: auto propagated local_err

2019-09-20 Thread Eric Blake
On 9/19/19 10:50 AM, Daniel P. Berrangé wrote: > To get some slightly less made-up stats, I did some searching: > > - 4408 methods with an 'errp' parameter declared > > git grep 'Error \*\*errp'| wc -l > > - 696 methods with an 'Error *local' declared (what other names > do we

Re: [PATCH v11 04/14] block/backup: introduce BlockCopyState

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
20.09.2019 15:46, Max Reitz wrote: > On 13.09.19 20:25, Vladimir Sementsov-Ogievskiy wrote: >> 10.09.2019 13:23, Vladimir Sementsov-Ogievskiy wrote: >>> Split copying code part from backup to "block-copy", including separate >>> state structure and function renaming. This is needed to share it

Re: [PATCH v11 04/14] block/backup: introduce BlockCopyState

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
20.09.2019 15:56, Vladimir Sementsov-Ogievskiy wrote: > 20.09.2019 15:46, Max Reitz wrote: >> On 13.09.19 20:25, Vladimir Sementsov-Ogievskiy wrote: >>> 10.09.2019 13:23, Vladimir Sementsov-Ogievskiy wrote: Split copying code part from backup to "block-copy", including separate state

Re: [PATCH v5 0/5] qcow2: async handling of fragmented io

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
20.09.2019 15:40, Max Reitz wrote: > On 20.09.19 13:53, Vladimir Sementsov-Ogievskiy wrote: >> 20.09.2019 14:10, Max Reitz wrote: >>> On 16.09.19 19:53, Vladimir Sementsov-Ogievskiy wrote: Hi all! Here is an asynchronous scheme for handling fragmented qcow2 reads and writes.

Re: [PATCH v11 04/14] block/backup: introduce BlockCopyState

2019-09-20 Thread Max Reitz
On 13.09.19 20:25, Vladimir Sementsov-Ogievskiy wrote: > 10.09.2019 13:23, Vladimir Sementsov-Ogievskiy wrote: >> Split copying code part from backup to "block-copy", including separate >> state structure and function renaming. This is needed to share it with >> backup-top filter driver in further

Re: [PATCH v5 0/5] qcow2: async handling of fragmented io

2019-09-20 Thread Max Reitz
On 20.09.19 13:53, Vladimir Sementsov-Ogievskiy wrote: > 20.09.2019 14:10, Max Reitz wrote: >> On 16.09.19 19:53, Vladimir Sementsov-Ogievskiy wrote: >>> Hi all! >>> >>> Here is an asynchronous scheme for handling fragmented qcow2 >>> reads and writes. Both qcow2 read and write functions loops

Re: [PATCH v2] nbd/server: attach client channel to the export's AioContext

2019-09-20 Thread Eric Blake
On 9/12/19 6:00 AM, Sergio Lopez wrote: > On creation, the export's AioContext is set to the same one as the > BlockBackend, while the AioContext in the client QIOChannel is left > untouched. > > As a result, when using data-plane, nbd_client_receive_next_request() > schedules coroutines in the

[RFC PATCH] virtio-blk: advertise F_WCE (F_FLUSH) if F_CONFIG_WCE is also advertised

2019-09-20 Thread Evgeny Yakovlev
Virtio spec 1.1 (and earlier), 5.2.5.1 Driver Requirements: Device Initialization: "Devices SHOULD always offer VIRTIO_BLK_F_FLUSH, and MUST offer it if they offer VIRTIO_BLK_F_CONFIG_WCE. It looks like currently F_CONFIG_WCE and F_WCE are not connected to each other. qemu will advertise

Re: [PATCH v5 0/5] qcow2: async handling of fragmented io

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
20.09.2019 14:10, Max Reitz wrote: > On 16.09.19 19:53, Vladimir Sementsov-Ogievskiy wrote: >> Hi all! >> >> Here is an asynchronous scheme for handling fragmented qcow2 >> reads and writes. Both qcow2 read and write functions loops through >> sequential portions of data. The series aim it to

Re: [RFC] error: auto propagated local_err

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
18.09.2019 16:02, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > Here is a proposal (three of them, actually) of auto propagation for > local_err, to not call error_propagate on every exit point, when we > deal with local_err. > > It also may help make Greg's series[1] about error_append_hint

Re: [PATCH 2/2] iotests: Test internal snapshots with -blockdev

2019-09-20 Thread Max Reitz
On 17.09.19 13:04, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > tests/qemu-iotests/267 | 165 > tests/qemu-iotests/267.out | 182 +++ > tests/qemu-iotests/common.filter | 5 +- > tests/qemu-iotests/group

Re: COMPARE_AND_WRITE support for rbd driver

2019-09-20 Thread Paolo Bonzini
On 19/09/19 07:36, Yaowei Bai wrote: > baiyao...@cmss.chinamobile.com > Bcc: > Subject: COMPARE_AND_WRITE support for rbd driver > Reply-To: baiyao...@cmss.chinamobile.com > > Hey guys, > > I noticed that COMPARE_AND_WRITE had been supported by CEPH/librbd since > v12.1.1. And in my company,

Re: [PATCH v5 0/5] qcow2: async handling of fragmented io

2019-09-20 Thread Max Reitz
On 16.09.19 19:53, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > Here is an asynchronous scheme for handling fragmented qcow2 > reads and writes. Both qcow2 read and write functions loops through > sequential portions of data. The series aim it to parallelize these > loops iterations. > It

Re: [PATCH v2 1/2] iotests: Require Python 3.6 or later

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
20.09.2019 12:27, Kevin Wolf wrote: > Am 20.09.2019 um 10:40 hat Vladimir Sementsov-Ogievskiy geschrieben: >> 19.09.2019 19:29, Kevin Wolf wrote: >>> Running iotests is not required to build QEMU, so we can have stricter >>> version requirements for Python here and can make use of new features >>>

Re: [PATCH v2 1/2] iotests: Require Python 3.6 or later

2019-09-20 Thread Kevin Wolf
Am 20.09.2019 um 10:40 hat Vladimir Sementsov-Ogievskiy geschrieben: > 19.09.2019 19:29, Kevin Wolf wrote: > > Running iotests is not required to build QEMU, so we can have stricter > > version requirements for Python here and can make use of new features > > and drop compatibility code earlier. >

Re: [Qemu-block] [PATCH v3] virtio-blk: schedule virtio_notify_config to run on main context

2019-09-20 Thread Stefan Hajnoczi
On Mon, Sep 16, 2019 at 01:24:12PM +0200, Sergio Lopez wrote: > virtio_notify_config() needs to acquire the global mutex, which isn't > allowed from an iothread, and may lead to a deadlock like this: > > - main thead > * Has acquired: qemu_global_mutex. > * Is trying the acquire: iothread

Re: [PATCH v2 2/2] iotests: Remove Python 2 compatibility code

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
19.09.2019 19:29, Kevin Wolf wrote: > Some scripts check the Python version number and have two code paths to > accomodate both Python 2 and 3. Remove the code specific to Python 2 and > assert the minimum version of 3.6 instead (check skips Python tests in > this case, so the assertion would only

Re: [RFC 4/4] ahci media error reporting

2019-09-20 Thread Kevin Wolf
Am 19.09.2019 um 22:43 hat John Snow geschrieben: > I'd have to check -- because I can't say the AHCI emulator was designed > so much as congealed -- but you might need calls to ncq_finish. > > usually, ncq_cb handles the return from any NCQ command and will call > ncq_err and ncq_finish as

Re: [PATCH v2 1/2] iotests: Require Python 3.6 or later

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
19.09.2019 19:29, Kevin Wolf wrote: > Running iotests is not required to build QEMU, so we can have stricter > version requirements for Python here and can make use of new features > and drop compatibility code earlier. > > This makes qemu-iotests skip all Python tests if a Python version before

[PATCH v3 0/3] proper locking on bitmap add/remove paths

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
Hi all! We need to lock qcow2 mutex on accessing in-image metadata, especially on updating this metadata. Let's implement it. v3: 01: add John's r-b 02: - fix bdrv_remove_persistent_dirty_bitmap return value - drop extra zeroing of ret in qcow2_remove_persistent_dirty_bitmap 03: add John's

[PATCH v3 3/3] block/qcow2: proper locking on bitmap add/remove paths

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
qmp_block_dirty_bitmap_add and do_block_dirty_bitmap_remove do acquire aio context since 0a6c86d024c52b. But this is not enough: we also must lock qcow2 mutex when access in-image metadata. Especially it concerns freeing qcow2 clusters. To achieve this, move qcow2_can_store_new_dirty_bitmap and

[PATCH v3 2/3] block/dirty-bitmap: return int from bdrv_remove_persistent_dirty_bitmap

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
It's more comfortable to not deal with local_err. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h| 5 ++--- include/block/block_int.h| 6 +++--- include/block/dirty-bitmap.h | 5 ++--- block/dirty-bitmap.c | 9 + block/qcow2-bitmap.c

[PATCH v3 1/3] block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
block/dirty-bitmap.c seems to be more appropriate for it and bdrv_remove_persistent_dirty_bitmap already in it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- block.c | 22 -- block/dirty-bitmap.c | 22 ++ 2 files

Re: [Qemu-devel] [PATCH v12 1/2] block/backup: fix max_transfer handling for copy_range

2019-09-20 Thread Vladimir Sementsov-Ogievskiy
20.09.2019 4:13, John Snow wrote: > > > On 9/19/19 2:50 AM, Vladimir Sementsov-Ogievskiy wrote: >> 18.09.2019 22:57, John Snow wrote: >>> >>> >>> On 9/17/19 12:07 PM, Vladimir Sementsov-Ogievskiy wrote: Of course, QEMU_ALIGN_UP is a typo, it should be QEMU_ALIGN_DOWN, as we are trying