Re: [Qemu-block] [PATCH 3/7] qcow2: Make perform_cow() call do_perform_cow() twice

2017-05-26 Thread Kevin Wolf
Am 23.05.2017 um 13:22 hat Alberto Garcia geschrieben: > Instead of calling perform_cow() twice with a different COW region > each time, call it just once and make perform_cow() handle both > regions. > > This patch simply moves code around. The next one will do the actual > reordering of the COW

Re: [Qemu-block] [PATCH 3/7] qcow2: Make perform_cow() call do_perform_cow() twice

2017-05-26 Thread Alberto Garcia
On Fri 26 May 2017 10:11:29 AM CEST, Kevin Wolf wrote: >> --- a/block/qcow2-cluster.c >> +++ b/block/qcow2-cluster.c >> @@ -414,6 +414,10 @@ static int coroutine_fn do_perform_cow(BlockDriverState >> *bs, >> struct iovec iov; >> int ret; >> >> +if (bytes == 0) { >> +return

Re: [Qemu-block] [PATCH 3/7] qcow2: Make perform_cow() call do_perform_cow() twice

2017-05-26 Thread Kevin Wolf
Am 26.05.2017 um 11:10 hat Alberto Garcia geschrieben: > On Fri 26 May 2017 10:11:29 AM CEST, Kevin Wolf wrote: > >> --- a/block/qcow2-cluster.c > >> +++ b/block/qcow2-cluster.c > >> @@ -414,6 +414,10 @@ static int coroutine_fn > >> do_perform_cow(BlockDriverState *bs, > >> struct iovec iov;

Re: [Qemu-block] [Qemu-devel] [PATCH 7/7] qcow2: Merge the writing of the COW regions with the guest data

2017-05-26 Thread Kevin Wolf
Am 24.05.2017 um 21:05 hat Alberto Garcia geschrieben: > On Wed 24 May 2017 06:43:31 PM CEST, Anton Nefedov wrote: > >> +if (m->data_qiov) { > >> +qemu_iovec_reset(&qiov); > >> +qemu_iovec_add(&qiov, start_buffer, start->nb_bytes); > >> +qemu_iovec_concat(&qiov, m->data_

Re: [Qemu-block] [PATCH 0/7] qcow2: Reduce the number of I/O ops when doing COW

2017-05-26 Thread Kevin Wolf
Am 24.05.2017 um 18:09 hat Anton Nefedov geschrieben: > I agree; as mentioned we have similar patches and they don't conflict much. > We noticed a performance regression on HDD though, for the > presumably optimized case (random 4k write over a large backed > image); so the patches were put on hold

Re: [Qemu-block] [Qemu-devel] Behavior of QMP "query-block"

2017-05-26 Thread Kevin Wolf
Am 25.05.2017 um 20:44 hat Bruno Alvisio geschrieben: > Hello John, > > Thanks. Yes, typo when I wrote the e-mail. > > It might be possible that XEN does that (I will ask in the XEN forum). > However, > I have noticed that it is not the case for all of the VMs I have launched. In > some of them

Re: [Qemu-block] [PATCH v7 09/20] qcow: convert QCow to use QCryptoBlock for encryption

2017-05-26 Thread Alberto Garcia
On Thu 25 May 2017 06:38:40 PM CEST, Daniel P. Berrange wrote: > @@ -105,6 +116,13 @@ static int qcow_open(BlockDriverState *bs, QDict > *options, int flags, > int ret; > QCowHeader header; > Error *local_err = NULL; > +QCryptoBlockOpenOptions *crypto_opts = NULL; > +unsigne

Re: [Qemu-block] [PATCH v7 11/20] qcow2: convert QCow2 to use QCryptoBlock for encryption

2017-05-26 Thread Alberto Garcia
On Thu 25 May 2017 06:38:42 PM CEST, Daniel P. Berrange wrote: > This converts the qcow2 driver to make use of the QCryptoBlock > APIs for encrypting image content, using the legacy QCow2 AES > scheme. > > With this change it is now required to use the QCryptoSecret > object for providing passwords

Re: [Qemu-block] [PATCH 0/7] qcow2: Reduce the number of I/O ops when doing COW

2017-05-26 Thread Anton Nefedov
On 05/26/2017 01:17 PM, Kevin Wolf wrote: Am 24.05.2017 um 18:09 hat Anton Nefedov geschrieben: I agree; as mentioned we have similar patches and they don't conflict much. We noticed a performance regression on HDD though, for the presumably optimized case (random 4k write over a large backed i

Re: [Qemu-block] [PATCH 0/7] qcow2: Reduce the number of I/O ops when doing COW

2017-05-26 Thread Alberto Garcia
On Fri 26 May 2017 02:47:55 PM CEST, Anton Nefedov wrote: > Tried the another machine; about 10% improvement here [...] > [root@localhost ~]# fio --name=randwrite --blocksize=4k > --filename=/dev/sdb --rw=randwrite --direct=1 --ioengine=libaio > --size=2g --io_size=32m In my tests I sometimes de

Re: [Qemu-block] [PATCH 0/7] qcow2: Reduce the number of I/O ops when doing COW

2017-05-26 Thread Anton Nefedov
On 05/26/2017 04:08 PM, Alberto Garcia wrote: On Fri 26 May 2017 02:47:55 PM CEST, Anton Nefedov wrote: Tried the another machine; about 10% improvement here [...] [root@localhost ~]# fio --name=randwrite --blocksize=4k --filename=/dev/sdb --rw=randwrite --direct=1 --ioengine=libaio --size=

Re: [Qemu-block] [PATCH 0/7] qcow2: Reduce the number of I/O ops when doing COW

2017-05-26 Thread Alberto Garcia
On Fri 26 May 2017 03:32:49 PM CEST, Anton Nefedov wrote: >>> [root@localhost ~]# fio --name=randwrite --blocksize=4k >>> --filename=/dev/sdb --rw=randwrite --direct=1 --ioengine=libaio >>> --size=2g --io_size=32m >> >> In my tests I sometimes detected slight performance decreases in that >> HDD s

Re: [Qemu-block] [PATCH v7 09/20] qcow: convert QCow to use QCryptoBlock for encryption

2017-05-26 Thread Daniel P. Berrange
On Fri, May 26, 2017 at 02:03:40PM +0200, Alberto Garcia wrote: > On Thu 25 May 2017 06:38:40 PM CEST, Daniel P. Berrange wrote: > > @@ -105,6 +116,13 @@ static int qcow_open(BlockDriverState *bs, QDict > > *options, int flags, > > int ret; > > QCowHeader header; > > Error *local_er

Re: [Qemu-block] [PATCH v7 12/20] qcow2: extend specification to cover LUKS encryption

2017-05-26 Thread Daniel P. Berrange
On Thu, May 25, 2017 at 02:39:56PM -0500, Eric Blake wrote: > On 05/25/2017 11:38 AM, Daniel P. Berrange wrote: > > Update the qcow2 specification to describe how the LUKS header is > > placed inside a qcow2 file, when using LUKS encryption for the > > qcow2 payload instead of the legacy AES-CBC en

Re: [Qemu-block] [PATCH v7 19/20] qcow2: report encryption specific image information

2017-05-26 Thread Daniel P. Berrange
On Thu, May 25, 2017 at 02:52:30PM -0500, Eric Blake wrote: > On 05/25/2017 11:38 AM, Daniel P. Berrange wrote: > > Currently 'qemu-img info' reports a simple "encrypted: yes" > > field. This is not very useful now that qcow2 can support > > multiple encryption formats. Users want to know which for

Re: [Qemu-block] [PATCH 7/7] qcow2: Merge the writing of the COW regions with the guest data

2017-05-26 Thread Alberto Garcia
On Tue 23 May 2017 01:23:02 PM CEST, Alberto Garcia wrote: You can still review this now if you want, but here's a couple of minor things I'll correct in the next revision: > +if (m->data_qiov) { > +qemu_iovec_reset(&qiov); > +qemu_iovec_add(&qiov, start_buffer, start->nb_byte

Re: [Qemu-block] [Qemu-devel] Behavior of QMP "query-block"

2017-05-26 Thread John Snow
On 05/26/2017 07:40 AM, Kevin Wolf wrote: > Am 25.05.2017 um 20:44 hat Bruno Alvisio geschrieben: >> Hello John, >> >> Thanks. Yes, typo when I wrote the e-mail. >> >> It might be possible that XEN does that (I will ask in the XEN forum). >> However, >> I have noticed that it is not the case for

[Qemu-block] [PATCH v3 00/16] block: Preallocated truncate

2017-05-26 Thread Max Reitz
=== Series dependencies === This series depends on v6 of Stefan's series "qemu-img: add measure sub-command" (http://lists.nongnu.org/archive/html/qemu-devel/2017-05/msg01706.html). === Actual cover letter === This series adds preallocation to bdrv_truncate() and subsequently qemu-img resize. T

[Qemu-block] [PATCH v3 01/16] block: Add PreallocMode to BD.bdrv_truncate()

2017-05-26 Thread Max Reitz
Add a PreallocMode parameter to the bdrv_truncate() function implemented by each block driver. Currently, we always pass PREALLOC_MODE_OFF and no driver accepts anything else. Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi --- include/block/block_int.h | 3 ++- block.c

[Qemu-block] [PATCH v3 02/16] block: Add PreallocMode to bdrv_truncate()

2017-05-26 Thread Max Reitz
For block drivers that just pass a truncate request to the underlying protocol, we can now pass the preallocation mode instead of aborting if it is not PREALLOC_MODE_OFF. Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi --- include/block/block.h | 3 ++- block.c| 5 +++--

[Qemu-block] [PATCH v3 06/16] block/file-posix: Extract raw_regular_truncate()

2017-05-26 Thread Max Reitz
This functionality is part of raw_create() which we will be able to reuse nicely in raw_truncate(). Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daudé --- block/file-posix.c | 144 + 1 file changed, 78 i

[Qemu-block] [PATCH v3 05/16] block/file-posix: Small fixes in raw_create()

2017-05-26 Thread Max Reitz
Variables should be declared at the start of a block, and if a certain parameter value is not supported it may be better to return -ENOTSUP instead of -EINVAL. Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Daudé --- block/file-posix.c | 5 +++-- 1 file cha

[Qemu-block] [PATCH v3 08/16] block/file-posix: Preallocation for truncate

2017-05-26 Thread Max Reitz
By using raw_regular_truncate() in raw_truncate(), we can now easily support preallocation. Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi --- block/file-posix.c | 22 ++ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/block/file-posix.c b/block/file-p

[Qemu-block] [PATCH v3 03/16] block: Add PreallocMode to blk_truncate()

2017-05-26 Thread Max Reitz
blk_truncate() itself will pass that value to bdrv_truncate(), and all callers of blk_truncate() just set the parameter to PREALLOC_MODE_OFF for now. Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi --- include/sysemu/block-backend.h | 3 ++- block/block-backend.c | 5 +++-- block

[Qemu-block] [PATCH v3 04/16] qemu-img: Expose PreallocMode for resizing

2017-05-26 Thread Max Reitz
Add a --preallocation command line option to qemu-img resize which can be used to set the PreallocMode parameter of blk_truncate(). Signed-off-by: Max Reitz --- qemu-img.c| 33 ++--- qemu-img.texi | 7 ++- 2 files changed, 36 insertions(+), 4 deletions(-) di

[Qemu-block] [PATCH v3 11/16] block/qcow2: Metadata preallocation for truncate

2017-05-26 Thread Max Reitz
We can support PREALLOC_MODE_METADATA by invoking preallocate() in qcow2_truncate(). Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi --- block/qcow2.c | 24 ++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index a8223a0.

[Qemu-block] [PATCH v3 14/16] block/qcow2: falloc/full preallocating growth

2017-05-26 Thread Max Reitz
Implement the preallocation modes falloc and full for growing qcow2 images. Signed-off-by: Max Reitz --- block/qcow2.h | 5 +++ block/qcow2-refcount.c | 12 ++ block/qcow2.c | 100 - 3 files changed, 108 insertions(+), 9 de

[Qemu-block] [PATCH v3 15/16] iotests: Add preallocated resize test for raw

2017-05-26 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/106 | 92 ++ tests/qemu-iotests/106.out | 50 + tests/qemu-iotests/group | 1 + 3 files changed, 143 insertions(+) create mode 100755 tests/qem

[Qemu-block] [PATCH v3 16/16] iotests: Add preallocated growth test for qcow2

2017-05-26 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/125 | 130 +++ tests/qemu-iotests/125.out | 386 + tests/qemu-iotests/group | 1 + 3 files changed, 517 insertions(+) create mode 100755 tests/qemu-iotest

[Qemu-block] [PATCH v3 07/16] block/file-posix: Generalize raw_regular_truncate

2017-05-26 Thread Max Reitz
Currently, raw_regular_truncate() is intended for setting the size of a newly created file. However, we also want to use it for truncating an existing file in which case only the newly added space (when growing) should be preallocated. This also means that if resizing failed, we should try to rest

[Qemu-block] [PATCH v3 10/16] block/qcow2: Lock s->lock in preallocate()

2017-05-26 Thread Max Reitz
preallocate() is and will be called only from places that do not lock s->lock: Currently that is qcow2_create2(), as of a future patch it will be called from qcow2_truncate(), too. It therefore makes sense to move locking that mutex into preallocate() itself. Signed-off-by: Max Reitz --- block/

[Qemu-block] [PATCH v3 09/16] block/qcow2: Generalize preallocate()

2017-05-26 Thread Max Reitz
This patch adds two new parameters to the preallocate() function so we will be able to use it not just for preallocating a new image but also for preallocated image growth. The offset parameter allows the caller to specify a virtual offset from which to start preallocating. For newly created image

[Qemu-block] [PATCH v3 13/16] block/qcow2: Rename "fail_block" to just "fail"

2017-05-26 Thread Max Reitz
Now alloc_refcount_block() only contains a single fail label, so it makes more sense to just name it "fail" instead of "fail_block". Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/block/qcow2-refcount.c b/blo

[Qemu-block] [PATCH v3 12/16] block/qcow2: Add qcow2_refcount_area()

2017-05-26 Thread Max Reitz
This function creates a collection of self-describing refcount structures (including a new refcount table) at the end of a qcow2 image file. Optionally, these structures can also describe a number of additional clusters beyond themselves; this will be important for preallocated truncation, which wi

Re: [Qemu-block] [PATCH v2 0/3] block: Fix backing paths for filenames with colons

2017-05-26 Thread Max Reitz
On 2017-05-22 21:52, Max Reitz wrote: > There are (at least) two issues with filenames that contain colons when > trying to use relative backing filenames with them, for each of which > there is a patch in this series. > > The first patch fixes an issue in the general block layer > (path_combine()

Re: [Qemu-block] [PATCH] block/gluster: glfs_lseek() workaround

2017-05-26 Thread Jeff Cody
On Tue, May 23, 2017 at 01:27:50PM -0400, Jeff Cody wrote: > On current released versions of glusterfs, glfs_lseek() will sometimes > return invalid values for SEEK_DATA or SEEK_HOLE. For SEEK_DATA and > SEEK_HOLE, the returned value should be >= the passed offset, or < 0 in > the case of error: >

[Qemu-block] [PULL 01/12] blockjob: remove unnecessary check

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini !job is always checked prior to the call, drop it from here. Reviewed-by: Stefan Hajnoczi Signed-off-by: Paolo Bonzini Reviewed-by: Jeff Cody Message-id: 20170508141310.8674-2-pbonz...@redhat.com Signed-off-by: Jeff Cody --- blockjob.c | 2 +- 1 file changed, 1 insertion

[Qemu-block] [PULL 00/12] Block patches

2017-05-26 Thread Jeff Cody
The following changes since commit 9964e96dccf7f7c936ee854a795415d19b60: Merge remote-tracking branch 'jasowang/tags/net-pull-request' into staging (2017-05-23 15:01:31 +0100) are available in the git repository at: git://github.com/codyprime/qemu-kvm-jtc.git tags/block-pull-request fo

[Qemu-block] [PULL 11/12] blockjob: use deferred_to_main_loop to indicate the coroutine has ended

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini All block jobs are using block_job_defer_to_main_loop as the final step just before the coroutine terminates. At this point, block_job_enter should do nothing, but currently it restarts the freed coroutine. Now, the job->co states should probably be changed to an enum (e.g.

[Qemu-block] [PULL 08/12] blockjob: group BlockJob transaction functions together

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini Yet another pure code movement patch, preparing for the next change. Reviewed-by: Stefan Hajnoczi Signed-off-by: Paolo Bonzini Message-id: 20170508141310.8674-9-pbonz...@redhat.com Signed-off-by: Jeff Cody --- blockjob.c | 128 ++---

[Qemu-block] [PULL 05/12] blockjob: separate monitor and blockjob APIs

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini We have two different headers for block job operations, blockjob.h and blockjob_int.h. The former contains APIs called by the monitor, the latter contains APIs called by the block job drivers and the block layer itself. Keep the two APIs separate in the blockjob.c file too.

[Qemu-block] [PULL 03/12] blockjob: introduce block_job_early_fail

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini Outside blockjob.c, block_job_unref is only used when a block job fails to start, and block_job_ref is not used at all. The reference counting thus is pretty well hidden. Introduce a separate function to be used by block jobs; because block_job_ref and block_job_unref now be

[Qemu-block] [PULL 06/12] blockjob: move iostatus reset inside block_job_user_resume

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini Outside blockjob.c, the block_job_iostatus_reset function is used once in the monitor and once in BlockBackend. When we introduce the block job mutex, block_job_iostatus_reset's client is going to be the block layer (for which blockjob.c will take the block job mutex) rather

[Qemu-block] [PULL 09/12] blockjob: strengthen a bit test-blockjob-txn

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini Unlike test-blockjob-txn, QMP releases the reference to the transaction before the jobs finish. Thus, qemu-iotest 124 showed a failure while working on the next patch that the unit tests did not have. Make the test a little nastier. Reviewed-by: Stefan Hajnoczi Reviewed-by

[Qemu-block] [PULL 02/12] blockjob: remove iostatus_reset callback

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini This is unused since commit 66a0fae ("blockjob: Don't touch BDS iostatus", 2016-05-19). Reviewed-by: Stefan Hajnoczi Reviewed-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Paolo Bonzini Reviewed-by: Jeff Cody Message-id: 20170508141310.8674-3-pbonz...@

[Qemu-block] [PULL 04/12] blockjob: introduce block_job_pause/resume_all

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini Remove use of block_job_pause/resume from outside blockjob.c, thus making them static. The new functions are used by the block layer, so place them in blockjob_int.h. Reviewed-by: Stefan Hajnoczi Reviewed-by: John Snow Signed-off-by: Paolo Bonzini Reviewed-by: Jeff Cody

[Qemu-block] [PULL 07/12] blockjob: introduce block_job_cancel_async, check iostatus invariants

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini The new functions helps respecting the invariant that the coroutine is entered with false user_resume, zero pause count and no error recorded in the iostatus. Resetting the iostatus is now common to all of block_job_cancel_async, block_job_user_resume and block_job_iostatus_r

[Qemu-block] [PULL 12/12] block/gluster: glfs_lseek() workaround

2017-05-26 Thread Jeff Cody
On current released versions of glusterfs, glfs_lseek() will sometimes return invalid values for SEEK_DATA or SEEK_HOLE. For SEEK_DATA and SEEK_HOLE, the returned value should be >= the passed offset, or < 0 in the case of error: LSEEK(2): off_t lseek(int fd, off_t offset, int whence);

[Qemu-block] [PULL 10/12] blockjob: reorganize block_job_completed_txn_abort

2017-05-26 Thread Jeff Cody
From: Paolo Bonzini This splits the part that touches job states from the part that invokes callbacks. It will make the code simpler to understand once job states will be protected by a different mutex than the AioContext lock. Reviewed-by: Stefan Hajnoczi Signed-off-by: Paolo Bonzini Message

Re: [Qemu-block] [Qemu-devel] Behavior of QMP "query-block"

2017-05-26 Thread Kevin Wolf
Am 26.05.2017 um 18:55 hat John Snow geschrieben: > On 05/26/2017 07:40 AM, Kevin Wolf wrote: > > There is one completely crazy thing that Xen does with respect to disks. > > Instead of having support for their PV disks (i.e. virtio-blk, just > > different) in the BIOS, they add _both_ an IDE disk

[Qemu-block] [PATCH 00/29] qed: Convert to coroutines

2017-05-26 Thread Kevin Wolf
The qed block driver is one of the last remaining block drivers that use the AIO callback style interfaces. This series converts it to the coroutine model that other drivers are using and removes some AIO functions from the block layer API afterwards. If this isn't compelling enough, the diffstat

[Qemu-block] [PATCH 04/29] qed: Remove callback from qed_read_l2_table()

2017-05-26 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- block/qed-cluster.c | 94 ++--- block/qed-table.c | 15 +++-- block/qed.h | 3 +- 3 files changed, 36 insertions(+), 76 deletions(-) diff --git a/block/qed-cluster.c b/block/qed-cluster.c index 8f5da74..

[Qemu-block] [PATCH 10/29] qed: Remove callback from qed_write_header()

2017-05-26 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- block/qed.c | 36 ++-- 1 file changed, 14 insertions(+), 22 deletions(-) diff --git a/block/qed.c b/block/qed.c index 5548475..134c98a 100644 --- a/block/qed.c +++ b/block/qed.c @@ -98,8 +98,7 @@ int qed_write_header_sync(BDRVQEDState

[Qemu-block] [PATCH 07/29] qed: Make qed_copy_from_backing_file() synchronous

2017-05-26 Thread Kevin Wolf
Note that this code is generally not running in coroutine context, so this is an actual blocking synchronous operation. We'll fix this in a moment. Signed-off-by: Kevin Wolf --- block/qed.c | 78 +++-- 1 file changed, 29 insertions(+), 49 d

[Qemu-block] [PATCH 01/29] qed: Use bottom half to resume waiting requests

2017-05-26 Thread Kevin Wolf
The qed driver serialises allocating write requests. When the active allocation is finished, the AIO callback is called, but after this, the next allocating request is immediately processed instead of leaving the coroutine. Resuming another allocation request in the same request coroutine means tha

[Qemu-block] [PATCH 09/29] qed: Make qed_write_header() synchronous

2017-05-26 Thread Kevin Wolf
Note that this code is generally not running in coroutine context, so this is an actual blocking synchronous operation. We'll fix this in a moment. Signed-off-by: Kevin Wolf --- block/qed.c | 76 +++-- 1 file changed, 29 insertions(+), 47 d

[Qemu-block] [PATCH 05/29] qed: Remove callback from qed_find_cluster()

2017-05-26 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- block/qed-cluster.c | 39 ++- block/qed.c | 20 +++- block/qed.h | 4 ++-- 3 files changed, 35 insertions(+), 28 deletions(-) diff --git a/block/qed-cluster.c b/block/qed-cluster.c index d279944..8

[Qemu-block] [PATCH 13/29] qed: Remove callback from qed_write_table()

2017-05-26 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- block/qed-table.c | 47 --- block/qed.c | 12 +++- block/qed.h | 8 +++- 3 files changed, 22 insertions(+), 45 deletions(-) diff --git a/block/qed-table.c b/block/qed-table.c index 0cc93a7..ebee2c5

[Qemu-block] [PATCH 08/29] qed: Remove callback from qed_copy_from_backing_file()

2017-05-26 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- block/qed.c | 57 +++-- 1 file changed, 23 insertions(+), 34 deletions(-) diff --git a/block/qed.c b/block/qed.c index 584a5ba..61ef732 100644 --- a/block/qed.c +++ b/block/qed.c @@ -861,13 +861,9 @@ static int qed

[Qemu-block] [PATCH 03/29] qed: Remove callback from qed_read_table()

2017-05-26 Thread Kevin Wolf
Instead of passing the return value to a callback, return it to the caller so that the callback can be inlined there. Signed-off-by: Kevin Wolf --- block/qed-table.c | 79 ++- 1 file changed, 25 insertions(+), 54 deletions(-) diff --git a/bloc

[Qemu-block] [PATCH 17/29] qed: Add return value to qed_aio_write_l1_update()

2017-05-26 Thread Kevin Wolf
Don't recurse into qed_aio_next_io() and qed_aio_complete() here, but just return an error code and let the caller handle it. Signed-off-by: Kevin Wolf --- block/qed.c | 19 +-- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/block/qed.c b/block/qed.c index 3b1cce4

[Qemu-block] [PATCH 02/29] qed: Make qed_read_table() synchronous

2017-05-26 Thread Kevin Wolf
Note that this code is generally not running in coroutine context, so this is an actual blocking synchronous operation. We'll fix this in a moment. Signed-off-by: Kevin Wolf --- block/qed-table.c | 56 ++- 1 file changed, 18 insertions(+), 38 d

[Qemu-block] [PATCH 06/29] qed: Make qed_read_backing_file() synchronous

2017-05-26 Thread Kevin Wolf
Note that this code is generally not running in coroutine context, so this is an actual blocking synchronous operation. We'll fix this in a moment. Signed-off-by: Kevin Wolf --- block/qed.c | 32 ++-- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/blo

[Qemu-block] [PATCH 20/29] qed: Add return value to qed_aio_write_cow()

2017-05-26 Thread Kevin Wolf
Don't recurse into qed_aio_next_io() and qed_aio_complete() here, but just return an error code and let the caller handle it. While refactoring qed_aio_write_alloc() to accomodate the change, qed_aio_write_zero_cluster() ended up with a single line, so I chose to inline that line and remove the fu

[Qemu-block] [PATCH 11/29] qed: Make qed_write_table() synchronous

2017-05-26 Thread Kevin Wolf
Note that this code is generally not running in coroutine context, so this is an actual blocking synchronous operation. We'll fix this in a moment. Signed-off-by: Kevin Wolf --- block/qed-table.c | 84 --- 1 file changed, 30 insertions(+), 54 d

[Qemu-block] [PATCH 14/29] qed: Make qed_aio_read_data() synchronous

2017-05-26 Thread Kevin Wolf
Note that this code is generally not running in coroutine context, so this is an actual blocking synchronous operation. We'll fix this in a moment. Signed-off-by: Kevin Wolf --- block/qed.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/block/qed.c b/block/qed.c inde

[Qemu-block] [PATCH 12/29] qed: Remove GenericCB

2017-05-26 Thread Kevin Wolf
The GenericCB infrastructure isn't used any more. Remove it. Signed-off-by: Kevin Wolf --- block/Makefile.objs | 2 +- block/qed-gencb.c | 33 - block/qed.h | 11 --- 3 files changed, 1 insertion(+), 45 deletions(-) delete mode 100644 block/qed

[Qemu-block] [PATCH 25/29] qed: Implement .bdrv_co_readv/writev

2017-05-26 Thread Kevin Wolf
Most of the qed code is now synchronous and matches the coroutine model. One notable exception is the serialisation between requests which can still schedule a callback. Before we can replace this with coroutine locks, let's convert the driver's external interfaces to the coroutine versions. We ne

[Qemu-block] [PATCH 15/29] qed: Make qed_aio_write_main() synchronous

2017-05-26 Thread Kevin Wolf
Note that this code is generally not running in coroutine context, so this is an actual blocking synchronous operation. We'll fix this in a moment. Signed-off-by: Kevin Wolf --- block/qed.c | 61 +++-- 1 file changed, 19 insertions(+), 42 d

[Qemu-block] [PATCH 18/29] qed: Add return value to qed_aio_write_l2_update()

2017-05-26 Thread Kevin Wolf
Don't recurse into qed_aio_next_io() and qed_aio_complete() here, but just return an error code and let the caller handle it. Signed-off-by: Kevin Wolf --- block/qed.c | 43 ++- 1 file changed, 26 insertions(+), 17 deletions(-) diff --git a/block/qed.c b/

[Qemu-block] [PATCH 16/29] qed: Inline qed_commit_l2_update()

2017-05-26 Thread Kevin Wolf
qed_commit_l2_update() is unconditionally called at the end of qed_aio_write_l1_update(). Inline it. Signed-off-by: Kevin Wolf --- block/qed.c | 36 ++-- 1 file changed, 14 insertions(+), 22 deletions(-) diff --git a/block/qed.c b/block/qed.c index a596c4d..3b1cc

[Qemu-block] [PATCH 28/29] qed: Use a coroutine for need_check_timer

2017-05-26 Thread Kevin Wolf
This fixes the last place where we degraded from AIO to actual blocking synchronous I/O requests. Putting it into a coroutine means that instead of blocking, the coroutine simply yields while doing I/O. Signed-off-by: Kevin Wolf --- block/qed.c | 33 + 1 file chan

Re: [Qemu-block] [Qemu-devel] [PATCH 01/29] qed: Use bottom half to resume waiting requests

2017-05-26 Thread Eric Blake
On 05/26/2017 03:21 PM, Kevin Wolf wrote: > The qed driver serialises allocating write requests. When the active > allocation is finished, the AIO callback is called, but after this, the > next allocating request is immediately processed instead of leaving the > coroutine. Resuming another allocati

[Qemu-block] [PATCH 19/29] qed: Add return value to qed_aio_write_main()

2017-05-26 Thread Kevin Wolf
Don't recurse into qed_aio_next_io() and qed_aio_complete() here, but just return an error code and let the caller handle it. Signed-off-by: Kevin Wolf --- block/qed.c | 55 ++- 1 file changed, 30 insertions(+), 25 deletions(-) diff --git a/bl

[Qemu-block] [PATCH 23/29] qed: Remove ret argument from qed_aio_next_io()

2017-05-26 Thread Kevin Wolf
All callers pass ret = 0, so we can just remove it. Signed-off-by: Kevin Wolf --- block/qed.c | 17 ++--- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/block/qed.c b/block/qed.c index 3a26fdf..078591b 100644 --- a/block/qed.c +++ b/block/qed.c @@ -253,11 +253,11 @@ s

[Qemu-block] [PATCH 24/29] qed: Remove recursion in qed_aio_next_io()

2017-05-26 Thread Kevin Wolf
Instead of calling itself recursively as the last thing, just convert qed_aio_next_io() into a loop. This patch is best reviewed with 'git show -w' because most of it is just whitespace changes. Signed-off-by: Kevin Wolf --- block/qed.c | 63 +++--

[Qemu-block] [PATCH 22/29] qed: Add return value to qed_aio_read/write_data()

2017-05-26 Thread Kevin Wolf
Don't recurse into qed_aio_next_io() and qed_aio_complete() here, but just return an error code and let the caller handle it. Signed-off-by: Kevin Wolf --- block/qed.c | 72 ++--- block/qed.h | 21 -- 2 files changed, 31 ins

[Qemu-block] [PATCH 26/29] qed: Use CoQueue for serialising allocations

2017-05-26 Thread Kevin Wolf
Now that we're running in coroutine context, the ad-hoc serialisation code (which drops a request that has to wait out of coroutine context) can be replaced by a CoQueue. This means that when we resume a serialised request, it is running in coroutine context again and its I/O isn't blocking any mo

[Qemu-block] [PATCH 21/29] qed: Add return value to qed_aio_write_inplace/alloc()

2017-05-26 Thread Kevin Wolf
Don't recurse into qed_aio_next_io() and qed_aio_complete() here, but just return an error code and let the caller handle it. Signed-off-by: Kevin Wolf --- block/qed.c | 43 --- 1 file changed, 20 insertions(+), 23 deletions(-) diff --git a/block/qed.c b/

[Qemu-block] [PATCH 27/29] qed: Simplify request handling

2017-05-26 Thread Kevin Wolf
Now that we process a request in the same coroutine from beginning to end and don't drop out of it any more, we can look like a proper coroutine-based driver and simply call qed_aio_next_io() and get a return value from it instead of spawning an additional coroutine that reenters the parent when it

[Qemu-block] [PATCH 29/29] block: Remove bdrv_aio_readv/writev_flush()

2017-05-26 Thread Kevin Wolf
These functions are unused now. Signed-off-by: Kevin Wolf --- block/io.c| 171 -- include/block/block.h | 8 --- 2 files changed, 179 deletions(-) diff --git a/block/io.c b/block/io.c index fdd7485..1fec424 100644 --- a/block/io.c ++

Re: [Qemu-block] [Qemu-devel] [PATCH 02/29] qed: Make qed_read_table() synchronous

2017-05-26 Thread Eric Blake
On 05/26/2017 03:21 PM, Kevin Wolf wrote: > Note that this code is generally not running in coroutine context, so > this is an actual blocking synchronous operation. We'll fix this in a > moment. > > Signed-off-by: Kevin Wolf > --- > block/qed-table.c | 56 > ++--

Re: [Qemu-block] [Qemu-devel] [PATCH 03/29] qed: Remove callback from qed_read_table()

2017-05-26 Thread Eric Blake
On 05/26/2017 03:21 PM, Kevin Wolf wrote: > Instead of passing the return value to a callback, return it to the > caller so that the callback can be inlined there. > > Signed-off-by: Kevin Wolf > --- > block/qed-table.c | 79 > ++- > 1 file ch

Re: [Qemu-block] [Qemu-devel] [PATCH 04/29] qed: Remove callback from qed_read_l2_table()

2017-05-26 Thread Eric Blake
On 05/26/2017 03:21 PM, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > block/qed-cluster.c | 94 > ++--- > block/qed-table.c | 15 +++-- > block/qed.h | 3 +- > 3 files changed, 36 insertions(+), 76 deletions(-) The diffstats

Re: [Qemu-block] [Qemu-devel] [PATCH 05/29] qed: Remove callback from qed_find_cluster()

2017-05-26 Thread Eric Blake
On 05/26/2017 03:21 PM, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > block/qed-cluster.c | 39 ++- > block/qed.c | 20 +++- > block/qed.h | 4 ++-- > 3 files changed, 35 insertions(+), 28 deletions(-) > This diffstat

Re: [Qemu-block] [Qemu-devel] [PATCH 06/29] qed: Make qed_read_backing_file() synchronous

2017-05-26 Thread Eric Blake
On 05/26/2017 03:21 PM, Kevin Wolf wrote: > Note that this code is generally not running in coroutine context, so > this is an actual blocking synchronous operation. We'll fix this in a > moment. > > Signed-off-by: Kevin Wolf > --- > block/qed.c | 32 ++-- > 1 file ch

Re: [Qemu-block] [Qemu-devel] [PATCH 07/29] qed: Make qed_copy_from_backing_file() synchronous

2017-05-26 Thread Eric Blake
On 05/26/2017 03:21 PM, Kevin Wolf wrote: > Note that this code is generally not running in coroutine context, so > this is an actual blocking synchronous operation. We'll fix this in a > moment. > > Signed-off-by: Kevin Wolf > --- > block/qed.c | 78 > +++---

Re: [Qemu-block] [Qemu-devel] [PATCH 08/29] qed: Remove callback from qed_copy_from_backing_file()

2017-05-26 Thread Eric Blake
On 05/26/2017 03:21 PM, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > block/qed.c | 57 +++-- > 1 file changed, 23 insertions(+), 34 deletions(-) > > /** > - * Populate back untouched region of new data cluster > + * Populate untouche

Re: [Qemu-block] [Qemu-devel] [PATCH 09/29] qed: Make qed_write_header() synchronous

2017-05-26 Thread Eric Blake
On 05/26/2017 03:21 PM, Kevin Wolf wrote: > Note that this code is generally not running in coroutine context, so > this is an actual blocking synchronous operation. We'll fix this in a > moment. > > Signed-off-by: Kevin Wolf > --- > block/qed.c | 76 > +++---

Re: [Qemu-block] [Qemu-devel] Behavior of QMP "query-block"

2017-05-26 Thread Konrad Rzeszutek Wilk
On Fri, May 26, 2017 at 10:02:32PM +0200, Kevin Wolf wrote: > Am 26.05.2017 um 18:55 hat John Snow geschrieben: > > On 05/26/2017 07:40 AM, Kevin Wolf wrote: > > > There is one completely crazy thing that Xen does with respect to disks. > > > Instead of having support for their PV disks (i.e. virti

[Qemu-block] [PATCH] nbd: Fully initialize client in case of failed negotiation

2017-05-26 Thread Eric Blake
If a non-NBD client connects to qemu-nbd, we would end up with a SIGSEGV in nbd_cilent_put() because we were trying to unregister the client's association to the export, even though we skipped inserting the client into that list. Easy trigger in two terminals: $ qemu-nbd -p 30001 --format=raw fil