Re: [Qemu-block] [PATCH 1/2] block: Fix dst total_sectors after copy offloading

2018-07-04 Thread Kevin Wolf
Am 04.07.2018 um 08:13 hat Fam Zheng geschrieben: > This was noticed by the new image fleecing tests case 222. The issue is > apparent and we should just do the same right things as in > bdrv_aligned_pwritev. > > Reported-by: John Snow > Signed-off-by: Fam Zheng > --- a/block/io.c > +++ b/block

Re: [Qemu-block] [PATCH 1/2] block: Fix dst total_sectors after copy offloading

2018-07-04 Thread Fam Zheng
On Wed, 07/04 09:44, Kevin Wolf wrote: > Am 04.07.2018 um 08:13 hat Fam Zheng geschrieben: > > This was noticed by the new image fleecing tests case 222. The issue is > > apparent and we should just do the same right things as in > > bdrv_aligned_pwritev. > > > > Reported-by: John Snow > > Signed

Re: [Qemu-block] [PATCH 2/2] block/backup: fix fleecing scheme: use serialized writes

2018-07-04 Thread Kevin Wolf
Am 03.07.2018 um 20:07 hat Vladimir Sementsov-Ogievskiy geschrieben: > Fleecing scheme works as follows: we want a kind of temporary snapshot > of active drive A. We create temporary image B, with B->backing = A. > Then we start backup(sync=none) from A to B. From this point, B reads > as point-in-

[Qemu-block] [PATCH 1/2] nbd/server: fix nbd_co_send_block_status

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
Call nbd_co_send_extents() with correct length parameter (extent.length may be smaller than original length). Also, switch length parameter type to uint32_t, to correspond with request->len and similar nbd_co_send_bitmap(). Signed-off-by: Vladimir Sementsov-Ogievskiy --- nbd/server.c | 5 +++--

[Qemu-block] [PATCH 0/2] nbd base:allocation several extents

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
Hi all! Patch 01 is a fix an should definitely go to 3.0. Patch 02 is an improvement, so as you want, 3.1 is ok. The idea is send more than one extent of base:allocation context, if possible, which is necessary for efficient block-status export, for clients which support it. Vladimir Sementsov-O

[Qemu-block] [PATCH 2/2] nbd/server: send more than one extent of base:allocation context

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
This is necessary for efficient block-status export, for clients which support it. Signed-off-by: Vladimir Sementsov-Ogievskiy --- nbd/server.c | 77 +--- 1 file changed, 58 insertions(+), 19 deletions(-) diff --git a/nbd/server.c b/nbd/se

[Qemu-block] [PATCH] block: Don't silently truncate node names

2018-07-04 Thread Kevin Wolf
If the user passes a too long node name string, we silently truncate it to fit into BlockDriverState.node_name, i.e. to 31 characters. Apart from surprising the user when the node has a different name than requested, this also bypasses the check for duplicate names, so that the same name can be ass

Re: [Qemu-block] [PATCH 2/2] block/backup: fix fleecing scheme: use serialized writes

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
04.07.2018 13:39, Kevin Wolf wrote: Am 03.07.2018 um 20:07 hat Vladimir Sementsov-Ogievskiy geschrieben: Fleecing scheme works as follows: we want a kind of temporary snapshot of active drive A. We create temporary image B, with B->backing = A. Then we start backup(sync=none) from A to B. From t

[Qemu-block] [PATCH] block/crypto: Fix memory leak in create error path

2018-07-04 Thread Kevin Wolf
Fixes: Coverity CID 1393782 Signed-off-by: Kevin Wolf --- block/crypto.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/crypto.c b/block/crypto.c index 994172a3de..146d81c90a 100644 --- a/block/crypto.c +++ b/block/crypto.c @@ -551,7 +551,7 @@ static int coroutine_fn b

Re: [Qemu-block] [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial

2018-07-04 Thread Cornelia Huck
On Tue, 3 Jul 2018 13:32:29 +0200 Kevin Wolf wrote: > > > > Has serial/gemoetry been fixed meanwhile and will it make it into the > > > > next release? > > > > > > I cannot find an archive that has it, but it is on the libvirt mailing > > > list as "[libvirt] [PATCH v3] qemu: format serial and

Re: [Qemu-block] [Qemu-devel] [PATCH] block/crypto: Fix memory leak in create error path

2018-07-04 Thread Daniel P . Berrangé
On Wed, Jul 04, 2018 at 02:58:02PM +0200, Kevin Wolf wrote: > Fixes: Coverity CID 1393782 > Signed-off-by: Kevin Wolf > --- > block/crypto.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/block/crypto.c b/block/crypto.c > index 994172a3de..146d81c90a 100644 > --- a/bloc

Re: [Qemu-block] [PATCH 2/2] block/backup: fix fleecing scheme: use serialized writes

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
04.07.2018 13:39, Kevin Wolf wrote: Am 03.07.2018 um 20:07 hat Vladimir Sementsov-Ogievskiy geschrieben: Fleecing scheme works as follows: we want a kind of temporary snapshot of active drive A. We create temporary image B, with B->backing = A. Then we start backup(sync=none) from A to B. From t

Re: [Qemu-block] [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial

2018-07-04 Thread Kevin Wolf
Am 04.07.2018 um 15:02 hat Cornelia Huck geschrieben: > On Tue, 3 Jul 2018 13:32:29 +0200 > Kevin Wolf wrote: > > > > > > Has serial/gemoetry been fixed meanwhile and will it make it into the > > > > > next release? > > > > > > > > I cannot find an archive that has it, but it is on the libvirt

Re: [Qemu-block] [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial

2018-07-04 Thread Daniel P . Berrangé
On Wed, Jul 04, 2018 at 03:34:40PM +0200, Kevin Wolf wrote: > Am 04.07.2018 um 15:02 hat Cornelia Huck geschrieben: > > On Tue, 3 Jul 2018 13:32:29 +0200 > > Kevin Wolf wrote: > > > > > > > > Has serial/gemoetry been fixed meanwhile and will it make it into > > > > > > the > > > > > > next relea

Re: [Qemu-block] [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial

2018-07-04 Thread Christian Borntraeger
On 07/04/2018 03:34 PM, Kevin Wolf wrote: > Am 04.07.2018 um 15:02 hat Cornelia Huck geschrieben: >> On Tue, 3 Jul 2018 13:32:29 +0200 >> Kevin Wolf wrote: >> >> Has serial/gemoetry been fixed meanwhile and will it make it into the >> next release? > > I cannot find an archive

Re: [Qemu-block] [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial

2018-07-04 Thread Cornelia Huck
On Wed, 4 Jul 2018 15:34:40 +0200 Kevin Wolf wrote: > Am 04.07.2018 um 15:02 hat Cornelia Huck geschrieben: > > On Tue, 3 Jul 2018 13:32:29 +0200 > > Kevin Wolf wrote: > > > > > > > > Has serial/gemoetry been fixed meanwhile and will it make it into > > > > > > the > > > > > > next release?

Re: [Qemu-block] [PATCH v2 2/3] block/fleecing-filter: new filter driver for fleecing

2018-07-04 Thread Max Reitz
On 2018-07-03 13:15, Kevin Wolf wrote: > Am 02.07.2018 um 14:09 hat Vladimir Sementsov-Ogievskiy geschrieben: >> 29.06.2018 20:40, Eric Blake wrote: >>> On 06/29/2018 12:30 PM, John Snow wrote: On 06/29/2018 11:15 AM, Vladimir Sementsov-Ogievskiy wrote: > We need to synchronize b

Re: [Qemu-block] [PATCH 0/4] block: Trivial fixes in offloading code

2018-07-04 Thread Max Reitz
On 2018-07-03 03:26, Fam Zheng wrote: > On Mon, 07/02 14:35, Max Reitz wrote: >> On 2018-07-02 04:58, Fam Zheng wrote: >>> These are the low priority ones spotted by Kevin and Max last week. >>> >>> Fam Zheng (4): >>> qcow2: Drop unused cluster_data >>> file-posix: Fix fd_open check in raw_co_c

Re: [Qemu-block] [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial

2018-07-04 Thread Kevin Wolf
Am 04.07.2018 um 15:43 hat Daniel P. Berrangé geschrieben: > On Wed, Jul 04, 2018 at 03:34:40PM +0200, Kevin Wolf wrote: > > I understand where you're coming from, but let's be honest: It's not as > > if disk geometry or serial numbers were features that absolutely need > > to be there to give QEMU

Re: [Qemu-block] [PATCH 1/2] block: add BDRV_REQ_SERIALISING flag

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
03.07.2018 21:07, Vladimir Sementsov-Ogievskiy wrote: Serialized writes should be used in copy-on-write of backup(sync=none) for image fleecing scheme. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 5 - block/io.c| 4 2 files changed, 8 inserti

[Qemu-block] [PATCH 3/3] block/blklogwrites: Add an option for the update interval of the log superblock

2018-07-04 Thread Ari Sundholm
This is a way to ensure that the log superblock is periodically updated. Before, this was only done on flush requests, which may not be enough if the VM exits abnormally, omitting the final flush. The default interval is 4096 write requests. Signed-off-by: Ari Sundholm --- block/blklogwrites.c

[Qemu-block] [PATCH 1/3] block/blklogwrites: Change log_sector_size from int64_t to uint64_t

2018-07-04 Thread Ari Sundholm
This was a simple oversight when working on intermediate versions of the original patch which introduced blklogwrites. Signed-off-by: Ari Sundholm --- block/blklogwrites.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/block/blklogwrites.c b/block/blklogwrites.c inde

[Qemu-block] [PATCH 0/3] blklogwrites improvements

2018-07-04 Thread Ari Sundholm
This patch set contains a trivial printf format fix, followed by two new features for the blklogwrites driver. The first feature is an option to append to an existing log, adopting its sector size and other metadata in its superblock. This can be useful, and was suggested by Kevin Wolf. The secon

[Qemu-block] [PATCH 2/3] block/blklogwrites: Add an option for appending to an old log

2018-07-04 Thread Ari Sundholm
Suggested by Kevin Wolf. May be useful when testing multiple batches of writes or doing long-term testing involving restarts of the VM. Signed-off-by: Ari Sundholm --- block/blklogwrites.c | 147 ++- qapi/block-core.json | 3 +- 2 files changed,

[Qemu-block] [PATCH 0/2] file-posix: Fix creation locking

2018-07-04 Thread Max Reitz
This series fixes two bugs Kevin spotted in file-posix's creation locking. Max Reitz (2): file-posix: Fix creation locking file-posix: Unlock FD after creation block/file-posix.c | 21 - 1 file changed, 16 insertions(+), 5 deletions(-) -- 2.17.1

[Qemu-block] [PATCH 1/2] file-posix: Fix creation locking

2018-07-04 Thread Max Reitz
raw_apply_lock_bytes() takes a bit mask of "permissions that are NOT shared". Also, make the "perm" and "shared" variables uint64_t, because I do not particularly like using ~ on signed integers (and other permission masks are usually uint64_t, too). Reported-by: Kevin Wolf Signed-off-by: Max Re

[Qemu-block] [PATCH 2/2] file-posix: Unlock FD after creation

2018-07-04 Thread Max Reitz
Closing the FD does not necessarily mean that it is unlocked. Fix this by relinquishing all permission locks before qemu_close(). Reported-by: Kevin Wolf Signed-off-by: Max Reitz --- block/file-posix.c | 17 ++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/block

[Qemu-block] [PATCH] throttle-groups: fix hang when group member leaves

2018-07-04 Thread Stefan Hajnoczi
Throttle groups consist of members sharing one throttling state (including bps/iops limits). Round-robin scheduling is used to ensure fairness. If a group member already has a timer pending then other groups members do not schedule their own timers. The next group member will have its turn when

[Qemu-block] [PATCH v2 0/3] blklogwrites improvements

2018-07-04 Thread Ari Sundholm
This patch set contains a trivial printf format fix, followed by two new features for the blklogwrites driver. The first feature is an option to append to an existing log, adopting its sector size and other metadata in its superblock. This can be useful, and was suggested by Kevin Wolf. The secon

[Qemu-block] [PATCH v2 3/3] block/blklogwrites: Add an option for the update interval of the log superblock

2018-07-04 Thread Ari Sundholm
This is a way to ensure that the log superblock is periodically updated. Before, this was only done on flush requests, which may not be enough if the VM exits abnormally, omitting the final flush. The default interval is 4096 write requests. Signed-off-by: Ari Sundholm --- block/blklogwrites.c

[Qemu-block] [PATCH v2 2/3] block/blklogwrites: Add an option for appending to an old log

2018-07-04 Thread Ari Sundholm
Suggested by Kevin Wolf. May be useful when testing multiple batches of writes or doing long-term testing involving restarts of the VM. Signed-off-by: Ari Sundholm --- block/blklogwrites.c | 147 ++- qapi/block-core.json | 3 +- 2 files changed,

[Qemu-block] [PATCH v2 1/3] block/blklogwrites: Change log_sector_size from int64_t to uint64_t

2018-07-04 Thread Ari Sundholm
This was a simple oversight when working on intermediate versions of the original patch which introduced blklogwrites. Signed-off-by: Ari Sundholm --- block/blklogwrites.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/block/blklogwrites.c b/block/blklogwrites.c inde

[Qemu-block] [PATCH] block/crypto: Simplify block_crypto_co_create_opts_luks to avoid a memory leak

2018-07-04 Thread Philippe Mathieu-Daudé
After 1ec4f4160a1 Coverity reported: Variable cryptoopts going out of scope leaks the storage it points to. Fixes: Coverity CID 1393782 (Resource leak) Signed-off-by: Philippe Mathieu-Daudé --- I think this check is superfluous but I respected the previous code: ret = block_crypto_co_cre

Re: [Qemu-block] [PATCH 1/2] block: add BDRV_REQ_SERIALISING flag

2018-07-04 Thread Kevin Wolf
Am 04.07.2018 um 16:44 hat Vladimir Sementsov-Ogievskiy geschrieben: > 03.07.2018 21:07, Vladimir Sementsov-Ogievskiy wrote: > > Serialized writes should be used in copy-on-write of backup(sync=none) > > for image fleecing scheme. > > > > Signed-off-by: Vladimir Sementsov-Ogievskiy > > --- > >

Re: [Qemu-block] [Qemu-devel] [PATCH] block: Don't silently truncate node names

2018-07-04 Thread no-reply
Hi, This series failed build test on s390x host. Please find the details below. N/A. Internal error while reading log file --- Email generated automatically by Patchew [http://patchew.org/]. Please send your feedback to patchew-de...@redhat.com

[Qemu-block] [PATCH 1/8] qobject: Catch another straggler for use of qdict_put_str()

2018-07-04 Thread Philippe Mathieu-Daudé
Patch created mechanically by rerunning: $ spatch --sp-file scripts/coccinelle/qobject.cocci \ --macro-file scripts/cocci-macro-file.h \ --dir . --in-place Signed-off-by: Philippe Mathieu-Daudé --- qobject/block-qdict.c | 2 +- 1 file changed, 1 insertion(+), 1 deleti

[Qemu-block] [PATCH 0/8] Coccinelle cleanups

2018-07-04 Thread Philippe Mathieu-Daudé
Nothing exciting here, patches created mechanically (common after soft freeze). Philippe Mathieu-Daudé (8): qobject: Catch another straggler for use of qdict_put_str() error: Remove NULL checks on error_propagate() calls crypto: Remove useless casts xen: Remove useless casts tests/bios-t

Re: [Qemu-block] [PATCH 1/2] block: add BDRV_REQ_SERIALISING flag

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
04.07.2018 18:08, Kevin Wolf wrote: Am 04.07.2018 um 16:44 hat Vladimir Sementsov-Ogievskiy geschrieben: 03.07.2018 21:07, Vladimir Sementsov-Ogievskiy wrote: Serialized writes should be used in copy-on-write of backup(sync=none) for image fleecing scheme. Signed-off-by: Vladimir Sementsov-Ogi

Re: [Qemu-block] [Qemu-devel] [libvirt] [PULL 25/26] block: Remove deprecated -drive option serial

2018-07-04 Thread Peter Maydell
On 4 July 2018 at 14:34, Kevin Wolf wrote: > Essentially, what is important to me isn't getting these options dropped > exactly in 3.0, but not setting a bad precedence that deprecation isn't > actually worth anything. We may easily end up with this deprecation > process: > > depreate a feature >

Re: [Qemu-block] [PATCH 1/2] block: add BDRV_REQ_SERIALISING flag

2018-07-04 Thread Kevin Wolf
Am 04.07.2018 um 18:11 hat Vladimir Sementsov-Ogievskiy geschrieben: > 04.07.2018 18:08, Kevin Wolf wrote: > > Am 04.07.2018 um 16:44 hat Vladimir Sementsov-Ogievskiy geschrieben: > > > 03.07.2018 21:07, Vladimir Sementsov-Ogievskiy wrote: > > > > Serialized writes should be used in copy-on-write o

Re: [Qemu-block] [PATCH 1/2] block: add BDRV_REQ_SERIALISING flag

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
04.07.2018 19:20, Kevin Wolf wrote: Am 04.07.2018 um 18:11 hat Vladimir Sementsov-Ogievskiy geschrieben: 04.07.2018 18:08, Kevin Wolf wrote: Am 04.07.2018 um 16:44 hat Vladimir Sementsov-Ogievskiy geschrieben: 03.07.2018 21:07, Vladimir Sementsov-Ogievskiy wrote: Serialized writes should be u

Re: [Qemu-block] [PATCH 1/2] block: add BDRV_REQ_SERIALISING flag

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
04.07.2018 19:36, Vladimir Sementsov-Ogievskiy wrote: 04.07.2018 19:20, Kevin Wolf wrote: Am 04.07.2018 um 18:11 hat Vladimir Sementsov-Ogievskiy geschrieben: 04.07.2018 18:08, Kevin Wolf wrote: Am 04.07.2018 um 16:44 hat Vladimir Sementsov-Ogievskiy geschrieben: 03.07.2018 21:07, Vladimir Se

[Qemu-block] [PATCH v2 4/4] block/backup: fix fleecing scheme: use serialized writes

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
Fleecing scheme works as follows: we want a kind of temporary snapshot of active drive A. We create temporary image B, with B->backing = A. Then we start backup(sync=none) from A to B. From this point, B reads as point-in-time snapshot of A (A continues to be active drive, accepting guest IO). Thi

[Qemu-block] [PATCH v2 2/4] block: split flags in copy_range

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
Pass read flags and write flags separately. This is needed to handle coming BDRV_REQ_NO_SERIALISING clearly in following patches. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 3 ++- include/block/block_int.h | 14 ++ include/sysemu/block-back

[Qemu-block] [PATCH v2 0/4] fix image fleecing

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
Hi all. This fixes image fleecing scheme for 3.0, details are in 04 patch. v2: 01,02: new patches 03: - improve comment - fix assert in bdrv_aligned_pwritev - add asserts to not use the flag on read requests - support copy_range 04: - expand "detected cases" range - s/fleecing/

[Qemu-block] [PATCH v2 3/4] block: add BDRV_REQ_SERIALISING flag

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
Serialized writes should be used in copy-on-write of backup(sync=none) for image fleecing scheme. We need to change an assert in bdrv_aligned_pwritev, added in 28de2dcd88de. The assert may fail now, because call to wait_serialising_requests here may become first call to it for this request with se

[Qemu-block] [PATCH v2 1/4] block: disallow BDRV_REQ_NO_SERIALISING for write

2018-07-04 Thread Vladimir Sementsov-Ogievskiy
Before commit 9ded4a01149 "backup: Use copy offloading", BDRV_REQ_NO_SERIALISING was used for only one case: read in copy-on-write operation during backup. Also, the flag was handled only on read path (in bdrv_co_preadv and bdrv_aligned_preadv). After 9ded4a01149, flag is used for not waiting seri

Re: [Qemu-block] [PULL 0/5] Block layer patches

2018-07-04 Thread Peter Maydell
On 3 July 2018 at 15:59, Kevin Wolf wrote: > The following changes since commit a395717cbd26e7593d3c3fe81faca121ec6d13e8: > > Merge remote-tracking branch 'remotes/cody/tags/block-pull-request' into > staging (2018-07-03 11:49:51 +0100) > > are available in the git repository at: > > git://re

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/4] fix image fleecing

2018-07-04 Thread no-reply
Hi, This series failed build test on s390x host. Please find the details below. Type: series Message-id: 20180704175006.519184-1-vsement...@virtuozzo.com Subject: [Qemu-devel] [PATCH v2 0/4] fix image fleecing === TEST SCRIPT BEGIN === #!/bin/bash # Testing script will be invoked under the git c

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/4] fix image fleecing

2018-07-04 Thread no-reply
Hi, This series failed docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 20180704175006.519184-1-vsement...@virtuozzo.com Subject: [Qemu-devel] [PATCH v2 0/4] fix

[Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-04 Thread Nir Soffer
I'm trying to access a qcow2 image via /dev/nbdX device. I'm connecting a fedora 27 image (created using virt-builder) on a block device: qemu-nbd -c /dev/nbd0 -f qcow2 -n --detect-zeroes=on --aio=native /dev/27837a03-64f9-4f2b-abb0-daa2195b01ae/acb196da-31fc-454d-856b-c31ab24715b3 It works, and

Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

2018-07-04 Thread Nir Soffer
On Thu, Jul 5, 2018 at 2:10 AM Nir Soffer wrote: > I'm trying to access a qcow2 image via /dev/nbdX device. > > I'm connecting a fedora 27 image (created using virt-builder) on a block > device: > > qemu-nbd -c /dev/nbd0 -f qcow2 -n --detect-zeroes=on --aio=native > /dev/27837a03-64f9-4f2b-abb0-d

Re: [Qemu-block] [Qemu-devel] [PATCH 1/8] qobject: Catch another straggler for use of qdict_put_str()

2018-07-04 Thread Markus Armbruster
Philippe Mathieu-Daudé writes: > Patch created mechanically by rerunning: > > $ spatch --sp-file scripts/coccinelle/qobject.cocci \ > --macro-file scripts/cocci-macro-file.h \ > --dir . --in-place > > Signed-off-by: Philippe Mathieu-Daudé > --- > qobject/block-qdict.c