Re: [Qemu-block] [Qemu-devel] [PATCH] block/iscsi: allow caching of the allocation map

2016-05-18 Thread Peter Lieven
Am 18.05.2016 um 09:41 schrieb Fam Zheng: On Tue, 05/17 15:59, Peter Lieven wrote: until now the allocation map was used only as a hint if a cluster is allocated or not. If a block was not allocated (or Qemu had no info about the allocation status) a get_block_status call was issued to check the

Re: [Qemu-block] [PATCH 2/2] block/nfs: refuse readahead if cache.direct is on

2016-05-18 Thread Peter Lieven
Am 18.05.2016 um 15:28 schrieb Jeff Cody: On Tue, May 17, 2016 at 04:11:55PM +0200, Peter Lieven wrote: Signed-off-by: Peter Lieven --- block/nfs.c | 5 + 1 file changed, 5 insertions(+) diff --git a/block/nfs.c b/block/nfs.c index 975510f..8b73a35 100644 --- a/block/nfs.c +++ b/block/n

[Qemu-block] [PATCH v4 28/28] qemu-img: Use new JSON output formatter

2016-05-18 Thread Eric Blake
Now that we can pretty-print straight to JSON from a visitor, we can eliminate the temporary conversion into QObject inside qemu-img. The changes to qemu-iotests 043 expected output demonstrates the fact that output is now done in qapi declaration order, rather than QDict hash order. Signed-off-b

[Qemu-block] [PATCH v4 22/28] qobject: Consolidate qobject_to_json() calls

2016-05-18 Thread Eric Blake
It's simpler to have a single conversion function that takes a bool parameter, rather than two functions where the choice of function determines an internal bool. Similar to commit fc471c18. While at it, the conversion currently cannot fail (maybe it SHOULD be possible to choose to fail, when enc

[Qemu-block] [PATCH v4 12/28] qapi: Add new visit_complete() function

2016-05-18 Thread Eric Blake
Making each output visitor provide its own output collection function was the only remaining reason for exposing visitor sub-types to the rest of the code base. Add a polymorphic visit_complete() function which is a no-op for input visitors, and which populates an opaque pointer for output visitor

[Qemu-block] [PATCH v4 21/28] qstring: Add qstring_wrap_str()

2016-05-18 Thread Eric Blake
Several spots in the code malloc a string, then wrap it in a QString, which has to duplicate the input. Adding a new constructor with transfer semantics makes this pattern more efficient, comparable to the just-added transfer semantics to go from QString back to raw string. Use the new qstring_wr

[Qemu-block] [PATCH v4 04/28] qapi: Add parameter to visit_end_*

2016-05-18 Thread Eric Blake
Rather than making the dealloc visitor track of stack of pointers remembered during visit_start_* in order to free them during visit_end_*, it's a lot easier to just make all callers pass the same pointer to visit_end_*. The generated code has access to the same pointer, while all other users are

[Qemu-block] [PATCH v4 10/28] qmp-output-visitor: Favor new visit_free() function

2016-05-18 Thread Eric Blake
Now that we have a polymorphic visit_free(), we no longer need qmp_output_visitor_cleanup(); however, we still need to expose the subtype for qmp_output_get_qobject(). Signed-off-by: Eric Blake --- v4: new patch --- include/qapi/qmp-output-visitor.h | 1 - block/qapi.c |

[Qemu-block] [PATCH v4 03/28] qemu-img: Don't leak errors when outputting JSON

2016-05-18 Thread Eric Blake
If our JSON output ever encounters an error, we would just silently leak the local variable. Instead, assert that our usage won't fail. Signed-off-by: Eric Blake --- v4: new patch (split out from v3 14/18) --- qemu-img.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --

[Qemu-block] [PATCH v4 06/28] opts-visitor: Favor new visit_free() function

2016-05-18 Thread Eric Blake
Now that we have a polymorphic visit_free(), we no longer need opts_visitor_cleanup(); which in turn means we no longer need to return a subtype from opts_visitor_new() nor a public upcast function. Signed-off-by: Eric Blake --- v4: new patch --- include/qapi/opts-visitor.h | 4 +--- block/cry

[Qemu-block] [PATCH v4 01/28] qapi: Rename (one) qjson.h to qobject-json.h

2016-05-18 Thread Eric Blake
We have two different JSON visitors in the tree; and having both named 'qjson.h' can cause include confusion. Rename the qapi version. Why did I pick that one? A later patch plans on deleting the top-level qjson.c once we have a native JSON output visitor; we could have renamed that one for less

[Qemu-block] [PATCH v2 2/2] iotests: Add regression test for drive-backup cluster size error

2016-05-18 Thread Fam Zheng
Signed-off-by: Fam Zheng --- tests/qemu-iotests/055 | 13 + tests/qemu-iotests/055.out | 4 ++-- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/055 b/tests/qemu-iotests/055 index c8e3578..7f990c2 100755 --- a/tests/qemu-iotests/055 +++ b/tests/

[Qemu-block] [PATCH v2 0/2] backup: Fail early if cannot determine cluster size

2016-05-18 Thread Fam Zheng
v2: More accurately, s:hang:trigger assertion failure: in the bug fix patch commit message. Add John's and Jeff's R-b. Add a test. [Eric] Fam Zheng (2): backup: Fail early if cannot determine cluster size iotests: Add regression test for drive-backup cluster size error block/ba

[Qemu-block] [PATCH v2 1/2] backup: Fail early if cannot determine cluster size

2016-05-18 Thread Fam Zheng
Otherwise the job is orphaned and block_job_cancel_sync in bdrv_close_all() when quitting will trigger assertion failure. Cc: qemu-sta...@nongnu.org Reviewed-by: John Snow Reviewed-by: Jeff Cody Signed-off-by: Fam Zheng --- block/backup.c | 34 ++ 1 file changed

Re: [Qemu-block] [Qemu-devel] [PATCH v2 1/1] block: clarify error message for qmp-eject

2016-05-18 Thread Fam Zheng
On Wed, 05/18 17:53, John Snow wrote: > If you use HMP's eject but the CDROM tray is locked, you may get a > confusing error message informing you that the "tray isn't open." > > As this is the point of eject, we can do a little better and help > clarify that the tray was locked and that it (might

Re: [Qemu-block] [Qemu-devel] [PATCH] backup: Fail early if cannot determine cluster size

2016-05-18 Thread Fam Zheng
On Wed, 05/18 08:25, Eric Blake wrote: > On 05/17/2016 11:53 PM, Fam Zheng wrote: > > Otherwise the job is orphaned and block_job_cancel_sync in > > bdrv_close_all() when quiting will hang. > > s/quiting/quitting/ > > > > > A simple reproducer is running blockdev-backup from null-co:// to > > nu

Re: [Qemu-block] [PATCH v2 1/1] block: clarify error message for qmp-eject

2016-05-18 Thread Eric Blake
On 05/18/2016 03:53 PM, John Snow wrote: > If you use HMP's eject but the CDROM tray is locked, you may get a > confusing error message informing you that the "tray isn't open." > > As this is the point of eject, we can do a little better and help > clarify that the tray was locked and that it (mi

[Qemu-block] [PATCH v2 0/1] block: clarify error message for qmp-eject

2016-05-18 Thread John Snow
v2: Reduce helper to just one parameter, push has_force logic back up into qmp interfaces. Always return -errno if we set errp, return +errno on soft errors where errp remains unset. John Snow (1): block: clarify error message for qmp-eject blockdev.c | 51

[Qemu-block] [PATCH v2 1/1] block: clarify error message for qmp-eject

2016-05-18 Thread John Snow
If you use HMP's eject but the CDROM tray is locked, you may get a confusing error message informing you that the "tray isn't open." As this is the point of eject, we can do a little better and help clarify that the tray was locked and that it (might) open up later, so try again. It's not ideal,

Re: [Qemu-block] [Qemu-devel] [PATCH] block/rbd: add .bdrv_reopen_prepare() stub

2016-05-18 Thread Jason Dillaman
On Wed, May 18, 2016 at 4:19 AM, Kevin Wolf wrote: >> Updating this setting on an open image won't do anything, but if you >> rbd_close() and rbd_open() it again the setting will take effect. >> rbd_close() will force a flush of any pending I/O in librbd and >> free the memory for librbd's ImageCt

Re: [Qemu-block] [PATCH v2 3/3] qemu-iotests: Some more write_zeroes tests

2016-05-18 Thread Kevin Wolf
Am 18.05.2016 um 16:12 hat Eric Blake geschrieben: > On 05/18/2016 06:27 AM, Kevin Wolf wrote: > > This covers some more write_zeroes cases which are relevant for the > > recent qcow2 optimisations that check the allocation status of the > > backing file for partial cluster write_zeroes requests. >

Re: [Qemu-block] [Qemu-devel] [PATCH] backup: Fail early if cannot determine cluster size

2016-05-18 Thread Eric Blake
On 05/17/2016 11:53 PM, Fam Zheng wrote: > Otherwise the job is orphaned and block_job_cancel_sync in > bdrv_close_all() when quiting will hang. s/quiting/quitting/ > > A simple reproducer is running blockdev-backup from null-co:// to > null-co://. Worth a qemu-iotests addition? > > Cc: qemu-

Re: [Qemu-block] [Qemu-devel] [PATCH] block: clarify error message for qmp-eject

2016-05-18 Thread John Snow
On 05/18/2016 01:36 AM, Markus Armbruster wrote: > Fam Zheng writes: > >> On Tue, 05/17 20:42, John Snow wrote: >>> If you use HMP's eject but the CDROM tray is locked, you may get a >>> confusing error message informing you that the "tray isn't open." >>> >>> As this is the point of eject, we

Re: [Qemu-block] [Qemu-devel] [PATCH] block: clarify error message for qmp-eject

2016-05-18 Thread John Snow
On 05/18/2016 02:36 AM, Fam Zheng wrote: > On Wed, 05/18 07:36, Markus Armbruster wrote: >> Fam Zheng writes: >> >>> On Tue, 05/17 20:42, John Snow wrote: If you use HMP's eject but the CDROM tray is locked, you may get a confusing error message informing you that the "tray isn't open.

Re: [Qemu-block] [PATCH v2 3/3] qemu-iotests: Some more write_zeroes tests

2016-05-18 Thread Eric Blake
On 05/18/2016 06:27 AM, Kevin Wolf wrote: > This covers some more write_zeroes cases which are relevant for the > recent qcow2 optimisations that check the allocation status of the > backing file for partial cluster write_zeroes requests. > > This needs to be separate from 034 because we can only

Re: [Qemu-block] [Qemu-devel] [PATCH] backup: Fail early if cannot determine cluster size

2016-05-18 Thread John Snow
On 05/18/2016 01:53 AM, Fam Zheng wrote: > Otherwise the job is orphaned and block_job_cancel_sync in > bdrv_close_all() when quiting will hang. > Tch, mea culpa. > A simple reproducer is running blockdev-backup from null-co:// to > null-co://. > > Cc: qemu-sta...@nongnu.org > Signed-off-by:

Re: [Qemu-block] [PATCH] backup: Fail early if cannot determine cluster size

2016-05-18 Thread Jeff Cody
On Wed, May 18, 2016 at 01:53:27PM +0800, Fam Zheng wrote: > Otherwise the job is orphaned and block_job_cancel_sync in > bdrv_close_all() when quiting will hang. > > A simple reproducer is running blockdev-backup from null-co:// to > null-co://. > > Cc: qemu-sta...@nongnu.org > Signed-off-by: Fa

Re: [Qemu-block] [PATCH 2/2] block/nfs: refuse readahead if cache.direct is on

2016-05-18 Thread Jeff Cody
On Tue, May 17, 2016 at 04:11:55PM +0200, Peter Lieven wrote: > Signed-off-by: Peter Lieven > --- > block/nfs.c | 5 + > 1 file changed, 5 insertions(+) > > diff --git a/block/nfs.c b/block/nfs.c > index 975510f..8b73a35 100644 > --- a/block/nfs.c > +++ b/block/nfs.c > @@ -331,6 +331,11 @@ s

Re: [Qemu-block] [Qemu-devel] [PATCH] block: clarify error message for qmp-eject

2016-05-18 Thread Markus Armbruster
Fam Zheng writes: > On Wed, 05/18 07:36, Markus Armbruster wrote: >> Fam Zheng writes: >> >> > On Tue, 05/17 20:42, John Snow wrote: >> >> If you use HMP's eject but the CDROM tray is locked, you may get a >> >> confusing error message informing you that the "tray isn't open." >> >> >> >> As t

[Qemu-block] [PATCH v2 3/3] qemu-iotests: Some more write_zeroes tests

2016-05-18 Thread Kevin Wolf
This covers some more write_zeroes cases which are relevant for the recent qcow2 optimisations that check the allocation status of the backing file for partial cluster write_zeroes requests. This needs to be separate from 034 because we can only support qcow2 in this test case for multiple reasons

Re: [Qemu-block] [PATCH 3/3] qemu-iotests: Some more write_zeroes tests

2016-05-18 Thread Kevin Wolf
Am 17.05.2016 um 22:35 hat Eric Blake geschrieben: > No test of crossing a cluster boundary at this point? I would probably do: > > $QEMU_IO -c "write -z 27k 2k" "$TEST_IMG" | _filter_qemu_io > > then check that the map shows an allocation starting at 24k for 8192 bytes > What you have looks fi

[Qemu-block] [RFC] backup: export interfaces for extra serialization

2016-05-18 Thread Changlong Xie
On 05/06/2016 11:46 PM, Stefan Hajnoczi wrote: Did you run stress tests where the primary is writing to the disk while the secondary reads from the same sectors? I thought about this some more and I'm wondering about the following scenario: NBD writes to secondary_disk and the guest reads from

Re: [Qemu-block] [Qemu-devel] [PATCH v18 7/8] Implement new driver for block replication

2016-05-18 Thread Changlong Xie
On 05/06/2016 11:46 PM, Stefan Hajnoczi wrote: On Fri, Apr 15, 2016 at 04:10:37PM +0800, Changlong Xie wrote: +static void replication_close(BlockDriverState *bs) +{ +BDRVReplicationState *s = bs->opaque; + +if (s->mode == REPLICATION_MODE_SECONDARY) { +g_free(s->top_id); +}

Re: [Qemu-block] [PATCH 0/3] Drop virtio-{blk,scsi} op blockers

2016-05-18 Thread Michael S. Tsirkin
On Wed, May 18, 2016 at 04:24:07PM +0800, Fam Zheng wrote: > We are ready to get rid of dataplane's op blockers altogether. Most operations > are already unblocked in virtio-blk, and those remained for virtio-scsi only > because we haven't got around to add counterpart unblocking code. > > The fir

Re: [Qemu-block] [PATCH 2/3] virtio-blk: Remove op blocker for dataplane

2016-05-18 Thread Michael S. Tsirkin
On Wed, May 18, 2016 at 04:24:09PM +0800, Fam Zheng wrote: > Block layer is prepared to unspecialize dataplane, an evidence is this > almost complete list of unblocked operations. It has all types except > two (actually three if DATAPLANE itself counts but blockdev.c makes sure > attaching twice is

Re: [Qemu-block] [PATCH 3/3] virtio-scsi: Remove op blocker for dataplane

2016-05-18 Thread Michael S. Tsirkin
On Wed, May 18, 2016 at 04:24:10PM +0800, Fam Zheng wrote: > The previous patch dropped all op blockers from virtio-blk data plane. > The situation of virtio-scsi is exactly the same it can drop them too. > > Signed-off-by: Fam Zheng Acked-by: Michael S. Tsirkin > --- > hw/scsi/virtio-scsi.c

[Qemu-block] [PATCH 2/3] virtio-blk: Remove op blocker for dataplane

2016-05-18 Thread Fam Zheng
Block layer is prepared to unspecialize dataplane, an evidence is this almost complete list of unblocked operations. It has all types except two (actually three if DATAPLANE itself counts but blockdev.c makes sure attaching twice is not possible): MIRROR_TARGET and BACKUP_TARGET. blockdev-mirror r

[Qemu-block] [PATCH 1/3] blockdev-backup: Don't move target AioContext if it's attached

2016-05-18 Thread Fam Zheng
If the BDS is attached, it will want to stay on the AioContext where its BlockBackend is. Don't call bdrv_set_aio_context in this case. Signed-off-by: Fam Zheng --- blockdev.c | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index 1892b8e..

[Qemu-block] [PATCH 3/3] virtio-scsi: Remove op blocker for dataplane

2016-05-18 Thread Fam Zheng
The previous patch dropped all op blockers from virtio-blk data plane. The situation of virtio-scsi is exactly the same it can drop them too. Signed-off-by: Fam Zheng --- hw/scsi/virtio-scsi.c | 62 - include/hw/virtio/virtio-scsi.h | 11

[Qemu-block] [PATCH 0/3] Drop virtio-{blk,scsi} op blockers

2016-05-18 Thread Fam Zheng
We are ready to get rid of dataplane's op blockers altogether. Most operations are already unblocked in virtio-blk, and those remained for virtio-scsi only because we haven't got around to add counterpart unblocking code. The first patch fixes an existing bug with blockdev-backup. Then the op bloc

Re: [Qemu-block] [PATCH] block/rbd: add .bdrv_reopen_prepare() stub

2016-05-18 Thread Kevin Wolf
Am 17.05.2016 um 20:48 hat Josh Durgin geschrieben: > On 05/17/2016 03:03 AM, Sebastian Färber wrote: > >Hi Kevin, > > > >>A correct reopen implementation must consider all options and flags that > >>.bdrv_open() looked at. > >> > >>The options are okay, as both "filename" and "password-secret" are

Re: [Qemu-block] [Qemu-devel] [PATCH 0/2] block/nfs: add support for libnfs pagecache

2016-05-18 Thread Fam Zheng
On Tue, 05/17 16:11, Peter Lieven wrote: > this adds support for the upcoming libnfs cachepage to Qemu. > While at it neglect to use readahead if cache.direct is on. > > Peter Lieven (2): > block/nfs: add support for libnfs pagecache > block/nfs: refuse readahead if cache.direct is on > > bl

Re: [Qemu-block] [Qemu-devel] [PATCH] block/iscsi: allow caching of the allocation map

2016-05-18 Thread Fam Zheng
On Tue, 05/17 15:59, Peter Lieven wrote: > until now the allocation map was used only as a hint if a cluster > is allocated or not. If a block was not allocated (or Qemu had > no info about the allocation status) a get_block_status call was > issued to check the allocation status and possibly avoid

Re: [Qemu-block] [PATCH] block/rbd: add .bdrv_reopen_prepare() stub

2016-05-18 Thread Sebastian Färber
> > There's no need to reset the librados state, so connections to the > cluster can stick around. I'm a bit unclear on the bdrv_reopen_* > functions though - what is their intended use and semantics? My motivation for implementing this basic reopen support is getting active block commit in qemu