Re: [Qemu-block] [PATCH v10 10/12] migration: add postcopy migration of dirty bitmaps

2018-03-13 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > 12.03.2018 19:09, Dr. David Alan Gilbert wrote: > > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > > > Postcopy migration of dirty bitmaps. Only named dirty bitmaps are > > > migrated. > > > > > > + > > > +

[Qemu-block] [PATCH v11 09/13] migration: add is_active_iterate handler

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
Only-postcopy savevm states (dirty-bitmap) don't need live iteration, so to disable them and stop transporting empty sections there is a new savevm handler. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Juan Quintela Reviewed-by: John

[Qemu-block] [PATCH v11 13/13] iotests: add dirty bitmap postcopy test

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
Test - start two vms (vm_a, vm_b) - in a - do writes from set A - do writes from set B - fix bitmap sha256 - clear bitmap - do writes from set A - start migration - than, in b - wait vm start (postcopy should start) - do writes from set B - check bitmap sha256

[Qemu-block] [PATCH v11 07/13] migration: include migrate_dirty_bitmaps in migrate_postcopy

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
Enable postcopy if dirty bitmap migration is enabled. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Juan Quintela Reviewed-by: John Snow Reviewed-by: Fam Zheng --- migration/migration.c | 2 +- 1

[Qemu-block] [PATCH v11 01/13] block/dirty-bitmap: add bdrv_dirty_bitmap_enable_successor()

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
Enabling bitmap successor is necessary to enable successors of bitmaps being migrated before target vm start. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Fam Zheng Message-id:

Re: [Qemu-block] [PATCH v11 05/13] migration: introduce postcopy-only pending

2018-03-13 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > There would be savevm states (dirty-bitmap) which can migrate only in > postcopy stage. The corresponding pending is introduced here. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Dr.

[Qemu-block] [PATCH v11 10/13] migration: allow qmp command migrate-start-postcopy for any postcopy

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
Allow migrate-start-postcopy for any postcopy type Signed-off-by: Vladimir Sementsov-Ogievskiy --- migration/migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 094196c236..59b4fe6090

Re: [Qemu-block] [PATCH 2/2] docs: Document the new default sizes of the qcow2 caches

2018-03-13 Thread Eric Blake
On 03/13/2018 10:02 AM, Alberto Garcia wrote: We have just reduced the refcount cache size to the minimum unless the user explicitly requests a larger one, so we have to update the documentation to reflect this change. Signed-off-by: Alberto Garcia --- docs/qcow2-cache.txt

Re: [Qemu-block] [PATCH v11 10/13] migration: allow qmp command migrate-start-postcopy for any postcopy

2018-03-13 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > Allow migrate-start-postcopy for any postcopy type > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Dr. David Alan Gilbert > --- > migration/migration.c | 2 +- > 1

[Qemu-block] [PATCH v11 06/13] qapi: add dirty-bitmaps migration capability

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Juan Quintela Reviewed-by: Fam Zheng --- qapi/migration.json | 6 +-

[Qemu-block] [PATCH v11 04/13] dirty-bitmap: add locked state

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
Add special state, when qmp operations on the bitmap are disabled. It is needed during bitmap migration. "Frozen" state is not appropriate here, because it looks like bitmap is unchanged. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow

[Qemu-block] [PATCH v11 12/13] iotests: add dirty bitmap migration test

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
The test starts two vms (vm_a, vm_b), create dirty bitmap in the first one, do several writes to corresponding device and then migrate vm_a to vm_b. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/169 | 156

[Qemu-block] [PATCH v11 11/13] migration: add postcopy migration of dirty bitmaps

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
Postcopy migration of dirty bitmaps. Only named dirty bitmaps are migrated. If destination qemu is already containing a dirty bitmap with the same name as a migrated bitmap (for the same node), then, if their granularities are the same the migration will be done, otherwise the error will be

Re: [Qemu-block] [PATCH 1/2] qcow2: Give the refcount cache the minimum possible size by default

2018-03-13 Thread Eric Blake
On 03/13/2018 10:02 AM, Alberto Garcia wrote: The L2 and refcount caches have default sizes that can be overriden using the l2-cache-size and refcount-cache-size (an additional parameter named cache-size sets the combined size of both caches). Unless forced by one of the aforementioned

Re: [Qemu-block] [Qemu-devel] [PULL 18/41] blockjobs: add block-job-finalize

2018-03-13 Thread Eric Blake
On 03/13/2018 11:17 AM, Kevin Wolf wrote: From: John Snow Instead of automatically transitioning from PENDING to CONCLUDED, gate the .prepare() and .commit() phases behind an explicit acknowledgement provided by the QMP monitor if auto_finalize = false has been requested.

Re: [Qemu-block] [PATCH v11 11/13] migration: add postcopy migration of dirty bitmaps

2018-03-13 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > Postcopy migration of dirty bitmaps. Only named dirty bitmaps are migrated. > > If destination qemu is already containing a dirty bitmap with the same name > as a migrated bitmap (for the same node), then, if their granularities

Re: [Qemu-block] [PATCH v11 11/13] migration: add postcopy migration of dirty bitmaps

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
13.03.2018 21:22, Dr. David Alan Gilbert wrote: * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: Postcopy migration of dirty bitmaps. Only named dirty bitmaps are migrated. If destination qemu is already containing a dirty bitmap with the same name as a migrated bitmap (for the

Re: [Qemu-block] [Qemu-devel] [PATCH v11 00/13] Dirty bitmaps postcopy migration

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
It looks like a bug in a recent commit to checkpatch. It don't support do { } while Best regards, Vladimir. От: no-re...@patchew.org Отправлено: 13 марта 2018 г. 22:03:29 Кому: Vladimir Sementsov-Ogievskiy Копия: f...@redhat.com;

Re: [Qemu-block] [Qemu-devel] [PATCH v11 00/13] Dirty bitmaps postcopy migration

2018-03-13 Thread Eric Blake
On 03/13/2018 03:01 PM, Vladimir Sementsov-Ogievskiy wrote: It looks like a bug in a recent commit to checkpatch. It don't support do { } while Checking PATCH 11/13: migration: add postcopy migration of dirty bitmaps... ERROR: braces {} are necessary for all arms of this statement #737:

Re: [Qemu-block] [Qemu-devel] [PATCH v11 00/13] Dirty bitmaps postcopy migration

2018-03-13 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > It looks like a bug in a recent commit to checkpatch. It don't support do { } > while Yes, adding Su Hang and Eric in and trimming some others out. So yes, ignore this patchew failure for this case, but we need to fix that

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/8] nbd block status base:allocation

2018-03-13 Thread Eric Blake
On 03/12/2018 10:11 PM, Eric Blake wrote:    CC  block/nbd-client.o    CC  block/sheepdog.o /var/tmp/patchew-tester-tmp-erqpie2w/src/block/nbd-client.c: In function ‘nbd_client_co_block_status’: /var/tmp/patchew-tester-tmp-erqpie2w/src/block/nbd-client.c:890:15: error:

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/8] nbd block status base:allocation

2018-03-13 Thread no-reply
Hi, This series failed docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 20180312152126.286890-1-vsement...@virtuozzo.com Subject: [Qemu-devel] [PATCH v2 0/8] nbd

Re: [Qemu-block] [Qemu-devel] [PATCH v2 resend] block/mirror: change the semantic of 'force' of block-job-cancel

2018-03-13 Thread John Snow
On 03/12/2018 10:13 PM, Jeff Cody wrote: > On Fri, Mar 02, 2018 at 11:20:55AM -0500, John Snow wrote: >> >> >> On 03/02/2018 10:39 AM, Eric Blake wrote: >>> On 02/26/2018 08:05 PM, Liang Li wrote: When doing drive mirror to a low speed shared storage, if there was heavy BLK IO write

Re: [Qemu-block] [PATCH v10 05/12] migration: introduce postcopy-only pending

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
12.03.2018 18:30, Dr. David Alan Gilbert wrote: * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: There would be savevm states (dirty-bitmap) which can migrate only in postcopy stage. The corresponding pending is introduced here. Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH] iotests: Fix stuck NBD process on 33

2018-03-13 Thread Anton Nefedov
On 13/3/2018 12:12 AM, Max Reitz wrote: On 2018-03-12 22:11, Eric Blake wrote: Commit afe35cde6 added additional actions to test 33, but forgot to reset the image between tests. As a result, './check -nbd 33' fails because the qemu-nbd process from the first half is still occupying the port,

[Qemu-block] [PATCH v2 2/2] iotests: Add regression test for commit base locking

2018-03-13 Thread Fam Zheng
Signed-off-by: Fam Zheng --- tests/qemu-iotests/153 | 8 tests/qemu-iotests/153.out | 4 2 files changed, 12 insertions(+) diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153 index adfd02695b..a7875e6899 100755 --- a/tests/qemu-iotests/153 +++

[Qemu-block] [PATCH v2 0/2] block: Fix permission during reopen

2018-03-13 Thread Fam Zheng
v2: Use update_flags_from_options. [Kevin] We write open the whole backing chain during reopen. It is not necessary and will cause image locking problems if the backing image is shared. Fam Zheng (2): block: Fix flags in reopen queue iotests: Add regression test for commit base locking

[Qemu-block] Question: an IO hang problem

2018-03-13 Thread sochin . jiang
Hi, guys, Recently, I encountered an IO hang problem in occasion which I cannot reproduce it now. I analyzed this problem carefully, the critical stack is as following: After reading the codes in linux-aio.c(see ioq_submit() function), I found two situations could lead us here. 1) no

Re: [Qemu-block] [PATCH 7/7] vpc: Support .bdrv_co_create

2018-03-13 Thread Kevin Wolf
Am 12.03.2018 um 22:49 hat Max Reitz geschrieben: > On 2018-03-09 22:46, Kevin Wolf wrote: > > This adds the .bdrv_co_create driver callback to vpc, which > > enables image creation over QMP. > > > > Signed-off-by: Kevin Wolf > > --- > > qapi/block-core.json | 33 ++-

Re: [Qemu-block] [PATCH] MAINTAINERS: add include/block/aio-wait.h

2018-03-13 Thread Stefan Hajnoczi
On Mon, Mar 12, 2018 at 01:22:04PM +, Stefan Hajnoczi wrote: > The include/block/aio-wait.h header file was added by commit > 7719f3c968c59e1bcda7e177679dc765b59e578f ("block: extract > AIO_WAIT_WHILE() from BlockDriverState") without updating MAINTAINERS. > > Signed-off-by: Stefan Hajnoczi

Re: [Qemu-block] [PATCH v10 05/12] migration: introduce postcopy-only pending

2018-03-13 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > 12.03.2018 18:30, Dr. David Alan Gilbert wrote: > > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > > > There would be savevm states (dirty-bitmap) which can migrate only in > > > postcopy stage. The

Re: [Qemu-block] [PULL 0/1] Block patches

2018-03-13 Thread Peter Maydell
On 12 March 2018 at 16:01, Stefan Hajnoczi wrote: > The following changes since commit e4ae62b802cec437f877f2cadc4ef059cc0eca76: > > Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' > into staging (2018-03-09 17:28:16 +) > > are available in the

Re: [Qemu-block] [Qemu-devel] [PATCH v10 05/12] migration: introduce postcopy-only pending

2018-03-13 Thread Dr. David Alan Gilbert
* John Snow (js...@redhat.com) wrote: > > > On 03/12/2018 11:30 AM, Dr. David Alan Gilbert wrote: > > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > >> There would be savevm states (dirty-bitmap) which can migrate only in > >> postcopy stage. The corresponding pending is

Re: [Qemu-block] [PATCH v2 2/2] iotests: Add regression test for commit base locking

2018-03-13 Thread Max Reitz
On 2018-03-13 12:58, Fam Zheng wrote: > Signed-off-by: Fam Zheng > --- > tests/qemu-iotests/153 | 8 > tests/qemu-iotests/153.out | 4 > 2 files changed, 12 insertions(+) > > diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153 > index

[Qemu-block] [PATCH v3 1/1] block/mirror: change the semantic of 'force' of block-job-cancel

2018-03-13 Thread Jeff Cody
From: Liang Li When doing drive mirror to a low speed shared storage, if there was heavy BLK IO write workload in VM after the 'ready' event, drive mirror block job can't be canceled immediately, it would keep running until the heavy BLK IO workload stopped in the

[Qemu-block] [PULL 1/1] block: include original filename when reporting invalid URIs

2018-03-13 Thread Jeff Cody
From: Daniel P. Berrangé Consider passing a JSON based block driver to "qemu-img commit" $ qemu-img commit 'json:{"driver":"qcow2","file":{"driver":"gluster",\ "volume":"gv0","path":"sn1.qcow2", "server":[{"type":\

Re: [Qemu-block] [PATCH v2 1/2] block: Fix flags in reopen queue

2018-03-13 Thread Max Reitz
On 2018-03-13 12:58, Fam Zheng wrote: > Reopen flags are not synchronized according to the > bdrv_reopen_queue_child precedence until bdrv_reopen_prepare. It is a > bit too late: we already check the consistency in bdrv_check_perm before > that. > > This fixes the bug that when bdrv_reopen a RO

Re: [Qemu-block] [PATCH v2 3/8] nbd: BLOCK_STATUS for standard get_block_status function: server part

2018-03-13 Thread Eric Blake
On 03/12/2018 10:21 AM, Vladimir Sementsov-Ogievskiy wrote: Minimal realization: only one extent in server answer is supported. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: - constants and type defs were splitted out by Eric, except for

[Qemu-block] [PULL 0/1] Block patches

2018-03-13 Thread Jeff Cody
The following changes since commit 834eddf22ec762839b724538c7be1d1d3b2d9d3b: Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2018-03-13 10:49:02 +) are available in the git repository at: git://github.com/codyprime/qemu-kvm-jtc.git

Re: [Qemu-block] [PATCH v10 05/12] migration: introduce postcopy-only pending

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
13.03.2018 16:11, Vladimir Sementsov-Ogievskiy wrote: 13.03.2018 13:30, Dr. David Alan Gilbert wrote: * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: 12.03.2018 18:30, Dr. David Alan Gilbert wrote: * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: There would

[Qemu-block] [PATCH v3 0/2] block: Fix permission during reopen

2018-03-13 Thread Fam Zheng
v3: Fix test case. [Max] v2: Use update_flags_from_options. [Kevin] We write open the whole backing chain during reopen. It is not necessary and will cause image locking problems if the backing image is shared. Fam Zheng (2): block: Fix flags in reopen queue iotests: Add regression test for

[Qemu-block] [PATCH v3 2/2] iotests: Add regression test for commit base locking

2018-03-13 Thread Fam Zheng
Signed-off-by: Fam Zheng --- tests/qemu-iotests/153 | 12 tests/qemu-iotests/153.out | 5 + 2 files changed, 17 insertions(+) diff --git a/tests/qemu-iotests/153 b/tests/qemu-iotests/153 index adfd02695b..a0fd815483 100755 --- a/tests/qemu-iotests/153 +++

[Qemu-block] [PATCH v3 1/2] block: Fix flags in reopen queue

2018-03-13 Thread Fam Zheng
Reopen flags are not synchronized according to the bdrv_reopen_queue_child precedence until bdrv_reopen_prepare. It is a bit too late: we already check the consistency in bdrv_check_perm before that. This fixes the bug that when bdrv_reopen a RO node as RW, the flags for backing child are wrong.

Re: [Qemu-block] [PATCH v10 05/12] migration: introduce postcopy-only pending

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
13.03.2018 13:30, Dr. David Alan Gilbert wrote: * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: 12.03.2018 18:30, Dr. David Alan Gilbert wrote: * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: There would be savevm states (dirty-bitmap) which can migrate only

Re: [Qemu-block] [PATCH 7/7] vpc: Support .bdrv_co_create

2018-03-13 Thread Max Reitz
On 2018-03-13 12:32, Kevin Wolf wrote: > Am 12.03.2018 um 22:49 hat Max Reitz geschrieben: >> On 2018-03-09 22:46, Kevin Wolf wrote: >>> This adds the .bdrv_co_create driver callback to vpc, which >>> enables image creation over QMP. >>> >>> Signed-off-by: Kevin Wolf >>> --- >>>

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] nbd: BLOCK_STATUS for standard get_block_status function: server part

2018-03-13 Thread Eric Blake
On 03/13/2018 08:47 AM, Eric Blake wrote: On 03/12/2018 10:21 AM, Vladimir Sementsov-Ogievskiy wrote: Minimal realization: only one extent in server answer is supported. Signed-off-by: Vladimir Sementsov-Ogievskiy --- +/* nbd_negotiate_meta_queries + * Handle

Re: [Qemu-block] [PATCH v3 0/2] block: Fix permission during reopen

2018-03-13 Thread Max Reitz
On 2018-03-13 15:20, Fam Zheng wrote: > v3: Fix test case. [Max] > > v2: Use update_flags_from_options. [Kevin] > > We write open the whole backing chain during reopen. It is not necessary and > will cause image locking problems if the backing image is shared. > > Fam Zheng (2): > block: Fix

[Qemu-block] [PATCH v2 3/8] qcow: Support .bdrv_co_create

2018-03-13 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to qcow, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Jeff Cody --- qapi/block-core.json | 21 +- block/qcow.c | 196

[Qemu-block] [PATCH 2/2] docs: Document the new default sizes of the qcow2 caches

2018-03-13 Thread Alberto Garcia
We have just reduced the refcount cache size to the minimum unless the user explicitly requests a larger one, so we have to update the documentation to reflect this change. Signed-off-by: Alberto Garcia --- docs/qcow2-cache.txt | 31 ++- 1 file

Re: [Qemu-block] [PATCH v3 1/2] block: Fix flags in reopen queue

2018-03-13 Thread Kevin Wolf
Am 13.03.2018 um 15:20 hat Fam Zheng geschrieben: > Reopen flags are not synchronized according to the > bdrv_reopen_queue_child precedence until bdrv_reopen_prepare. It is a > bit too late: we already check the consistency in bdrv_check_perm before > that. > > This fixes the bug that when

Re: [Qemu-block] [PATCH v3 0/2] block: Fix permission during reopen

2018-03-13 Thread Kevin Wolf
Am 13.03.2018 um 15:20 hat Fam Zheng geschrieben: > v3: Fix test case. [Max] > > v2: Use update_flags_from_options. [Kevin] > > We write open the whole backing chain during reopen. It is not necessary and > will cause image locking problems if the backing image is shared. Thanks, applied to the

[Qemu-block] [PATCH 0/2] Give the refcount cache the minimum possible size by default

2018-03-13 Thread Alberto Garcia
Hi, we talked about this the other day, so here are the patches to change the default cache sizes in qcow2. Without this patch: * refcount-cache-size = l2-cache-size / 4 unless otherwise specified by the user. This is wasteful, the refcount cache is accessed sequentially during normal I/O, so

[Qemu-block] [PATCH 1/2] qcow2: Give the refcount cache the minimum possible size by default

2018-03-13 Thread Alberto Garcia
The L2 and refcount caches have default sizes that can be overriden using the l2-cache-size and refcount-cache-size (an additional parameter named cache-size sets the combined size of both caches). Unless forced by one of the aforementioned parameters, QEMU will set the unspecified sizes so that

Re: [Qemu-block] [PATCH v2 8/8] vpc: Require aligned size in .bdrv_co_create

2018-03-13 Thread Max Reitz
On 2018-03-13 15:47, Kevin Wolf wrote: > Perform the rounding to match a CHS geometry only in the legacy code > path in .bdrv_co_create_opts. QMP now requires that the user already > passes a CHS aligned image size, unless force-size=true is given. > > CHS alignment is required to make the image

Re: [Qemu-block] [PATCH v2 5/8] nbd: BLOCK_STATUS for standard get_block_status function: client part

2018-03-13 Thread Eric Blake
On 03/12/2018 10:21 AM, Vladimir Sementsov-Ogievskiy wrote: Minimal realization: only one extent in server answer is supported. Flag NBD_CMD_FLAG_REQ_ONE is used to force this behavior. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: - drop iotests changes, as

Re: [Qemu-block] [PATCH v2 8/8] iotests: new test 209 for NBD BLOCK_STATUS

2018-03-13 Thread Eric Blake
On 03/12/2018 10:21 AM, Vladimir Sementsov-Ogievskiy wrote: Signed-off-by: Vladimir Sementsov-Ogievskiy --- + +iotests.verify_image_format(supported_fmts=['qcow2']) Interesting that './check -nbd' doesn't run 209 (because that defaults to format -raw, but we

Re: [Qemu-block] [PATCH v2 0/8] nbd block status base:allocation

2018-03-13 Thread Eric Blake
On 03/12/2018 10:21 AM, Vladimir Sementsov-Ogievskiy wrote: Hi all. Here is minimal realization of base:allocation context of NBD block-status extension, which allows to get block status through NBD. v2 changes are in each patch after "---" line. Vladimir Sementsov-Ogievskiy (8):

Re: [Qemu-block] [PATCH v3 1/1] block/mirror: change the semantic of 'force' of block-job-cancel

2018-03-13 Thread Kevin Wolf
Am 13.03.2018 um 13:12 hat Jeff Cody geschrieben: > From: Liang Li > > When doing drive mirror to a low speed shared storage, if there was heavy > BLK IO write workload in VM after the 'ready' event, drive mirror block job > can't be canceled immediately, it would

[Qemu-block] [PATCH v2 2/8] qemu-iotests: Enable write tests for parallels

2018-03-13 Thread Kevin Wolf
Originally we added parallels as a read-only format to qemu-iotests where we did just some tests with a binary image. Since then, write and image creation support has been added to the driver, so we can now enable it in _supported_fmt generic. The driver doesn't support migration yet, though, so

[Qemu-block] [PATCH v2 0/8] block: .bdrv_co_create for format drivers

2018-03-13 Thread Kevin Wolf
This series adds a .bdrv_co_create implementation to almost all format drivers that support creating images where its still missing. The only exception is VMDK because its support for extents will make the QAPI design a bit more complicated. The other format driver not covered in this series are

[Qemu-block] [PATCH v2 1/8] parallels: Support .bdrv_co_create

2018-03-13 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to parallels, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Jeff Cody --- qapi/block-core.json | 18 - block/parallels.c| 199

[Qemu-block] [PATCH v2 5/8] vdi: Make comments consistent with other drivers

2018-03-13 Thread Kevin Wolf
This makes the .bdrv_co_create(_opts) implementation of vdi look more like the other recently converted block drivers. Signed-off-by: Kevin Wolf --- block/vdi.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/block/vdi.c b/block/vdi.c index

[Qemu-block] [PATCH v2 4/8] qed: Support .bdrv_co_create

2018-03-13 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to qed, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- qapi/block-core.json | 25 ++- block/qed.c | 204

Re: [Qemu-block] [PATCH v2 6/8] vhdx: Support .bdrv_co_create

2018-03-13 Thread Max Reitz
On 2018-03-13 15:47, Kevin Wolf wrote: > This adds the .bdrv_co_create driver callback to vhdx, which > enables image creation over QMP. > > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 40 +- > block/vhdx.c | 216 >

Re: [Qemu-block] [PATCH v10 05/12] migration: introduce postcopy-only pending

2018-03-13 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > 13.03.2018 13:30, Dr. David Alan Gilbert wrote: > > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > > > 12.03.2018 18:30, Dr. David Alan Gilbert wrote: > > > > * Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH v2 7/8] vpc: Support .bdrv_co_create

2018-03-13 Thread Max Reitz
On 2018-03-13 15:47, Kevin Wolf wrote: > This adds the .bdrv_co_create driver callback to vpc, which > enables image creation over QMP. > > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 33 ++- > block/vpc.c | 152 >

Re: [Qemu-block] [PATCH v2 5/8] vdi: Make comments consistent with other drivers

2018-03-13 Thread Max Reitz
On 2018-03-13 15:47, Kevin Wolf wrote: > This makes the .bdrv_co_create(_opts) implementation of vdi look more > like the other recently converted block drivers. > > Signed-off-by: Kevin Wolf > --- > block/vdi.c | 12 +--- > 1 file changed, 9 insertions(+), 3

Re: [Qemu-block] [PATCH v2 0/8] block: .bdrv_co_create for format drivers

2018-03-13 Thread Kevin Wolf
Am 13.03.2018 um 15:47 hat Kevin Wolf geschrieben: > This series adds a .bdrv_co_create implementation to almost all format > drivers that support creating images where its still missing. The only > exception is VMDK because its support for extents will make the QAPI > design a bit more

[Qemu-block] [PATCH v2 6/8] vhdx: Support .bdrv_co_create

2018-03-13 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to vhdx, which enables image creation over QMP. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 40 +- block/vhdx.c | 216 ++- 2 files changed, 203

[Qemu-block] [PATCH v2 8/8] vpc: Require aligned size in .bdrv_co_create

2018-03-13 Thread Kevin Wolf
Perform the rounding to match a CHS geometry only in the legacy code path in .bdrv_co_create_opts. QMP now requires that the user already passes a CHS aligned image size, unless force-size=true is given. CHS alignment is required to make the image compatible with Virtual PC, but not for use with

[Qemu-block] [PATCH v2 7/8] vpc: Support .bdrv_co_create

2018-03-13 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to vpc, which enables image creation over QMP. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 33 ++- block/vpc.c | 152 ++- 2 files changed, 147

Re: [Qemu-block] [PATCH v2 7/8] iotests: add file_path helper

2018-03-13 Thread Eric Blake
On 03/12/2018 10:21 AM, Vladimir Sementsov-Ogievskiy wrote: Simple way to have auto generated filenames with auto cleanup. Like FilePath but without using 'with' statement and without additional indentation of the whole test. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PULL 07/41] blockjobs: add block_job_verb permission table

2018-03-13 Thread Kevin Wolf
From: John Snow Which commands ("verbs") are appropriate for jobs in which state is also somewhat burdensome to keep track of. As of this commit, it looks rather useless, but begins to look more interesting the more states we add to the STM table. A recurring theme is that no

[Qemu-block] [PULL 04/41] blockjobs: add status enum

2018-03-13 Thread Kevin Wolf
From: John Snow We're about to add several new states, and booleans are becoming unwieldly and difficult to reason about. It would help to have a more explicit bookkeeping of the state of blockjobs. To this end, add a new "status" field and add our existing states in a

[Qemu-block] [PULL 10/41] blockjobs: add NULL state

2018-03-13 Thread Kevin Wolf
From: John Snow Add a new state that specifically demarcates when we begin to permanently demolish a job after it has performed all work. This makes the transition explicit in the STM table and highlights conditions under which a job may be demolished. Alongside this state,

[Qemu-block] [PULL 21/41] tests/test-blockjob: test cancellations

2018-03-13 Thread Kevin Wolf
From: John Snow Whatever the state a blockjob is in, it should be able to be canceled by the block layer. Signed-off-by: John Snow Signed-off-by: Kevin Wolf --- tests/test-blockjob.c | 233

[Qemu-block] [PULL 14/41] blockjobs: add block_job_txn_apply function

2018-03-13 Thread Kevin Wolf
From: John Snow Simply apply a function transaction-wide. A few more uses of this in forthcoming patches. Signed-off-by: John Snow Signed-off-by: Kevin Wolf --- blockjob.c | 25 - 1 file changed, 16 insertions(+),

[Qemu-block] [PULL 05/41] blockjobs: add state transition table

2018-03-13 Thread Kevin Wolf
From: John Snow The state transition table has mostly been implied. We're about to make it a bit more complex, so let's make the STM explicit instead. Perform state transitions with a function that for now just asserts the transition is appropriate. Transitions: Undefined ->

[Qemu-block] [PULL 18/41] blockjobs: add block-job-finalize

2018-03-13 Thread Kevin Wolf
From: John Snow Instead of automatically transitioning from PENDING to CONCLUDED, gate the .prepare() and .commit() phases behind an explicit acknowledgement provided by the QMP monitor if auto_finalize = false has been requested. This allows us to perform graph changes in

Re: [Qemu-block] [PATCH v2 0/8] nbd block status base:allocation

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
13.03.2018 18:55, Eric Blake wrote: On 03/12/2018 10:21 AM, Vladimir Sementsov-Ogievskiy wrote: Hi all. Here is minimal realization of base:allocation context of NBD block-status extension, which allows to get block status through NBD. v2 changes are in each patch after "---" line. Vladimir

[Qemu-block] [PULL 15/41] blockjobs: add prepare callback

2018-03-13 Thread Kevin Wolf
From: John Snow Some jobs upon finalization may need to perform some work that can still fail. If these jobs are part of a transaction, it's important that these callbacks fail the entire transaction. We allow for a new callback in addition to commit/abort/clean that allows us

[Qemu-block] [PULL 20/41] iotests: test manual job dismissal

2018-03-13 Thread Kevin Wolf
From: John Snow Signed-off-by: John Snow Signed-off-by: Kevin Wolf --- tests/qemu-iotests/056 | 187 + tests/qemu-iotests/056.out | 4 +- 2 files changed, 189 insertions(+), 2 deletions(-)

[Qemu-block] [PULL 25/41] luks: Turn invalid assertion into check

2018-03-13 Thread Kevin Wolf
The .bdrv_getlength implementation of the crypto block driver asserted that the payload offset isn't after EOF. This is an invalid assertion to make as the image file could be corrupted. Instead, check it and return -EIO if the file is too small for the payload offset. Zero length images are

[Qemu-block] [PULL 27/41] qemu-iotests: Test luks QMP image creation

2018-03-13 Thread Kevin Wolf
Signed-off-by: Kevin Wolf Reviewed-by: Daniel P. Berrangé --- tests/qemu-iotests/209 | 210 +++ tests/qemu-iotests/209.out | 136 tests/qemu-iotests/common.rc | 2 +-

[Qemu-block] [PULL 12/41] blockjobs: ensure abort is called for cancelled jobs

2018-03-13 Thread Kevin Wolf
From: John Snow Presently, even if a job is canceled post-completion as a result of a failing peer in a transaction, it will still call .commit because nothing has updated or changed its return code. The reason why this does not cause problems currently is because backup's

[Qemu-block] [PULL 28/41] vdi: Pull option parsing from vdi_co_create

2018-03-13 Thread Kevin Wolf
From: Max Reitz In preparation of QAPI-fying VDI image creation, we have to create a BlockdevCreateOptionsVdi type which is received by a (future) vdi_co_create(). vdi_co_create_opts() now converts the QemuOpts object into such a BlockdevCreateOptionsVdi object. The

[Qemu-block] [PULL 36/41] qed: Support .bdrv_co_create

2018-03-13 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to qed, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- qapi/block-core.json | 25 ++- block/qed.c | 204

[Qemu-block] [PULL 19/41] blockjobs: Expose manual property

2018-03-13 Thread Kevin Wolf
From: John Snow Expose the "manual" property via QAPI for the backup-related jobs. As of this commit, this allows the management API to request the "concluded" and "dismiss" semantics for backup jobs. Signed-off-by: John Snow Signed-off-by: Kevin Wolf

[Qemu-block] [PULL 39/41] vpc: Support .bdrv_co_create

2018-03-13 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to vpc, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- qapi/block-core.json | 33 ++- block/vpc.c | 152

[Qemu-block] [PULL 17/41] blockjobs: add PENDING status and event

2018-03-13 Thread Kevin Wolf
From: John Snow For jobs utilizing the new manual workflow, we intend to prohibit them from modifying the block graph until the management layer provides an explicit ACK via block-job-finalize to move the process forward. To distinguish this runstate from "ready" or "waiting,"

[Qemu-block] [PULL 22/41] luks: Separate image file creation from formatting

2018-03-13 Thread Kevin Wolf
The crypto driver used to create the image file in a callback from the crypto subsystem. If we want to implement .bdrv_co_create, this needs to go away because that callback will get a reference to an already existing block node. Move the image file creation to block_crypto_create_generic().

[Qemu-block] [PULL 31/41] block: Fix flags in reopen queue

2018-03-13 Thread Kevin Wolf
From: Fam Zheng Reopen flags are not synchronized according to the bdrv_reopen_queue_child precedence until bdrv_reopen_prepare. It is a bit too late: we already check the consistency in bdrv_check_perm before that. This fixes the bug that when bdrv_reopen a RO node as RW, the

[Qemu-block] [PULL 26/41] luks: Catch integer overflow for huge sizes

2018-03-13 Thread Kevin Wolf
When you request an image size close to UINT64_MAX, the addition of the crypto header may cause an integer overflow. Catch it instead of silently truncating the image size. Signed-off-by: Kevin Wolf Reviewed-by: Daniel P. Berrangé --- block/crypto.c | 5

Re: [Qemu-block] [PATCH v10 05/12] migration: introduce postcopy-only pending

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
13.03.2018 19:16, John Snow wrote: On 03/13/2018 12:14 PM, Vladimir Sementsov-Ogievskiy wrote: Hmm, I agree, it is the simplest thing we can do for now, and I'll rethink later, how (and is it worth doing) to go to postcopy automatically in case of only-dirty-bitmaps. Should I respin? Please

[Qemu-block] [PULL 33/41] parallels: Support .bdrv_co_create

2018-03-13 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to parallels, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Jeff Cody --- qapi/block-core.json | 18 - block/parallels.c| 199

Re: [Qemu-block] [Qemu-devel] [PATCH] iotests: Update output of 051 and 186 after commit 1454509726719e0933c

2018-03-13 Thread Kevin Wolf
Am 13.03.2018 um 17:22 hat Thomas Huth geschrieben: > On 13.03.2018 17:08, Kevin Wolf wrote: > > Am 06.03.2018 um 17:52 hat Thomas Huth geschrieben: > >> On 06.03.2018 17:45, Alberto Garcia wrote: > >>> Signed-off-by: Alberto Garcia > >>> --- > >>> tests/qemu-iotests/051.pc.out

Re: [Qemu-block] [PATCH v10 05/12] migration: introduce postcopy-only pending

2018-03-13 Thread Vladimir Sementsov-Ogievskiy
13.03.2018 18:35, Dr. David Alan Gilbert wrote: * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: 13.03.2018 13:30, Dr. David Alan Gilbert wrote: * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: 12.03.2018 18:30, Dr. David Alan Gilbert wrote: * Vladimir

[Qemu-block] [PULL 30/41] vdi: Implement .bdrv_co_create

2018-03-13 Thread Kevin Wolf
From: Max Reitz Signed-off-by: Max Reitz Signed-off-by: Kevin Wolf --- qapi/block-core.json | 2 +- block/vdi.c | 24 +++- 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/qapi/block-core.json

[Qemu-block] [PULL 13/41] blockjobs: add commit, abort, clean helpers

2018-03-13 Thread Kevin Wolf
From: John Snow The completed_single function is getting a little mucked up with checking to see which callbacks exist, so let's factor them out. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf

[Qemu-block] [PULL 11/41] blockjobs: add block_job_dismiss

2018-03-13 Thread Kevin Wolf
From: John Snow For jobs that have reached their CONCLUDED state, prior to having their last reference put down (meaning jobs that have completed successfully, unsuccessfully, or have been canceled), allow the user to dismiss the job's lingering status report via

  1   2   >