[Qemu-block] [PATCH v2] job: Fix nested aio_poll() hanging in job_txn_apply

All callers have acquired ctx already. Doing that again results in aio_poll() hang. This fixes the problem that a BDRV_POLL_WHILE() in the callback cannot make progress because ctx is recursively locked, for example, when drive-backup finishes. There are two callers of job_finalize(): fam@lem

[Qemu-block] [PATCH v2 04/13] block/commit: refactor commit to use job callbacks

Use the component callbacks; prepare, abort, and clean. NB: prepare is only called when the job has not yet failed; and abort can be called after prepare. complete -> prepare -> abort -> clean complete -> abort -> clean Signed-off-by: John Snow --- block/commit.c | 90 +

[Qemu-block] [PATCH v2 06/13] block/commit: refactor stream to use job callbacks

Signed-off-by: John Snow --- block/stream.c | 23 +++ 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/block/stream.c b/block/stream.c index 700eb239e4..81a7ec8ece 100644 --- a/block/stream.c +++ b/block/stream.c @@ -54,16 +54,16 @@ static int coroutine_fn stream

[Qemu-block] [PATCH v2 02/13] block/mirror: add block job creation flags

Add support for taking and passing forward job creaton flags. Signed-off-by: John Snow --- block/mirror.c| 5 +++-- blockdev.c| 3 ++- include/block/block_int.h | 5 - 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/block/mirror.c b/block/mirror.c i

[Qemu-block] [PATCH v2 12/13] qapi/block-stream: expose new job properties

Signed-off-by: John Snow --- blockdev.c | 9 + hmp.c| 5 +++-- qapi/block-core.json | 16 +++- 3 files changed, 27 insertions(+), 3 deletions(-) diff --git a/blockdev.c b/blockdev.c index 429cdf9901..0cf8febe6c 100644 --- a/blockdev.c +++ b/blockde

[Qemu-block] [PATCH v2 13/13] block/backup: qapi documentation fixup

Fix documentation to match the other jobs amended for 3.1. Signed-off-by: John Snow --- qapi/block-core.json | 18 ++ 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/qapi/block-core.json b/qapi/block-core.json index f877e9e414..c0b3d33dbb 100644 --- a/qapi/block-co

[Qemu-block] [PATCH v2 11/13] qapi/block-mirror: expose new job properties

Signed-off-by: John Snow --- blockdev.c | 14 ++ qapi/block-core.json | 30 -- 2 files changed, 42 insertions(+), 2 deletions(-) diff --git a/blockdev.c b/blockdev.c index 98b91e75a7..429cdf9901 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3597,6

[Qemu-block] [PATCH v2 05/13] block/mirror: conservative mirror_exit refactor

For purposes of minimum code movement, refactor the mirror_exit callback to use the post-finalization callbacks in a trivial way. Signed-off-by: John Snow --- block/mirror.c | 26 ++ 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/block/mirror.c b/block/mir

[Qemu-block] [PATCH v2 10/13] qapi/block-commit: expose new job properties

Signed-off-by: John Snow --- blockdev.c | 8 qapi/block-core.json | 16 +++- 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index ec90eb1cf9..98b91e75a7 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3204,6 +3204,8 @@ void qmp_

[Qemu-block] [PATCH v2 08/13] tests/test-blockjob: remove exit callback

We remove the exit callback and the completed boolean along with it. We can simulate it just fine by waiting for the job to defer to the main loop, and then giving it one final kick to get the main loop portion to run. Signed-off-by: John Snow --- tests/test-blockjob.c | 16 ++-- 1 f

[Qemu-block] [PATCH v2 03/13] block/stream: add block job creation flags

Add support for taking and passing forward job creaton flags. Signed-off-by: John Snow --- block/stream.c| 5 +++-- blockdev.c| 3 ++- include/block/block_int.h | 5 - 3 files changed, 9 insertions(+), 4 deletions(-) diff --git a/block/stream.c b/block/stream.c i

[Qemu-block] [PATCH v2 07/13] tests/blockjob: replace Blockjob with Job

These tests don't actually test blockjobs anymore, they test generic Job lifetimes. Change the types accordingly. Signed-off-by: John Snow --- tests/test-blockjob.c | 98 ++- 1 file changed, 50 insertions(+), 48 deletions(-) diff --git a/tests/tes

[Qemu-block] [PATCH v2 09/13] jobs: remove .exit callback

Now that all of the jobs use the component finalization callbacks, there's no use for the heavy-hammer .exit callback anymore. job_exit becomes a glorified type shim so that we can call job_completed from aio_bh_schedule_oneshot. Move these three functions down into job.c to eliminate a forward r

[Qemu-block] [PATCH v2 00/13] jobs: jobs: Job Exit Refactoring Pt 2

This is part two of a two part series that refactors the exit logic of jobs. This series forces all jobs to use the "finalize" semantics that were introduced previously, but only exposed via the backup jobs. Patches 1-3 add plumbing for the auto-dismiss and auto-finalize flags but do not expose t

[Qemu-block] [PATCH v2 01/13] block/commit: add block job creation flags

Add support for taking and passing forward job creation flags. Signed-off-by: John Snow --- block/commit.c| 5 +++-- blockdev.c| 7 --- include/block/block_int.h | 5 - 3 files changed, 11 insertions(+), 6 deletions(-) diff --git a/block/commit.c b/block/comm

[Qemu-block] [PATCH v2 9/9] jobs: remove job_defer_to_main_loop

Now that the job infrastructure is handling the job_completed call for all implemented jobs, we can remove the interface that allowed jobs to schedule their own completion. Signed-off-by: John Snow --- include/qemu/job.h | 17 - job.c | 40 ++-

[Qemu-block] [PATCH v2 5/9] block/mirror: utilize job_exit shim

Change the manual deferment to mirror_exit into the implicit callback to job_exit and the mirror_exit callback. This does change the order of some bdrv_unref calls and job_completed, but thanks to the new context in which we call .exit, this is safe to defer the possible flushing of any nodes to t

[Qemu-block] [PATCH v2 0/9] jobs: Job Exit Refactoring Pt 1

This is part one of a two part series that refactors the exit logic of jobs. Part one removes job_defer_to_main_loop. Part two removes the job->exit() callback introduced in part one. It's redundant to have each job manage deferring to the main loop itself. Unifying this makes sense from an API s

[Qemu-block] [PATCH v2 8/9] jobs: remove ret argument to job_completed; privatize it

Jobs are now expected to return their retcode on the stack, from the .run callback, so we can remove that argument. job_cancel does not need to set -ECANCELED because job_completed will update the return code itself if the job was canceled. While we're here, make job_completed static to job.c and

[Qemu-block] [PATCH v2 3/9] jobs: add exit shim

All jobs do the same thing when they leave their running loop: - Store the return code in a structure - wait to receive this structure in the main thread - signal job completion via job_completed Few jobs do anything beyond exactly this. Consolidate this exit logic for a net reduction in SLOC. Mo

[Qemu-block] [PATCH v2 6/9] jobs: utilize job_exit shim

Utilize the job_exit shim by not calling job_defer_to_main_loop, and where applicable, converting the deferred callback into the job_exit callback. This converts backup, stream, create, and the unit tests all at once. Most of these jobs do not see any changes to the order in which they clean up th

[Qemu-block] [PATCH v2 1/9] jobs: change start callback to run callback

Presently we codify the entry point for a job as the "start" callback, but a more apt name would be "run" to clarify the idea that when this function returns we consider the job to have "finished," except for any cleanup which occurs in separate callbacks later. As part of this clarification, chan

[Qemu-block] [PATCH v2 4/9] block/commit: utilize job_exit shim

Change the manual deferment to commit_complete into the implicit callback to job_exit, renaming commit_complete to commit_exit. This conversion does change the timing of when job_completed is called to after the bdrv_replace_node and bdrv_unref calls, which could have implications for bjob->blk wh

[Qemu-block] [PATCH v2 7/9] block/backup: make function variables consistently named

Rename opaque_job to job to be consistent with other job implementations. Rename 'job', the BackupBlockJob object, to 's' to also be consistent. Suggested-by: Eric Blake Signed-off-by: John Snow --- block/backup.c | 62 +- 1 file changed,

[Qemu-block] [PATCH v2 2/9] jobs: canonize Error object

Jobs presently use both an Error object in the case of the create job, and char strings in the case of generic errors elsewhere. Unify the two paths as just j->err, and remove the extra argument from job_completed. The integer error code for job_completed is kept for now, to be removed shortly in

Re: [Qemu-block] [PATCH v3 2/3] scripts: add render_block_graph function for QEMUMachine

On Thu, Aug 23, 2018 at 06:46:54PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Render block nodes graph with help of graphviz. This new function is > for debugging, so there is no sense to put it into qemu.py as a method > of QEMUMachine. Let's instead put it separately. > > Signed-off-by: Vladim

Re: [Qemu-block] [PULL 0/2] Block patches

On 22 August 2018 at 04:43, Jeff Cody wrote: > The following changes since commit ee135aa0428fe5af2af7be04ff16d2b596a9330a: > > Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-3.1-20180821' > into staging (2018-08-21 13:27:11 +0100) > > are available in the Git repository at: > > g

Re: [Qemu-block] [Qemu-devel] [PATCH v3 2/3] scripts: add render_block_graph function for QEMUMachine

On Thu, Aug 23, 2018 at 06:46:54PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Render block nodes graph with help of graphviz. This new function is > for debugging, so there is no sense to put it into qemu.py as a method > of QEMUMachine. Let's instead put it separately. > > Signed-off-by: Vladim

[Qemu-block] [PATCH v3 3/3] not-for-commit: example of new command usage for debugging

Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/222 | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222 index 0ead56d574..91d88aa5c0 100644 --- a/tests/qemu-iotests/222 +++ b/tests/qemu-iotests/222 @@ -137,6 +137,8 @@ with iotes

[Qemu-block] [PATCH v3 0/3] block nodes graph visualization

Hi all! On the way of backup schemes development (and in general any complicated developments in Qemu block layer) it would be good to have an ability to print out graph of block nodes with their permissions. Just look at attached picture. v3: again, major rework, after long discussion with Max:

[Qemu-block] [PATCH v3 1/3] qapi: add x-debug-query-block-graph

Add a new command, returning block nodes (and their users) graph. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 91 +++ include/block/block.h | 1 + include/sysemu/block-backend.h | 2 + block.c| 129 +++

[Qemu-block] [PATCH v3 2/3] scripts: add render_block_graph function for QEMUMachine

Render block nodes graph with help of graphviz. This new function is for debugging, so there is no sense to put it into qemu.py as a method of QEMUMachine. Let's instead put it separately. Signed-off-by: Vladimir Sementsov-Ogievskiy --- scripts/render_block_graph.py | 120 +++