[Qemu-block] [PATCH v5 00/21] blockjobs: add explicit job management

This series seeks to address two distinct but closely related issues concerning the job management API. (1) For jobs that complete when a monitor is not attached and receiving events or notifications, there's no way to discern the job's final return code. Jobs must remain in the query

[Qemu-block] [PATCH v5 02/21] blockjobs: model single jobs as transactions

model all independent jobs as single job transactions. It's one less case we have to worry about when we add more states to the transition machine. This way, we can just treat all job lifetimes exactly the same. This helps tighten assertions of the STM graph and removes some conditionals that

[Qemu-block] [PATCH v5 07/21] blockjobs: add block_job_verb permission table

Which commands ("verbs") are appropriate for jobs in which state is also somewhat burdensome to keep track of. As of this commit, it looks rather useless, but begins to look more interesting the more states we add to the STM table. A recurring theme is that no verb will apply to an 'undefined'

[Qemu-block] [PATCH v5 17/21] blockjobs: add PENDING status and event

For jobs utilizing the new manual workflow, we intend to prohibit them from modifying the block graph until the management layer provides an explicit ACK via block-job-finalize to move the process forward. To distinguish this runstate from "ready" or "waiting," we add a new "pending" event and

[Qemu-block] [PATCH v5 05/21] blockjobs: add state transition table

The state transition table has mostly been implied. We're about to make it a bit more complex, so let's make the STM explicit instead. Perform state transitions with a function that for now just asserts the transition is appropriate. Transitions: Undefined -> Created: During job initialization.

[Qemu-block] [PATCH v5 08/21] blockjobs: add ABORTING state

Add a new state ABORTING. This makes transitions from normative states to error states explicit in the STM, and serves as a disambiguation for which states may complete normally when normal end-states (CONCLUDED) are added in future commits. Notably, Paused/Standby jobs do not transition

[Qemu-block] [PATCH v5 11/21] blockjobs: add block_job_dismiss

For jobs that have reached their CONCLUDED state, prior to having their last reference put down (meaning jobs that have completed successfully, unsuccessfully, or have been canceled), allow the user to dismiss the job's lingering status report via block-job-dismiss. This gives management APIs the

[Qemu-block] [PATCH v5 12/21] blockjobs: ensure abort is called for cancelled jobs

Presently, even if a job is canceled post-completion as a result of a failing peer in a transaction, it will still call .commit because nothing has updated or changed its return code. The reason why this does not cause problems currently is because backup's implementation of .commit checks for

[Qemu-block] [PATCH v5 01/21] blockjobs: fix set-speed kick

If speed is '0' it's not actually "less than" the previous speed. Kick the job in this case too. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf --- blockjob.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

[Qemu-block] [PATCH v5 20/21] iotests: test manual job dismissal

Signed-off-by: John Snow --- tests/qemu-iotests/056 | 187 + tests/qemu-iotests/056.out | 4 +- 2 files changed, 189 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056 index

[Qemu-block] [PATCH v5 18/21] blockjobs: add block-job-finalize

Instead of automatically transitioning from PENDING to CONCLUDED, gate the .prepare() and .commit() phases behind an explicit acknowledgement provided by the QMP monitor if auto_finalize = false has been requested. This allows us to perform graph changes in prepare and/or commit so that graph

[Qemu-block] [PATCH v5 19/21] blockjobs: Expose manual property

Expose the "manual" property via QAPI for the backup-related jobs. As of this commit, this allows the management API to request the "concluded" and "dismiss" semantics for backup jobs. Signed-off-by: John Snow --- blockdev.c | 31 +++---

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/2] block latency histogram

Hi, This series failed docker-quick@centos6 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 20180309144918.44975-1-vsement...@virtuozzo.com Subject: [Qemu-devel] [PATCH v3 0/2]

[Qemu-block] [PATCH v5 03/21] Blockjobs: documentation touchup

Trivial; Document what the job creation flags do, and some general tidying. Signed-off-by: John Snow --- include/block/blockjob.h | 8 include/block/blockjob_int.h | 4 +++- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/include/block/blockjob.h

[Qemu-block] [PATCH v5 04/21] blockjobs: add status enum

We're about to add several new states, and booleans are becoming unwieldly and difficult to reason about. It would help to have a more explicit bookkeeping of the state of blockjobs. To this end, add a new "status" field and add our existing states in a redundant manner alongside the bools they

[Qemu-block] [PATCH v5 06/21] iotests: add pause_wait

Split out the pause command into the actual pause and the wait. Not every usage presently needs to resubmit a pause request. The intent with the next commit will be to explicitly disallow redundant or meaningless pause/resume requests, so the tests need to become more judicious to reflect that.

[Qemu-block] [PATCH v5 09/21] blockjobs: add CONCLUDED state

add a new state "CONCLUDED" that identifies a job that has ceased all operations. The wording was chosen to avoid any phrasing that might imply success, error, or cancellation. The task has simply ceased all operation and can never again perform any work. ("finished", "done", and "completed"

[Qemu-block] [PATCH v5 13/21] blockjobs: add commit, abort, clean helpers

The completed_single function is getting a little mucked up with checking to see which callbacks exist, so let's factor them out. Signed-off-by: John Snow Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf --- blockjob.c | 35

[Qemu-block] [PATCH v5 10/21] blockjobs: add NULL state

Add a new state that specifically demarcates when we begin to permanently demolish a job after it has performed all work. This makes the transition explicit in the STM table and highlights conditions under which a job may be demolished. Alongside this state, add a new helper command

[Qemu-block] [PATCH v5 15/21] blockjobs: add prepare callback

Some jobs upon finalization may need to perform some work that can still fail. If these jobs are part of a transaction, it's important that these callbacks fail the entire transaction. We allow for a new callback in addition to commit/abort/clean that allows us the opportunity to have fairly

[Qemu-block] [PATCH v5 16/21] blockjobs: add waiting status

For jobs that are stuck waiting on others in a transaction, it would be nice to know that they are no longer "running" in that sense, but instead are waiting on other jobs in the transaction. Jobs that are "waiting" in this sense cannot be meaningfully altered any longer as they have left their

[Qemu-block] [PATCH v5 21/21] tests/test-blockjob: test cancellations

Whatever the state a blockjob is in, it should be able to be canceled by the block layer. Signed-off-by: John Snow --- tests/test-blockjob.c | 233 +- 1 file changed, 229 insertions(+), 4 deletions(-) diff --git

[Qemu-block] [PATCH] iotests: 163 is not quick

Testing on ext4, most 'quick' qcow2 tests took less than 5 seconds, but 163 took more than 20. Let's remove it from the quick set. Signed-off-by: Eric Blake --- tests/qemu-iotests/group | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

Re: [Qemu-block] [PATCH v3 3/7] qapi: Replace qobject_to_X(o) by qobject_to(o, X)

On 02/28/2018 12:08 PM, Max Reitz wrote: +    options = qobject_to(options_obj, QDict); Bikeshedding - would it read any easier as: options = qobject_to(QDict, options_obj); So at this point, I'm 70:30 in favor of doing the rename to have qobject_to(type, obj) for consistency with

Re: [Qemu-block] [PATCH v3 0/7] block: Handle null backing link

On 02/24/2018 09:40 AM, Max Reitz wrote: Currently, we try to rewrite every occurrence of "backing": null into "backing": "" in qmp_blockdev_add(). However, that breaks using the same "backing": null construction in json:{} file names (which do not go through qmp_blockdev_add()). Currently,

Re: [Qemu-block] [PATCH v6 0/2] block latency histogram

On 03/09/2018 10:52 AM, Vladimir Sementsov-Ogievskiy wrote: v6: Use correct header qapi/qapi-builtin-types.h, to fix build again. Sorry for spam =( Vladimir Sementsov-Ogievskiy (2): block/accounting: introduce latency histogram qapi: add block latency histogram interface This

Re: [Qemu-block] [PATCH v6 1/2] block/accounting: introduce latency histogram

On 03/09/2018 10:52 AM, Vladimir Sementsov-Ogievskiy wrote: Introduce latency histogram statics for block devices. For each accounted operation type latency region [0, +inf) is Hard to read; I suggest: s/type latency/type, the latency/ divided into subregions by several points. Then,

Re: [Qemu-block] [PATCH v6 2/2] qapi: add block latency histogram interface

On 03/09/2018 10:52 AM, Vladimir Sementsov-Ogievskiy wrote: Set (and clear) histogram through new command block-latency-histogram-set and show new statistics in query-blockstats results. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 111