[Qemu-block] [PATCH v2 00/40] Generic background jobs

2018-05-18 Thread Kevin Wolf
Before we can make x-blockdev-create a background job, we need to
generalise the job infrastructure so that it can be used without any
associated block node.

This series extracts a Job object from the block job infrastructure,
which should contain everything related to jobs that doesn't require the
block layer to be involved.

Job creation with a centralised job-create command (if we even want
this) as well as the actual conversion of x-blockdev-create to Job is
left for another series.

v2:
- Already applied a small prefix of the series and rebased, so a few
  patches are missing compared to v1

- Fixed up forgotten renames of identifiers in comments (several
  patches all over the series) [Max]

- Patch 1 and 2 (new):
  QAPI documentation improvements for existing block-job-* commands

- Patch 5 (was patch 10; 'job: Add JobDriver.job_type'):
  Added 'const' to job_type()/job_type_str() parameter [Max]

- Patch 7 (was patch 12; 'job: Maintain a list of all jobs'):
  block_job_next(NULL) doesn't rely on BlockJob.job being the first
  field any more [Eric]

- Patch 8 (was patch 13; 'job: Move state transitions to Job'):
  Renamed JobVerb paramater from 'bv' to 'verb' [Eric]

- Patch 12 (was patch 17; 'job: Move defer_to_main_loop to Job'):
  Added commit message [Max, Eric, John]
  Removed stale comment [Max]

- Patch 26 (was patch 31; 'job: Move transactions to Job'):
  job_event_pending() can keep returning void [Max]

- Patch 27 (was patch 32; 'job: Move completion and cancellation to
  Job') and patch 28 (new):
  Added TODO in patch 27, and resolved it in patch 28 by moving the
  job_cancel_all() call out of bdrv_close_all() to the callers [Max]

- Patch 30 (was patch 34; 'job: Add job_dismiss()'):
  Fixed uninitialised use of 'job' [Max]

- Patch 33 (was patch 37; 'job: Move progress fields to Job'):
  Fixed documentation for job_progress_update/set_remaining() [Eric]

- Patch 34 (new):
  Separate patch for adding qapi/job.json that just moves the three
  Job-related enums JobType, JobStatus and JobVerb (without any commands
  or events) [Max]

- Patch 35 (was patch 38; 'job: Add JOB_STATUS_CHANGE QMP event'):
  Checking JOB_STATUS_CHANGE events more consistently in tests [Max]
  Updated qemu-iotests 094 reference output [Max]

- Patch 36 (was patch 39; 'job: Add lifecycle QMP commands'):
  Removed 'force' option from 'job-cancel' and make force=true the only
  behaviour. This makes mirror behave the same as other jobs in the
  context of 'job-cancel'. [Max]
  Fixed 'job-pause' documentation [Max]

- Patch 37 (was patch 40; 'job: Add query-jobs QMP command'):
  QAPI documentation improvements [Max]

- Patch 38 (new):
  Remove redundant BlockJob.driver [Max]

- Patch 40 (was patch 42; 'qemu-iotests: Test job-* with block jobs'):
  Filtered out progress in 'query-jobs' immediately after the job is
  started because on tmpfs the first request will already have
  completed. [Max]

Kevin Wolf (40):
  blockjob: Update block-job-pause/resume documentation
  blockjob: Improve BlockJobInfo.offset/len documentation
  job: Create Job, JobDriver and job_create()
  job: Rename BlockJobType into JobType
  job: Add JobDriver.job_type
  job: Add job_delete()
  job: Maintain a list of all jobs
  job: Move state transitions to Job
  job: Add reference counting
  job: Move cancelled to Job
  job: Add Job.aio_context
  job: Move defer_to_main_loop to Job
  job: Move coroutine and related code to Job
  job: Add job_sleep_ns()
  job: Move pause/resume functions to Job
  job: Replace BlockJob.completed with job_is_completed()
  job: Move BlockJobCreateFlags to Job
  blockjob: Split block_job_event_pending()
  job: Add job_event_*()
  job: Move single job finalisation to Job
  job: Convert block_job_cancel_async() to Job
  job: Add job_drain()
  job: Move .complete callback to Job
  job: Move job_finish_sync() to Job
  job: Switch transactions to JobTxn
  job: Move transactions to Job
  job: Move completion and cancellation to Job
  block: Cancel job in bdrv_close_all() callers
  job: Add job_yield()
  job: Add job_dismiss()
  job: Add job_is_ready()
  job: Add job_transition_to_ready()
  job: Move progress fields to Job
  job: Introduce qapi/job.json
  job: Add JOB_STATUS_CHANGE QMP event
  job: Add lifecycle QMP commands
  job: Add query-jobs QMP command
  blockjob: Remove BlockJob.driver
  iotests: Move qmp_to_opts() to VM
  qemu-iotests: Test job-* with block jobs

 qapi/block-core.json  |  116 +
 qapi/job.json |  252 ++
 qapi/qapi-schema.json |1 +
 include/block/block_int.h |2 +-
 include/block/blockjob.h  |  324 +---
 include/block/blockjob_int.h  |  176 +--
 include/qemu/job.h|  562 +
 block.c   |2 +-
 block/backup.c|   59 ++-
 block/commit.c|   44 +-
 block/mirror.c|  113 +++--
 block/replication.c   |   10 +-
 block/stream.c   

Re: [Qemu-block] [PATCH v2 00/40] Generic background jobs

2018-05-23 Thread Kevin Wolf
Am 18.05.2018 um 15:20 hat Kevin Wolf geschrieben:
> Before we can make x-blockdev-create a background job, we need to
> generalise the job infrastructure so that it can be used without any
> associated block node.
> 
> This series extracts a Job object from the block job infrastructure,
> which should contain everything related to jobs that doesn't require the
> block layer to be involved.
> 
> Job creation with a centralised job-create command (if we even want
> this) as well as the actual conversion of x-blockdev-create to Job is
> left for another series.

Applied to the block branch.

Kevin