[Qemu-block] [PATCH v10 0/3] qapi: child add/delete support

ChangLog: v9~v10: 1. Rebase to the newest codes 2. Address comments from Berto and Max, update the documents in block-core.json and qmp-commands.hx 3. Remove redundant codes in quorum_add_child() and quorum_del_child() v8: 1. Rebase to the newest codes 2. Address the comments from Eric Blake v

[Qemu-block] [PATCH v10 3/3] qmp: add monitor command to add/remove a child

From: Wen Congyang The new QMP command name is x-blockdev-change. It's just for adding/removing quorum's child now, and doesn't support all kinds of children, all kinds of operations, nor all block drivers. So it is experimental now. Signed-off-by: Wen Congyang Signed-off-by: zhanghailiang Sig

[Qemu-block] [PATCH v10 1/3] Add new block driver interface to add/delete a BDS's child

From: Wen Congyang In some cases, we want to take a quorum child offline, and take another child online. Signed-off-by: Wen Congyang Signed-off-by: zhanghailiang Signed-off-by: Gonglei Signed-off-by: Changlong Xie Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia --- block.c

[Qemu-block] [PATCH v10 2/3] quorum: implement bdrv_add_child() and bdrv_del_child()

From: Wen Congyang Signed-off-by: Wen Congyang Signed-off-by: zhanghailiang Signed-off-by: Gonglei Signed-off-by: Changlong Xie --- block.c | 8 ++-- block/quorum.c| 122 +- include/block/block.h | 4 ++ 3 files change

Re: [Qemu-block] Why qemu processes can bypass cgroup's blkio.weight ?

I used dd if=/zero of=/dev/vdc1 oflag=direct bs=1M to fill the qcow2 file before starting the read test. I have also observed iotop benchmark on host, the two qemu processes have the same throughput. Besides, I tried the write test as well, the same results. But according some blogs I found on th

Re: [Qemu-block] [PATCH 08/13] throttle: Add support for burst periods

Am 05.02.2016 um 11:59 hat Alberto Garcia geschrieben: > This patch adds support for burst periods to the throttling code. > With this feature the user can keep performing bursts as defined by > the LeakyBucket.max rate for a configurable period of time. > > Signed-off-by: Alberto Garcia > --- >

Re: [Qemu-block] [PATCH 1/3] block/backup: make backup cluster size configurable

On 02/14/2016 01:46 AM, Fam Zheng wrote: > On Fri, 02/12 18:06, John Snow wrote: >> typedef struct CowRequest { >> @@ -46,6 +43,8 @@ typedef struct BackupBlockJob { >> CoRwlock flush_rwlock; >> uint64_t sectors_read; >> HBitmap *bitmap; >> +int64_t cluster_size; >> +int64_

Re: [Qemu-block] [PATCH 08/13] throttle: Add support for burst periods

On Tue 16 Feb 2016 11:45:32 AM CET, Kevin Wolf wrote: >> +/* If the bucket is not full yet we have to make sure that we >> + * fulfill the goal of bkt->max units per second. */ >> +if (bkt->burst_length > 1) { >> +/* We use 1/10 of the max value to smooth the throttling. >> +

Re: [Qemu-block] [PATCH 00/13] throttle: Allow I/O bursts for a user-defined period of time

On Mon 15 Feb 2016 05:40:29 PM CET, Stefan Hajnoczi wrote: > On Fri, Feb 05, 2016 at 12:59:10PM +0200, Alberto Garcia wrote: >> - With this series we set "a maximum of X operations/second for a >>period of T seconds". If would also be possible to make it "a >>maximum of X operations/second

Re: [Qemu-block] [PATCH 2/3] block/backup: avoid copying less than full target clusters

On 02/14/2016 01:49 AM, Fam Zheng wrote: > On Fri, 02/12 18:06, John Snow wrote: >> During incremental backups, if the target has a cluster size that is >> larger than the backup cluster size and we are backing up to a target >> that cannot (for whichever reason) pull clusters up from a backing i

[Qemu-block] [PATCH] qed: fix bdrv_qed_drain

The current implementation of bdrv_qed_drain can cause a double completion of a request. The problem is that bdrv_qed_drain calls qed_plug_allocating_write_reqs unconditionally, but this is not correct if an allocating write is queued. In this case, qed_unplug_allocating_write_reqs will restart t

Re: [Qemu-block] [PATCH v6 00/16] Implement TLS support to QEMU NBD server & client

On 10/02/2016 19:40, Daniel P. Berrange wrote: > This is an update of the series previously posted: > > v1: https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg06126.html > v2: https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg01580.html > v3: https://lists.gnu.org/archiv

[Qemu-block] [PATCH 01/16] block: make bdrv_start_throttled_reqs return void

The return value is unused and I am not sure why it would be useful. Signed-off-by: Paolo Bonzini --- block/io.c | 8 ++-- 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/block/io.c b/block/io.c index a69bfc4..e58cfe2 100644 --- a/block/io.c +++ b/block/io.c @@ -75,10 +75,8 @@

[Qemu-block] [PATCH 07/16] block: change drain to look only at one child at a time

bdrv_requests_pending is checking children to also wait until internal requests (such as metadata writes) have completed. However, checking children is in general overkill because, apart from this special case, the parent's in_flight count will always be incremented by at least one for every reque

[Qemu-block] [PATCH 05/16] mirror: use bottom half to re-enter coroutine

mirror is calling bdrv_drain from an AIO callback---more precisely, the bdrv_drain happens far away from the AIO callback, in the coroutine that the AIO callback enters. This used to be okay because bdrv_drain more or less tried to guess when all AIO callbacks were done; however it will cause a de

[Qemu-block] [PATCH 00/16] AioContext fine-grained locking, part 1 of 3, including bdrv_drain rewrite

So the fine-grained locking series has grown from 2 parts to 3... This first part stops where we remove RFifoLock. In the previous submission, the bulk of the series took care of associating an AioContext to a thread, so that aio_poll could run event handlers only on that thread. This was done t

[Qemu-block] [PATCH 04/16] block: plug whole tree at once, introduce bdrv_io_unplugged_begin/end

Extract the handling of io_plug "depth" from linux-aio.c and let the main bdrv_drain loop do nothing but wait on I/O. Like the two newly introduced functions, bdrv_io_plug and bdrv_io_unplug now operate on all children. The visit order is now symmetrical between plug and unplug, making it possibl

[Qemu-block] [PATCH 02/16] block: move restarting of throttled reqs to block/throttle-groups.c

We want to remove throttled_reqs from block/io.c. This is the easy part---hide the handling of throttled_reqs during disable/enable of throttling within throttle-groups.c. Signed-off-by: Paolo Bonzini --- block/io.c | 15 +-- block/throttle-groups.c | 15

[Qemu-block] [PATCH 09/16] block: wait for all pending I/O when doing synchronous requests

Synchronous I/O should in general happen either in the main thread (e.g. for bdrv_open and bdrv_create) or between bdrv_drained_begin and bdrv_drained_end. Therefore, the simplest way to wait for it to finish is to wait for _all_ pending I/O to complete. In fact, there was one case in bdrv_close

[Qemu-block] [PATCH 08/16] blockjob: introduce .drain callback for jobs

This is required to decouple block jobs from running in an AioContext. With multiqueue block devices, a BlockDriverState does not really belong to a single AioContext. The solution is to first wait until all I/O operations are complete; then loop in the main thread for the block job to complete e

[Qemu-block] [PATCH 06/16] block: add BDS field to count in-flight requests

Unlike tracked_requests, this field also counts throttled requests, and remains non-zero if an AIO operation needs a BH to be "really" completed. With this change, it is no longer necessary to have a dummy BdrvTrackedRequest for requests that are never serialising, and it is no longer necessary to

[Qemu-block] [PATCH 12/16] aio: introduce aio_context_in_iothread

This will be used by the synchronous I/O helpers, to choose between aio_poll or QemuEvent. Signed-off-by: Paolo Bonzini --- include/block/aio.h | 7 +++ iothread.c | 9 + stubs/Makefile.objs | 1 + stubs/iothread.c| 8 4 files changed, 25 insertions(+) create m

[Qemu-block] [PATCH 03/16] block: introduce bdrv_no_throttling_begin/end

Extract the handling of throttling from bdrv_flush_io_queue. Signed-off-by: Paolo Bonzini --- block.c | 1 - block/io.c| 56 +-- block/throttle-groups.c | 4 include/block/block_int.h | 6 ++--- 4 files chang

[Qemu-block] [PATCH 11/16] sheepdog: disable dataplane

sheepdog has some calls to aio_poll that are hard to eliminate, for example in sd_sheepdog_goto's call to do_req. Since I don't have means to test sheepdog well, disable dataplane altogether for this driver. Signed-off-by: Paolo Bonzini --- block/sheepdog.c | 19 +++ 1 file chan

[Qemu-block] [PATCH 10/16] nfs: replace aio_poll with bdrv_drain

Signed-off-by: Paolo Bonzini --- block/nfs.c | 50 ++ 1 file changed, 30 insertions(+), 20 deletions(-) diff --git a/block/nfs.c b/block/nfs.c index 5eb8c13..082ef3a 100644 --- a/block/nfs.c +++ b/block/nfs.c @@ -47,6 +47,7 @@ typedef struct NFSCli

[Qemu-block] [PATCH 13/16] block: only call aio_poll from iothread

aio_poll is not thread safe; for example bdrv_drain can hang if the last in-flight I/O operation is completed in the I/O thread after the main thread has checked bs->in_flight. The bug remains latent as long as all of it is called within aio_context_acquire/aio_context_release, but this will chang

[Qemu-block] [PATCH 14/16] iothread: release AioContext around aio_poll

This is the first step towards having fine-grained critical sections in dataplane threads, which will resolve lock ordering problems between address_space_* functions (which need the BQL when doing MMIO, even after we complete RCU-based dispatch) and the AioContext. Because AioContext does not use

[Qemu-block] [PATCH 16/16] aio: convert from RFifoLock to QemuRecMutex

It is simpler and a bit faster, and QEMU does not need the contention callbacks (and thus the fairness) anymore. Signed-off-by: Paolo Bonzini --- async.c | 8 ++--- include/block/aio.h | 3 +- include/qemu/rfifolock.h | 54 tests/.gitignore

[Qemu-block] [PATCH 15/16] qemu-thread: introduce QemuRecMutex

GRecMutex is new in glib 2.32, so we cannot use it. Introduce a recursive mutex in qemu-thread instead, which will be used instead of RFifoLock. Signed-off-by: Paolo Bonzini --- include/qemu/thread-posix.h | 6 ++ include/qemu/thread-win32.h | 10 ++ include/qemu/thread.h |

[Qemu-block] [PATCH v3 01/14] monitor: Use BB list for BB name completion

Signed-off-by: Max Reitz --- monitor.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/monitor.c b/monitor.c index 73eac17..7620e20 100644 --- a/monitor.c +++ b/monitor.c @@ -42,6 +42,7 @@ #include "ui/console.h" #include "ui/input.h" #include "sysemu/blockdev.h" +#i

[Qemu-block] [PATCH v3 02/14] block: Use blk_next() where appropriate

blk_by_name() and blk_by_legacy_dinfo() are not supposed to iterate over hidden BlockBackends (that is, BlockBackends that do not have a name). As of a follow-up patch, blk_backends will actually contain all BBs, not only non-hidden ones, so use blk_next() instead of iterating over that list. Sign

[Qemu-block] [PATCH v3 03/14] block: Add blk_all_next()

Some places in block/block-backend.c need to iterate over actually all BlockBackends, so this adds a function to do so. blk_next() in contrast only iterates over the BlockBackends which are owned by the monitor. Right now, both do the same, but this will change as of a follow-up patch. Signed-off-

[Qemu-block] [PATCH v3 00/14] blockdev: Further BlockBackend work

At this point, the main merit of this series is to remove blk_hide_on_behalf_of_hmp_drive_del() and the bdrv_states list. A follow-up to it will make bdrv_open() return a BDS pointer. v3: It certainly feels like pretty much a rewrite, and git-backport-diff thinks so, too. The general idea was to

[Qemu-block] [PATCH v3 07/14] blockdev: Add list of monitor-owned BlockBackends

The monitor does hold references to some BlockBackends so it should have a list of those BBs; blk_backends is a different list, as it contains references to all BBs (after a follow-up patch, that is), and that should not be changed because we do need such a list. monitor_remove_blk() is idempotent

[Qemu-block] [PATCH v3 04/14] block: Add blk_name_taken()

There may be BlockBackends which are not returned by blk_by_name(), but do exist and have a name. blk_name_taken() allows testing whether a specific name is in use already, independent of whether the BlockBackend with that name is accessible through blk_by_name(). Signed-off-by: Max Reitz --- bl

[Qemu-block] [PATCH v3 06/14] block: Use blk_{commit, flush}_all() consistently

Replace bdrv_commmit_all() and bdrv_flush_all() by their BlockBackend equivalents. Signed-off-by: Max Reitz --- blockdev.c | 2 +- cpus.c | 5 +++-- qemu-char.c | 3 ++- 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/blockdev.c b/blockdev.c index 8eff47d..46cd8a9 100644 ---

[Qemu-block] [PATCH v3 11/14] block: Add blk_next_root_bs()

This function iterates over all BDSs attached to a BB. We are going to need it when rewriting bdrv_next() so it no longer uses bdrv_states. Signed-off-by: Max Reitz --- block/block-backend.c | 24 include/sysemu/block-backend.h | 1 + 2 files changed, 25 insert

[Qemu-block] [PATCH v3 05/14] block: Add blk_commit_all()

Later, we will remove bdrv_commit_all() and move its contents here, and in order to replace bdrv_commit_all() calls by calls to blk_commit_all() before doing so, we need to add it as an alias now. Signed-off-by: Max Reitz --- block/block-backend.c | 5 + include/sysemu/block-backend

[Qemu-block] [PATCH v3 12/14] block: Rewrite bdrv_next()

Instead of using the bdrv_states list, iterate over all the BlockDriverStates attached to BlockBackends, and over all the monitor-owned BDSs afterwards (except for those attached to a BB). Signed-off-by: Max Reitz --- block.c | 17 ++--- 1 file changed, 14 insertions(+), 3 deletions(

[Qemu-block] [PATCH v3 10/14] block: Add bdrv_next_monitor_owned()

Add a function for iterating over all monitor-owned BlockDriverStates so the generic block layer can do so. Signed-off-by: Max Reitz --- blockdev.c | 7 +++ include/block/block.h | 1 + stubs/Makefile.objs | 1 + stubs/bdrv-next-monitor-owned.c | 6

[Qemu-block] [PATCH v3 08/14] blockdev: Remove blk_hide_on_behalf_of_hmp_drive_del()

This function first removed the BlockBackend from the blk_backends list and cleared its name so it would no longer be found by blk_name(); since blk_next() now iterates through monitor_block_backends (which the BB is removed from in hmp_drive_del()), this is no longer necessary. Second, bdrv_make_

[Qemu-block] [PATCH v3 09/14] block: Move some bdrv_*_all() functions to BB

Move bdrv_commit_all() and bdrv_flush_all() to the BlockBackend level. Signed-off-by: Max Reitz --- block.c | 20 block/block-backend.c | 44 +++ block/io.c| 20

[Qemu-block] [PATCH v3 14/14] block: Remove bdrv_states list

Signed-off-by: Max Reitz --- block.c | 31 +++ blockdev.c| 7 --- include/block/block.h | 1 - include/block/block_int.h | 4 4 files changed, 3 insertions(+), 40 deletions(-) diff --git a/block.c b/block.c index 69fc6

[Qemu-block] [PATCH v3 13/14] block: Use bdrv_next() instead of bdrv_states

There is no point in manually iterating through the bdrv_states list when there is bdrv_next(). Signed-off-by: Max Reitz --- block.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/block.c b/block.c index f3597ac..69fc696 100644 --- a/block.c +++ b/block.c @@ -33

Re: [Qemu-block] [PATCH] qed: fix bdrv_qed_drain

On Tue, 02/16 16:53, Paolo Bonzini wrote: > The current implementation of bdrv_qed_drain can cause a double > completion of a request. > > The problem is that bdrv_qed_drain calls qed_plug_allocating_write_reqs > unconditionally, but this is not correct if an allocating write > is queued. In this