[Qemu-block] [PATCH v2 11/13] block: Remove the AioContext parameter from bdrv_reopen_multiple()

2019-03-06 Thread Alberto Garcia
This parameter has been unused since 1a63a907507fbbcfaee3f622907ec244b Signed-off-by: Alberto Garcia --- block.c | 4 ++-- block/replication.c | 3 +-- include/block/block.h | 2 +- qemu-io-cmds.c| 2 +- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a

Re: [Qemu-block] [PATCH] iotests: Wait for qemu to end in 223

2019-03-06 Thread Alberto Garcia
7;tests/qemu-iotests/scratch/t.qcow2': > Failed to get shared "write" lock > +Is another process using the image [tests/qemu-iotests/scratch/t.qcow2]? > +qemu-img: Could not open > 'driver=nbd,server.type=unix,server.path=tests/qemu-iotests/scratch/qemu-nbd.soc

[Qemu-block] iotest 223 is flaky

2019-03-05 Thread Alberto Garcia
I can make it fail by running it a few times: 223 1s ... - output mismatch (see 223.out.bad) --- tests/qemu-iotests/223.out 2019-02-08 17:25:15.584387100 +0200 +++ tests/qemu-iotests/223.out.bad 2019-03-05 18:05:16.855223976 +0200 @@ -92,9 +92,10 @@ === Use qemu-nbd as server === -[

Re: [Qemu-block] [PATCH] qcow2: Default to 4KB for the qcow2 cache entry size

2019-03-05 Thread Alberto Garcia
ping On Wed 13 Feb 2019 05:48:53 PM CET, Alberto Garcia wrote: > QEMU 2.12 (commit 1221fe6f636754ab5f2c1c87caa77633e9123622) introduced > a new setting called l2-cache-entry-size that allows making entries on > the qcow2 L2 cache smaller than the cluster size. > > I have been per

Re: [Qemu-block] [PATCH 09/13] block: Add 'runtime_opts' and 'mutable_opts' fields to BlockDriver

2019-03-01 Thread Alberto Garcia
On Fri 01 Mar 2019 03:18:09 PM CET, Kevin Wolf wrote: > Hm, actually, do you even need the runtime_opts list? > > You use it in bdrv_reset_options_allowed() to iterate through each > option and then check whether it was present in the old set of > options, but not in the new set of options. > > In

Re: [Qemu-block] [PATCH 09/13] block: Add 'runtime_opts' and 'mutable_opts' fields to BlockDriver

2019-03-01 Thread Alberto Garcia
On Fri 01 Mar 2019 01:56:42 PM CET, Kevin Wolf wrote: >> >> >> diff --git a/include/block/block_int.h b/include/block/block_int.h >> >> >> index fd0e88d17a..e680dda86b 100644 >> >> >> --- a/include/block/block_int.h >> >> >> +++ b/include/block/block_int.h >> >> >> @@ -345,6 +345,13 @@ struct Block

Re: [Qemu-block] [PATCH 09/13] block: Add 'runtime_opts' and 'mutable_opts' fields to BlockDriver

2019-03-01 Thread Alberto Garcia
On Fri 01 Mar 2019 01:36:08 PM CET, Kevin Wolf wrote: > Am 01.03.2019 um 13:12 hat Alberto Garcia geschrieben: >> On Tue 12 Feb 2019 07:02:31 PM CET, Kevin Wolf wrote: >> >> diff --git a/include/block/block_int.h b/include/block/block_int.h >> >> index fd0e88d1

Re: [Qemu-block] [PATCH 09/13] block: Add 'runtime_opts' and 'mutable_opts' fields to BlockDriver

2019-03-01 Thread Alberto Garcia
On Tue 12 Feb 2019 07:02:31 PM CET, Kevin Wolf wrote: >> diff --git a/include/block/block_int.h b/include/block/block_int.h >> index fd0e88d17a..e680dda86b 100644 >> --- a/include/block/block_int.h >> +++ b/include/block/block_int.h >> @@ -345,6 +345,13 @@ struct BlockDriver { >> >> /* List

Re: [Qemu-block] [PATCH 2/2] block-stream: include base into job node list

2019-02-26 Thread Alberto Garcia
On Mon 25 Feb 2019 05:39:14 PM CET, Vladimir Sementsov-Ogievskiy wrote: >> --- a/block/stream.c >> +++ b/block/stream.c >> @@ -259,6 +259,15 @@ void stream_start(const char *job_id, BlockDriverState >> *bs, >> &error_abort); >> } >> >> +if (base) { >> +

Re: [Qemu-block] [PATCH] qcow2: Assert that L2 table offsets fit in the L1 table

2019-02-25 Thread Alberto Garcia
ping On Fri 08 Feb 2019 04:44:53 PM CET, Alberto Garcia wrote: > L1 table entries have a field to store the offset of an L2 table. > The rest of the bits of the entry are currently reserved except from > bit 63, which stores the COPIED flag. > > The offset is always taken from

Re: [Qemu-block] [PATCH 08/13] block: Allow changing the backing file on reopen

2019-02-21 Thread Alberto Garcia
(sorry I accidentally sent an incomplete reply a few minutes ago) On Tue 12 Feb 2019 06:27:56 PM CET, Kevin Wolf wrote: >> - In bdrv_reopen_commit(): perform the actual node replacement by >>calling bdrv_set_backing_hd(). It may happen that there are >>temporary implicit nodes between the

Re: [Qemu-block] [PATCH 08/13] block: Allow changing the backing file on reopen

2019-02-21 Thread Alberto Garcia
ith a new set of options ({"driver": "qcow2", "file": { >> ... }}) then it is interpreted that the _existing_ backing file must >> be reopened with those options. >> >> Signed-off-by: Alberto Garcia >> --- >> block.c | 124 >&

Re: [Qemu-block] [PATCH 06/13] block: Handle child references in bdrv_reopen_queue()

2019-02-19 Thread Alberto Garcia
On Tue 12 Feb 2019 05:28:06 PM CET, Kevin Wolf wrote: >>1) Set of child options: the options are removed from the parent's >> options QDict and are passed to the child with a recursive >> bdrv_reopen_queue() call. This case was already working fine. > > Small addition: This is only

Re: [Qemu-block] [PATCH 01/13] block: Allow freezing BdrvChild links

2019-02-18 Thread Alberto Garcia
On Thu 14 Feb 2019 04:52:52 PM CET, Kevin Wolf wrote: > Am 14.02.2019 um 15:13 hat Alberto Garcia geschrieben: >> On Tue 12 Feb 2019 03:47:47 PM CET, Kevin Wolf wrote: >> > Am 17.01.2019 um 16:33 hat Alberto Garcia geschrieben: >> >> @@ -3861,6 +3923,10

[Qemu-block] [PATCH] commit: Replace commit_top_bs on failure after deleting the block job

2019-02-15 Thread Alberto Garcia
;t seem to be possible to reproduce at the moment without changing the code. Signed-off-by: Alberto Garcia --- block/commit.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/block/commit.c b/block/commit.c index 53148e610b..5deb05925b 100644 --- a/block/commit.c +++ b/bloc

Re: [Qemu-block] [PATCH 02/13] block: Freeze the backing chain for the duration of the commit job

2019-02-14 Thread Alberto Garcia
On Tue 12 Feb 2019 03:54:15 PM CET, Kevin Wolf wrote: >> @@ -336,6 +340,10 @@ void commit_start(const char *job_id, BlockDriverState >> *bs, >> } >> } >> >> +if (bdrv_freeze_backing_chain(commit_top_bs, base, errp) < 0) { >> +goto fail; >> +} > > Don't error paths n

Re: [Qemu-block] [PATCH 01/13] block: Allow freezing BdrvChild links

2019-02-14 Thread Alberto Garcia
On Tue 12 Feb 2019 03:47:47 PM CET, Kevin Wolf wrote: > Am 17.01.2019 um 16:33 hat Alberto Garcia geschrieben: >> @@ -3861,6 +3923,10 @@ int bdrv_drop_intermediate(BlockDriverState *top, >> BlockDriverState *base, >> goto exit; >> } >> >> +

[Qemu-block] [PATCH] qcow2: Default to 4KB for the qcow2 cache entry size

2019-02-13 Thread Alberto Garcia
L2 tables. This is also to be expected because in this case no cache entry is ever evicted so reducing its size doesn't bring any benefit. This patch sets the default L2 cache entry size to 4KB except when the cache is large enough for the whole disk. Signed-off-by: Alberto Garcia --- block/

Re: [Qemu-block] [PATCH 04/13] block: Freeze the backing chain for the duration of the stream job

2019-02-12 Thread Alberto Garcia
On Tue 12 Feb 2019 04:15:58 PM CET, Kevin Wolf wrote: > Am 17.01.2019 um 16:33 hat Alberto Garcia geschrieben: >> Signed-off-by: Alberto Garcia >> --- >> block/stream.c | 16 >> 1 file changed, 16 insertions(+) >> >> diff --git a/b

Re: [Qemu-block] [PATCH RFC 1/1] Stream block job involves copy-on-read filter

2019-02-12 Thread Alberto Garcia
On Mon 11 Feb 2019 05:58:05 PM CET, Vladimir Sementsov-Ogievskiy wrote: >>> The problem is in the concept of "base" node. The code written in >>> manner that base is not changed during block job. However, job don't >>> own base and there is no guarantee that it will not change during >>> the job. >

Re: [Qemu-block] [PATCH RFC 1/1] Stream block job involves copy-on-read filter

2019-02-11 Thread Alberto Garcia
On Mon 11 Feb 2019 03:51:33 PM CET, Vladimir Sementsov-Ogievskiy wrote: I think this was already discussed in the previous version of this patch: if you insert a copy-on-read filter here then all guest reads will copy the data from the backing chain, but you don't want to copy a

Re: [Qemu-block] [PATCH v4 8/9] qcow2: bdrv_co_pwritev: move encryption code out of the lock

2019-02-11 Thread Alberto Garcia
On Mon 11 Feb 2019 03:44:20 PM CET, Vladimir Sementsov-Ogievskiy wrote: > Encryption will be done in threads, to take benefit of it, we should > move it out of the lock first. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH RFC 1/1] Stream block job involves copy-on-read filter

2019-02-11 Thread Alberto Garcia
On Fri 08 Feb 2019 04:29:48 PM CET, Andrey Shinkevich wrote: > On 08/02/2019 16:13, Alberto Garcia wrote: >> On Wed 23 Jan 2019 12:54:24 PM CET, Andrey Shinkevich wrote: >>> +static BlockDriverState *insert_filter(BlockDriverState *bs, Error **errp) >>> +{ >>>

[Qemu-block] [PATCH] qcow2: Assert that L2 table offsets fit in the L1 table

2019-02-08 Thread Alberto Garcia
QEMU. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 30eca26c47..179aa2c728 100644 --- a/block/qcow2-cluster.c +++ b/block/qcow2-cluster.c @@ -285,6 +285,9 @@ static int l2_all

Re: [Qemu-block] [PATCH RFC 1/1] Stream block job involves copy-on-read filter

2019-02-08 Thread Alberto Garcia
On Wed 23 Jan 2019 12:54:24 PM CET, Andrey Shinkevich wrote: > +static BlockDriverState *insert_filter(BlockDriverState *bs, Error **errp) > +{ > +BlockDriverState *cor_filter_bs; > +Error *local_err = NULL; > + > +cor_filter_bs = create_filter_node(bs, errp); > +if (cor_filter_bs =

Re: [Qemu-block] [PATCH v5 1/6] Stream block job involves copy-on-read filter driver

2019-02-08 Thread Alberto Garcia
On Sun 30 Dec 2018 09:09:12 PM CET, Andrey Shinkevich wrote: > +static BlockDriverState *child_file_bs(BlockDriverState *bs) > +{ > +return bs->file ? bs->file->bs : NULL; > +} In addition to Vladimir's comments, I wonder if it's useful to put this in block_int.h (together with backing_bs()).

Re: [Qemu-block] [PATCH 00/13] Add a 'x-blockdev-reopen' QMP command

2019-01-31 Thread Alberto Garcia
ping On Thu 17 Jan 2019 04:33:51 PM CET, Alberto Garcia wrote: > Hi, > > here's a patch series to implement a QMP command for bdrv_reopen(). > This is not too different from the previous iteration (RFC v2, see > changes below), but I'm not tagging it as RFC any longe

[Qemu-block] [PATCH] qtest.py: Wait for the result of qtest commands

2019-01-31 Thread Alberto Garcia
s under heavy workload. Signed-off-by: Alberto Garcia --- scripts/qtest.py | 6 ++ 1 file changed, 6 insertions(+) diff --git a/scripts/qtest.py b/scripts/qtest.py index adf1fe3f26..afac3fe900 100644 --- a/scripts/qtest.py +++ b/scripts/qtest.py @@ -31,6 +31,7 @@ class QEMUQtestProtocol(

Re: [Qemu-block] [Qemu-devel] Consistency of iotests 093 and 136

2019-01-29 Thread Alberto Garcia
On Mon 28 Jan 2019 07:38:08 PM CET, Markus Armbruster wrote: >> 093 submits several I/O requests using aio_read and aio_write with >> hmp_qemu_io(), then advances the clock using clock_step and finally >> calls query-blockstats to see how much of the I/O has been completed >> (it's an I/O throttli

Re: [Qemu-block] [PATCH 3/3] virtio-scsi: Forbid devices with different iothreads sharing a blockdev

2019-01-28 Thread Alberto Garcia
On Mon 28 Jan 2019 05:01:09 PM CET, Kevin Wolf wrote: > And actually... if you move the first virtio-blk device to an > iothread, then downgrading the others isn't going to save us, because > that would still be using the backend from two threads (one > successfully enabled iothread, and the main

Re: [Qemu-block] [PATCH 3/3] virtio-scsi: Forbid devices with different iothreads sharing a blockdev

2019-01-28 Thread Alberto Garcia
On Mon 28 Jan 2019 03:57:43 PM CET, Kevin Wolf wrote: >> > I think the proper solution on the block layer level would be that >> > AioContext is managed per BdrvChild and only BdrvChild objects with >> > the same AioContext can be attached to the same node. >> > bdrv_set_aio_context() would then f

Re: [Qemu-block] [Qemu-devel] Consistency of iotests 093 and 136

2019-01-28 Thread Alberto Garcia
On Thu 24 Jan 2019 07:07:47 PM CET, Eric Blake wrote: 093 and 136 seem really flaky to me. I can reproduce that by running: >>> >>> That's interesting, I can make 093 fail quite easily now (I haven't >>> tested the other one yet), but I don't think this happened >>> earlier. I'll try to

Re: [Qemu-block] [PATCH 3/3] virtio-scsi: Forbid devices with different iothreads sharing a blockdev

2019-01-28 Thread Alberto Garcia
On Thu 24 Jan 2019 10:12:22 AM CET, Kevin Wolf wrote: > Am 23.01.2019 um 17:16 hat Alberto Garcia geschrieben: >> On Wed 23 Jan 2019 04:47:30 PM CET, Paolo Bonzini wrote: >> >> You mean a common function with the code below? >> >> >> >>>&

Re: [Qemu-block] Consistency of iotests 093 and 136

2019-01-24 Thread Alberto Garcia
On Thu 24 Jan 2019 11:11:06 AM CET, Alberto Garcia wrote: > On Wed 23 Jan 2019 06:00:49 PM CET, Max Reitz wrote: >> Hi, >> >> 093 and 136 seem really flaky to me. I can reproduce that by running: > > That's interesting, I can make 093 fail quite easily now (I ha

Re: [Qemu-block] Consistency of iotests 093 and 136

2019-01-24 Thread Alberto Garcia
On Wed 23 Jan 2019 06:00:49 PM CET, Max Reitz wrote: > Hi, > > 093 and 136 seem really flaky to me. I can reproduce that by running: That's interesting, I can make 093 fail quite easily now (I haven't tested the other one yet), but I don't think this happened earlier. I'll try to figure out what'

Re: [Qemu-block] [PATCH 3/3] virtio-scsi: Forbid devices with different iothreads sharing a blockdev

2019-01-23 Thread Alberto Garcia
On Wed 23 Jan 2019 04:47:30 PM CET, Paolo Bonzini wrote: >> You mean a common function with the code below? >> +ctx = blk_get_aio_context(sd->conf.blk); +if (ctx != s->ctx && ctx != qemu_get_aio_context()) { +error_setg(errp, "Cannot attach a blockdev tha

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] Add a 'x-blockdev-reopen' QMP command

2019-01-23 Thread Alberto Garcia
On Tue 22 Jan 2019 09:15:25 AM CET, Vladimir Sementsov-Ogievskiy wrote: >>>a) Some options cannot be changed (most drivers don't allow that, in >>> fact). >>>b) If an option is missing, it should be reset to its default value >>> (rather than keeping its previous value). >> >>

Re: [Qemu-block] [PATCH 3/3] virtio-scsi: Forbid devices with different iothreads sharing a blockdev

2019-01-23 Thread Alberto Garcia
On Wed 23 Jan 2019 10:46:54 AM CET, Paolo Bonzini wrote: > On 22/01/19 16:53, Alberto Garcia wrote: >> This patch forbids attaching a disk to a SCSI device if its using a >> different AioContext. Test case included. >> >> Signed-off-by: Alberto Garcia > > Shou

[Qemu-block] [PATCH 3/3] virtio-scsi: Forbid devices with different iothreads sharing a blockdev

2019-01-22 Thread Alberto Garcia
This patch forbids attaching a disk to a SCSI device if its using a different AioContext. Test case included. Signed-off-by: Alberto Garcia --- hw/scsi/virtio-scsi.c | 7 +++ tests/qemu-iotests/240 | 22 ++ tests/qemu-iotests/240.out | 20

[Qemu-block] [PATCH 0/3] iothread-related fixes for virtio-scsi

2019-01-22 Thread Alberto Garcia
ject-add", "arguments": {"qom-type": "iothread", "id": "iothread1"}} { "execute": "device_add", "arguments": {"id": "virtio-blk0", "driver": "virtio-blk", "iothread"

[Qemu-block] [PATCH 1/3] virtio-scsi: Move BlockBackend back to the main AioContext on unplug

2019-01-22 Thread Alberto Garcia
This fixes a crash when attaching a disk to a SCSI device using iothreads, then detaching it and reattaching it again. Test case included. Signed-off-by: Alberto Garcia --- hw/scsi/virtio-scsi.c | 6 tests/qemu-iotests/240 | 89

[Qemu-block] [PATCH 2/3] scsi-disk: Acquire the AioContext in scsi_*_realize()

2019-01-22 Thread Alberto Garcia
This fixes a crash when attaching two disks with the same blockdev to a SCSI device that is using iothreads. Test case included. Signed-off-by: Alberto Garcia --- hw/scsi/scsi-disk.c| 23 --- tests/qemu-iotests/240 | 18 ++ tests/qemu-iotests/240

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/6] Acquire the AioContext during _realize()

2019-01-22 Thread Alberto Garcia
On Fri 18 Jan 2019 11:14:15 AM CET, Kevin Wolf wrote: > There are two ways to trigger the crash even without > x-blockdev-set-iothread: > > * device_del, then device_add for a device with iothread (virtio-scsi; > may or may not exist with virtio-blk) > https://bugzilla.redhat.com/show_bug.cgi?i

Re: [Qemu-block] qemu-img info ran at 100% CPU for 45 minutes without writing a byte (stopped it)

2019-01-18 Thread Alberto Garcia
On Wed 16 Jan 2019 02:15:24 AM CET, james harvey wrote: > I ran: > > # qemu-img convert /var/lib/libvirt/images/win7.qcow2 -O raw > /mnt/tmpqcow/win7.raw > > 45 minutes later, qemu-img had been running with 100% CPU every time I > checked, and it had allocated the raw file, but still hadn't actual

Re: [Qemu-block] [PATCH v3 8/9] qcow2: bdrv_co_pwritev: move encryption code out of the lock

2019-01-18 Thread Alberto Garcia
On Fri 18 Jan 2019 12:37:42 PM CET, Vladimir Sementsov-Ogievskiy wrote: > 18.01.2019 12:51, Alberto Garcia wrote: >> On Tue 08 Jan 2019 06:06:54 PM CET, Vladimir Sementsov-Ogievskiy wrote: >>> Encryption will be done in threads, to take benefit of it, we should >>> mo

Re: [Qemu-block] [PATCH v3 9/9] qcow2: do encryption in threads

2019-01-18 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:55 PM CET, Vladimir Sementsov-Ogievskiy wrote: > Do encryption/decryption in threads, like it is already done for > compression. This improves asynchronous encrypted io. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH v3 8/9] qcow2: bdrv_co_pwritev: move encryption code out of the lock

2019-01-18 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:54 PM CET, Vladimir Sementsov-Ogievskiy wrote: > Encryption will be done in threads, to take benefit of it, we should > move it out of the lock first. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/qcow2.c | 35 +-- > 1 file

Re: [Qemu-block] [Qemu-devel] [PATCH 00/13] Add a 'x-blockdev-reopen' QMP command

2019-01-17 Thread Alberto Garcia
On Thu 17 Jan 2019 04:50:20 PM CET, Eric Blake wrote: >> Removing the backing file can be done by simply passing the option { >> ..., "backing": null } to x-blockdev-reopen. >> > Hmm - that makes my proposal of "option":null as an explicit request > to the default a bit trickier, if we are alread

[Qemu-block] [PATCH 12/13] block: Add an 'x-blockdev-reopen' QMP command

2019-01-17 Thread Alberto Garcia
be possible to add a similar functionality to other block drivers (e.g. Quorum, blkverify). Although the API is unlikely to change, this command is marked experimental for the time being so there's room to see if the semantics need changes. Signed-off-by: Alberto Garcia --- blockdev.c

[Qemu-block] [PATCH 08/13] block: Allow changing the backing file on reopen

2019-01-17 Thread Alberto Garcia
ing file is by using a reference to an existing node (previously added with e.g. blockdev-add). If 'backing' contains a dictionary with a new set of options ({"driver": "qcow2", "file": { ... }}) then it is interpreted that the _existing_ backing file must be

[Qemu-block] [PATCH 01/13] block: Allow freezing BdrvChild links

2019-01-17 Thread Alberto Garcia
the link from the node it points to, and new API to freeze and unfreeze a backing chain. Signed-off-by: Alberto Garcia --- block.c | 66 +++ include/block/block.h | 5 include/block/block_int.h | 5 3 files changed, 76 in

[Qemu-block] [PATCH 05/13] block: Add 'keep_old_opts' parameter to bdrv_reopen_queue()

2019-01-17 Thread Alberto Garcia
. One of the things that we need is a way to tell bdrv_reopen_queue() whether we want to keep the old set of options or not, and that's what this patch does. All current callers are setting this new parameter to true and x-blockdev-reopen will set it to false. Signed-off-by: Alb

[Qemu-block] [PATCH 09/13] block: Add 'runtime_opts' and 'mutable_opts' fields to BlockDriver

2019-01-17 Thread Alberto Garcia
detect that, so we'll use these two new fields for that. Signed-off-by: Alberto Garcia --- block/blkdebug.c | 1 + block/crypto.c| 1 + block/file-posix.c| 10 ++ block/iscsi.c | 2 ++ block/null.c | 2 ++ block/nvme.c

[Qemu-block] [PATCH 11/13] block: Remove the AioContext parameter from bdrv_reopen_multiple()

2019-01-17 Thread Alberto Garcia
This parameter has been unused since 1a63a907507fbbcfaee3f622907ec244b Signed-off-by: Alberto Garcia --- block.c | 4 ++-- block/replication.c | 3 +-- include/block/block.h | 2 +- qemu-io-cmds.c| 2 +- 4 files changed, 5 insertions(+), 6 deletions(-) diff --git a

[Qemu-block] [PATCH 03/13] block: Freeze the backing chain for the duration of the mirror job

2019-01-17 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- block/mirror.c | 8 1 file changed, 8 insertions(+) diff --git a/block/mirror.c b/block/mirror.c index 22bef9f7e9..afbc30da61 100644 --- a/block/mirror.c +++ b/block/mirror.c @@ -630,6 +630,10 @@ static int mirror_exit_common(Job *job) } s

[Qemu-block] [PATCH 04/13] block: Freeze the backing chain for the duration of the stream job

2019-01-17 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- block/stream.c | 16 1 file changed, 16 insertions(+) diff --git a/block/stream.c b/block/stream.c index 7a49ac0992..39a2e10892 100644 --- a/block/stream.c +++ b/block/stream.c @@ -54,6 +54,14 @@ static int coroutine_fn stream_populate

[Qemu-block] [PATCH 13/13] qemu-iotests: Test the x-blockdev-reopen QMP command

2019-01-17 Thread Alberto Garcia
This patch adds several tests for the x-blockdev-reopen QMP command. Signed-off-by: Alberto Garcia --- tests/qemu-iotests/224 | 1001 tests/qemu-iotests/224.out |5 + tests/qemu-iotests/group |1 + 3 files changed, 1007 insertions

[Qemu-block] [PATCH 00/13] Add a 'x-blockdev-reopen' QMP command

2019-01-17 Thread Alberto Garcia
revent them from being removed, but the implementation was quite simple so I decided to give it a go. As you'll see in the patches I chose to use a bool instead of a counter because I couldn't think of a case where it would make sense to have two users freezing the same backing link. Thanks, Be

[Qemu-block] [PATCH 06/13] block: Handle child references in bdrv_reopen_queue()

2019-01-17 Thread Alberto Garcia
want to open a new image during reopen, so we require that "backing" is always present. We'll relax this requirement a bit in the next patch. Signed-off-by: Alberto Garcia --- block.c | 51 --- include/blo

[Qemu-block] [PATCH 10/13] block: Add bdrv_reset_options_allowed()

2019-01-17 Thread Alberto Garcia
e loop at the end of bdrv_reopen_prepare() already takes care of that. Signed-off-by: Alberto Garcia --- block.c | 50 ++ 1 file changed, 50 insertions(+) diff --git a/block.c b/block.c index 10847416b2..eea7aefa99 100644 --- a/block.c +++ b/block.c

[Qemu-block] [PATCH 07/13] block: Allow omitting the 'backing' option in certain cases

2019-01-17 Thread Alberto Garcia
this option if the block device being reopened doesn't have a backing file attached _and_ no default backing file is specified in the image metadata. Signed-off-by: Alberto Garcia --- block.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/block.c b/block.c ind

[Qemu-block] [PATCH 02/13] block: Freeze the backing chain for the duration of the commit job

2019-01-17 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- block/commit.c | 8 1 file changed, 8 insertions(+) diff --git a/block/commit.c b/block/commit.c index 53148e610b..8824d135e0 100644 --- a/block/commit.c +++ b/block/commit.c @@ -73,6 +73,8 @@ static int commit_prepare(Job *job) { CommitBlockJob

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/6] Acquire the AioContext during _realize()

2019-01-17 Thread Alberto Garcia
On Wed 16 Jan 2019 02:54:44 PM CET, Stefan Hajnoczi wrote: > The x-blockdev-set-iothread command is for low-level tests so I don't > expect users to invoke it. As I said in a different e-mail maybe this is not necessary then. > This patch series makes virtio-blk and virtio-scsi more robust, > alt

Re: [Qemu-block] Crash when deleting an iothread that is being used

2019-01-17 Thread Alberto Garcia
On Thu 17 Jan 2019 11:23:31 AM CET, Stefan Hajnoczi wrote: > I'm asking because x-blockdev-set-iothread is a low-level testing > command and it can create IOThread configurations that real-world > users never reach. I see, I suppose I had the wrong assumption about that command then! Berto

Re: [Qemu-block] [PATCH v3 7/9] qcow2: qcow2_co_preadv: skip using hd_qiov when possible

2019-01-16 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:53 PM CET, Vladimir Sementsov-Ogievskiy wrote: > qemu_iovec_memset has @offset parameter, so using hd_qiov for it is not > needed. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH v3 6/9] qcow2: qcow2_co_preadv: improve locking

2019-01-16 Thread Alberto Garcia
think about the code changes that you are doing, but when you read the actual resulting code it's fairly obvious that you don't need to touch the lock there. So I'd rather remove that comment. Else the patch looks good. Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH v3 5/9] qcow2-threads: split out generic path

2019-01-16 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:51 PM CET, Vladimir Sementsov-Ogievskiy wrote: > Move generic part out of qcow2_co_do_compress, to reuse it for > encryption and rename things that would be shared with encryption path. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH v3 4/9] qcow2-threads: qcow2_co_do_compress: protect queuing by mutex

2019-01-16 Thread Alberto Garcia
On Tue 08 Jan 2019 06:06:50 PM CET, Vladimir Sementsov-Ogievskiy wrote: > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH 1/1] block: Eliminate the S_1KiB, S_2KiB, ... macros

2019-01-16 Thread Alberto Garcia
vely reverts commit 540b8492618 and 1240ac558d3. > > Signed-off-by: Markus Armbruster Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH v12 09/10] qcow2: skip writing zero buffers to empty COW areas

2019-01-16 Thread Alberto Garcia
On Wed 16 Jan 2019 10:32:33 AM CET, Anton Nefedov wrote: >>> +ret = handle_alloc_space(bs, l2meta); >> >> I insist that it would be nice to have a short comment explaining >> what this does. > > Right sorry forgot your comment. > I'd go with: > > +/* Try to efficiently initialize t

Re: [Qemu-block] [PATCH v12 09/10] qcow2: skip writing zero buffers to empty COW areas

2019-01-15 Thread Alberto Garcia
rform_cow(). > > iotest 060: > write to the discarded cluster does not trigger COW anymore. > Use a backing image instead. > > Signed-off-by: Anton Nefedov > Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia > +ret = handle_alloc_space(bs, l2meta);

Re: [Qemu-block] [PATCH] throttle-groups: fix restart coroutine iothread race

2019-01-15 Thread Alberto Garcia
On Tue 15 Jan 2019 03:18:00 PM CET, Stefan Hajnoczi wrote: >> So if my understanding is correct QEMU can be shut down when there >> are iothreads waiting for a mutex. Is that something that we should >> be worried about? > > Nothing joins the iothreads in vl.c:main(). > > The assumption is that any

[Qemu-block] Crash when deleting an iothread that is being used

2019-01-15 Thread Alberto Garcia
Here's how to reproduce the crash: { "execute": "qmp_capabilities" } { "execute": "blockdev-add", "arguments": {"driver": "null-co", "node-name": "hd0"}} { "execute": "object-add", "arguments": {"qom-type": "iothread", "id": "iothread0"}} { "execute": "x-blockdev-set-iothread", "arguments": {"no

Re: [Qemu-block] [PATCH v2 2/2] iotests: add 238 for throttling tgm unregister iothread segfault

2019-01-14 Thread Alberto Garcia
On Mon 14 Jan 2019 02:32:57 PM CET, Stefan Hajnoczi wrote: > Hot-unplug a scsi-hd using an iothread. The previous patch fixes a > segfault in this scenario. > > This patch adds a regression test. > > Suggested-by: Alberto Garcia > Suggested-by: Kevin Wolf > Signed

Re: [Qemu-block] [PATCH] throttle-groups: fix restart coroutine iothread race

2019-01-14 Thread Alberto Garcia
On Mon 14 Jan 2019 05:31:17 PM CET, Stefan Hajnoczi wrote: > On Mon, Jan 14, 2019 at 05:26:48PM +0100, Alberto Garcia wrote: >> On Mon 14 Jan 2019 05:15:25 PM CET, Stefan Hajnoczi wrote: >> >> > I've been able to reproduce this in an iotest, please see v2 of this >

Re: [Qemu-block] [PATCH] throttle-groups: fix restart coroutine iothread race

2019-01-14 Thread Alberto Garcia
On Mon 14 Jan 2019 05:15:25 PM CET, Stefan Hajnoczi wrote: >> > I've been able to reproduce this in an iotest, please see v2 of this >> > series. >> >> That iotest doesn't crash for me :-? > > Does my iotest pass for you? Yes, it does. I'm trying to figure out why because if I run the QMP command

[Qemu-block] [PATCH v2 2/6] block: Acquire the AioContext in scsi_*_realize()

2019-01-14 Thread Alberto Garcia
This fixes a crash when adding a virtio-scsi device with a drive that is using an iothread. Test case included. Signed-off-by: Alberto Garcia --- hw/scsi/scsi-disk.c| 23 --- tests/qemu-iotests/236 | 43 +++ tests/qemu

[Qemu-block] [PATCH v2 0/6] Acquire the AioContext during _realize()

2019-01-14 Thread Alberto Garcia
()' 006/6:[] [--] 'block: Acquire the AioContext in usb_msd_storage_realize()' Alberto Garcia (6): block: Acquire the AioContext in virtio_blk_device_realize() block: Acquire the AioContext in scsi_*_realize() block: Acquire the AioContext in floppy_drive_realize() block: Acqui

[Qemu-block] [PATCH v2 3/6] block: Acquire the AioContext in floppy_drive_realize()

2019-01-14 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- hw/block/fdc.c | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/hw/block/fdc.c b/hw/block/fdc.c index 6f19f127a5..d9bc80de83 100644 --- a/hw/block/fdc.c +++ b/hw/block/fdc.c @@ -512,6 +512,7 @@ static void floppy_drive_realize

[Qemu-block] [PATCH v2 5/6] block: Acquire the AioContext in ide_dev_initfn()

2019-01-14 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- hw/ide/qdev.c | 17 - 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c index 573b022e1e..f355f2a352 100644 --- a/hw/ide/qdev.c +++ b/hw/ide/qdev.c @@ -160,6 +160,7 @@ static void ide_dev_initfn

Re: [Qemu-block] [PATCH v2 1/2] throttle-groups: fix restart coroutine iothread race

2019-01-14 Thread Alberto Garcia
e lock - by this time it's too late. > > This patch adds a reference count to ThrottleGroupMember so we can > synchronously wait for restart coroutines to complete. Once they are > done it is safe to unregister the ThrottleGroupMember. > > Signed-off-by: Stefan Hajnoczi Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH] throttle-groups: fix restart coroutine iothread race

2019-01-14 Thread Alberto Garcia
On Mon 14 Jan 2019 02:35:53 PM CET, Stefan Hajnoczi wrote: > On Fri, Jan 11, 2019 at 03:14:08PM +0100, Alberto Garcia wrote: >> On Fri 11 Jan 2019 02:24:16 PM CET, Kevin Wolf wrote: >> >> >> The following QMP command leads to a crash when iothreads are used: >

[Qemu-block] [PATCH v2 4/6] block: Acquire the AioContext in nvme_realize()

2019-01-14 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- hw/block/nvme.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 7c8c63e8f5..72e94aff86 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -1203,6 +1203,7 @@ static void nvme_realize

[Qemu-block] [PATCH v2 1/6] block: Acquire the AioContext in virtio_blk_device_realize()

2019-01-14 Thread Alberto Garcia
This fixes a crash when adding a virtio-blk device with a drive that is using an iothread. Test case included. Signed-off-by: Alberto Garcia --- hw/block/virtio-blk.c | 22 - tests/qemu-iotests/236 | 78 ++ tests/qemu-iotests/236

[Qemu-block] [PATCH v2 6/6] block: Acquire the AioContext in usb_msd_storage_realize()

2019-01-14 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- hw/usb/dev-storage.c | 5 + 1 file changed, 5 insertions(+) diff --git a/hw/usb/dev-storage.c b/hw/usb/dev-storage.c index cd5551d94f..fa6e552f0f 100644 --- a/hw/usb/dev-storage.c +++ b/hw/usb/dev-storage.c @@ -593,13 +593,18 @@ static void

Re: [Qemu-block] [PATCH 0/6] Acquire the AioContext during _realize()

2019-01-11 Thread Alberto Garcia
On Fri 11 Jan 2019 04:09:17 PM CET, Kevin Wolf wrote: > Am 10.01.2019 um 16:03 hat Alberto Garcia geschrieben: >> As discussed yesterday, this series acquires the AioContext in the >> _realize() functions of several devices before making use of their >> block backends. This fi

Re: [Qemu-block] [PATCH 2/6] block: Acquire the AioContext in scsi_*_realize()

2019-01-11 Thread Alberto Garcia
On Fri 11 Jan 2019 04:02:13 PM CET, Kevin Wolf wrote: >> @@ -2568,7 +2579,10 @@ static void scsi_block_realize(SCSIDevice *dev, Error >> **errp) >> } >> >> /* check we are using a driver managing SG_IO (version 3 and after) */ >> +ctx = blk_get_aio_context(s->qdev.conf.blk); >> +

Re: [Qemu-block] [PATCH] throttle-groups: fix restart coroutine iothread race

2019-01-11 Thread Alberto Garcia
On Fri 11 Jan 2019 02:24:16 PM CET, Kevin Wolf wrote: >> >> The following QMP command leads to a crash when iothreads are used: >> >> >> >> { 'execute': 'device_del', 'arguments': {'id': 'data'} } >> > >> > How did you reproduce this? Do you have a test case? >> >> Ok, I finally reproduced it, t

Re: [Qemu-block] [PATCH] throttle-groups: fix restart coroutine iothread race

2019-01-11 Thread Alberto Garcia
On Wed 09 Jan 2019 04:34:10 PM CET, Alberto Garcia wrote: > On Wed 09 Jan 2019 12:01:44 PM CET, Stefan Hajnoczi wrote: >> The following QMP command leads to a crash when iothreads are used: >> >> { 'execute': 'device_del', 'arguments': {'id

[Qemu-block] [PATCH 5/6] block: Acquire the AioContext in ide_dev_initfn()

2019-01-10 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- hw/ide/qdev.c | 17 - 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c index 573b022e1e..f355f2a352 100644 --- a/hw/ide/qdev.c +++ b/hw/ide/qdev.c @@ -160,6 +160,7 @@ static void ide_dev_initfn

[Qemu-block] [PATCH 6/6] block: Acquire the AioContext in usb_msd_storage_realize()

2019-01-10 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- hw/usb/dev-storage.c | 5 + 1 file changed, 5 insertions(+) diff --git a/hw/usb/dev-storage.c b/hw/usb/dev-storage.c index cd5551d94f..fa6e552f0f 100644 --- a/hw/usb/dev-storage.c +++ b/hw/usb/dev-storage.c @@ -593,13 +593,18 @@ static void

[Qemu-block] [PATCH 1/6] block: Acquire the AioContext in virtio_blk_device_realize()

2019-01-10 Thread Alberto Garcia
othread", "id": "iothread0"}} { "execute": "x-blockdev-set-iothread", "arguments": {"node-name": "hd0", "iothread": "iothread0"}} { "execute": "device_add", "arguments": {"

[Qemu-block] [PATCH 4/6] block: Acquire the AioContext in nvme_realize()

2019-01-10 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- hw/block/nvme.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 7c8c63e8f5..72e94aff86 100644 --- a/hw/block/nvme.c +++ b/hw/block/nvme.c @@ -1203,6 +1203,7 @@ static void nvme_realize

[Qemu-block] [PATCH 0/6] Acquire the AioContext during _realize()

2019-01-10 Thread Alberto Garcia
As discussed yesterday, this series acquires the AioContext in the _realize() functions of several devices before making use of their block backends. This fixes at least a couple of crashes (in virtio-blk and scsi). Berto Alberto Garcia (6): block: Acquire the AioContext in

[Qemu-block] [PATCH 2/6] block: Acquire the AioContext in scsi_*_realize()

2019-01-10 Thread Alberto Garcia
othread", "id": "iothread0"}} { "execute": "x-blockdev-set-iothread", "arguments": {"node-name": "hd0", "iothread": "iothread0"}} { "execute": "device_add", "arguments": {

[Qemu-block] [PATCH 3/6] block: Acquire the AioContext in floppy_drive_realize()

2019-01-10 Thread Alberto Garcia
Signed-off-by: Alberto Garcia --- hw/block/fdc.c | 15 +++ 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/hw/block/fdc.c b/hw/block/fdc.c index 6f19f127a5..d9bc80de83 100644 --- a/hw/block/fdc.c +++ b/hw/block/fdc.c @@ -512,6 +512,7 @@ static void floppy_drive_realize

Re: [Qemu-block] [PATCH v2 1/1] include: Auto-generate the sizes lookup table

2019-01-10 Thread Alberto Garcia
On Tue 08 Jan 2019 01:20:21 PM CET, Kevin Wolf wrote: > During a downstream review, Max found a problem with the table that we > could fix while we're touching it: > > Upstream: All >= S_2GiB are not valid ints. (qemu assumes that > sizeof(int) == 4, right?) So S_2GiB should be 2147483648

Re: [Qemu-block] [PATCH] block: Acquire the AioContext in guess_disk_lchs()

2019-01-09 Thread Alberto Garcia
On Wed 09 Jan 2019 04:07:33 PM CET, Kevin Wolf wrote: > This looks like the same thing that I talked about with Markus > yesterday. He asked me where to put the acquire/release pair. My > answer was that there is more than one way to do it, but I suspect > that the realize functions of the devices

Re: [Qemu-block] [PATCH] throttle-groups: fix restart coroutine iothread race

2019-01-09 Thread Alberto Garcia
On Wed 09 Jan 2019 12:01:44 PM CET, Stefan Hajnoczi wrote: > The following QMP command leads to a crash when iothreads are used: > > { 'execute': 'device_del', 'arguments': {'id': 'data'} } How did you reproduce this? Do you have a test case? Berto

[Qemu-block] [PATCH] block: Acquire the AioContext in guess_disk_lchs()

2019-01-09 Thread Alberto Garcia
othread", "id": "iothread0"}} { "execute": "x-blockdev-set-iothread", "arguments": {"node-name": "hd0", "iothread": "iothread0"}} { "execute": "device_add", "arguments": {"

<    6   7   8   9   10   11   12   13   14   15   >