Re: [RFC PATCH v3 1/5] block/nvme: Use an array of EventNotifier

2020-08-19 Thread Stefan Hajnoczi
On Tue, Aug 18, 2020 at 06:45:05PM +0200, Philippe Mathieu-Daudé wrote: > In preparation of using multiple IRQ (thus multiple eventfds) > make BDRVNVMeState::irq_notifier an array (for now of a single > element, the admin queue notifier). > > Signed-off-by: Philippe Mathieu-Daudé > --- > block/n

Re: [RFC PATCH 19/22] block/export: Move strong user reference to block_exports

2020-08-19 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > The reference owned by the user/monitor that is created when adding the > export and dropped when removing it was tied to the 'exports' list in > nbd/server.c. Every block export will have a user reference, so move it > to the block export level and tie it to

Re: [PATCH v2] qemu-img: Explicit number replaced by a constant

2020-08-19 Thread Alberto Garcia
On Wed 19 Aug 2020 03:36:07 AM CEST, Yi Li wrote: > Signed-off-by: Yi Li Reviewed-by: Alberto Garcia Berto

Re: [PATCH v2] qemu-img: Explicit number replaced by a constant

2020-08-19 Thread Stefano Garzarella
On Wed, Aug 19, 2020 at 09:36:07AM +0800, Yi Li wrote: > Signed-off-by: Yi Li > --- > qemu-img.c | 10 +- > 1 file changed, 5 insertions(+), 5 deletions(-) > > diff --git a/qemu-img.c b/qemu-img.c > index 5308773811..aa2e31c8ae 100644 > --- a/qemu-img.c > +++ b/qemu-img.c > @@ -1200,10 +

Re: What is bs->reqs_lock for?

2020-08-19 Thread Vladimir Sementsov-Ogievskiy
18.08.2020 09:16, Paolo Bonzini wrote: On 13/08/20 18:34, Vladimir Sementsov-Ogievskiy wrote: I thought bs is attached to one aio context and aio context attached to one iothread. For now yes, but with multiqueue there would be many iothreads sending requests to the AioContext. The BDS would

Re: [PATCH v5 1/5] block/io: fix bdrv_co_block_status_above

2020-08-19 Thread Vladimir Sementsov-Ogievskiy
Thanks a lot for reviewing! 18.08.2020 17:15, Alberto Garcia wrote: On Wed 10 Jun 2020 02:04:22 PM CEST, Vladimir Sementsov-Ogievskiy wrote: + * The top layer deferred to this layer, and because this layer is + * short, any zeroes that we synthesize beyond EOF behave as

Re: [PATCH v6 1/4] copy-on-read: Support preadv/pwritev_part functions

2020-08-19 Thread Vladimir Sementsov-Ogievskiy
19.08.2020 00:24, Andrey Shinkevich wrote: Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy -- Best regards, Vladimir

Re: [RFC PATCH 20/22] block/export: Add block-export-del

2020-08-19 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > Implement a new QMP command block-export-del and make nbd-server-remove > a wrapper around it. > > Signed-off-by: Kevin Wolf > --- > qapi/block-export.json | 30 +++ > include/block/nbd.h| 1 - > block/export/export.c

Re: [PATCH 00/18] hw/riscv: Add Microchip PolarFire SoC Icicle Kit board support

2020-08-19 Thread via
On 8/19/20 2:34 AM, Bin Meng wrote: > EXTERNAL EMAIL: Do not click links or open attachments unless you know the > content is safe > > On Tue, Aug 18, 2020 at 9:55 PM Anup Patel wrote: >> On Tue, Aug 18, 2020 at 6:39 PM wrote: >>> On 8/18/20 7:17 AM, Anup Patel wrote: EXTERNAL EMAIL: Do not

Re: [PATCH v6 2/4] copy-on-read: add filter append/drop functions

2020-08-19 Thread Vladimir Sementsov-Ogievskiy
19.08.2020 00:24, Andrey Shinkevich wrote: Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 103 +++

Re: [PATCH v6 3/4] qapi: add filter-node-name to block-stream

2020-08-19 Thread Vladimir Sementsov-Ogievskiy
19.08.2020 00:24, Andrey Shinkevich wrote: Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. That will be needed for further iotests implementations. and for users as well. I think, the last sentence may be dropped.

Re: [PATCH v7 27/47] blkverify: Use bdrv_sum_allocated_file_size()

2020-08-19 Thread Kevin Wolf
Am 25.06.2020 um 17:21 hat Max Reitz geschrieben: > blkverify is a filter, so bdrv_get_allocated_file_size()'s default > implementation will return only the size of its filtered child. > However, because both of its children are disk images, it makes more > sense to sum both of their allocated size

Re: [PATCH v6 4/4] block: apply COR-filter to block-stream jobs

2020-08-19 Thread Vladimir Sementsov-Ogievskiy
19.08.2020 00:24, Andrey Shinkevich wrote: The patch completes the series with the COR-filter insertion to any block-stream operation. It also makes changes to the iotests 030. The test case 'test_stream_parallel' was deleted due to multiple errors. "case deleted due to errors" is a bad reasoni

Re: [RFC PATCH 21/22] block/export: Move blk to BlockExport

2020-08-19 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > Every block export has a BlockBackend representing the disk that is > exported. It should live in BlockExport therefore. > > Signed-off-by: Kevin Wolf > --- > include/block/export.h | 3 +++ > block/export/export.c | 3 +++ > nbd/server.c | 44

Re: [PATCH v7 25/47] block: Def. impl.s for get_allocated_file_size

2020-08-19 Thread Kevin Wolf
Am 25.06.2020 um 17:21 hat Max Reitz geschrieben: > If every BlockDriver were to implement bdrv_get_allocated_file_size(), > there are basically three ways it would be handled: > (1) For protocol drivers: Figure out the actual allocated file size in > some protocol-specific way > (2) For protoc

Re: [RFC PATCH 22/22] block/export: Add query-block-exports

2020-08-19 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > This adds a simple QMP command to query the list of block exports. > > Signed-off-by: Kevin Wolf > --- > qapi/block-export.json | 33 + > block/export/export.c | 23 +++ > 2 files changed, 56 insertions(+

Re: [PATCH v5 1/5] block/io: fix bdrv_co_block_status_above

2020-08-19 Thread Alberto Garcia
On Wed 19 Aug 2020 11:48:25 AM CEST, Vladimir Sementsov-Ogievskiy wrote: >>> + * The top layer deferred to this layer, and because this >>> layer is >>> + * short, any zeroes that we synthesize beyond EOF behave as >>> if they >>> + * were allocated at this lay

Re: [RFC PATCH 19/22] block/export: Move strong user reference to block_exports

2020-08-19 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > The reference owned by the user/monitor that is created when adding the > export and dropped when removing it was tied to the 'exports' list in > nbd/server.c. Every block export will have a user reference, so move it > to the block export level and tie it to

Re: [RFC PATCH 22/22] block/export: Add query-block-exports

2020-08-19 Thread Kevin Wolf
Am 19.08.2020 um 13:04 hat Max Reitz geschrieben: > On 13.08.20 18:29, Kevin Wolf wrote: > > This adds a simple QMP command to query the list of block exports. > > > > Signed-off-by: Kevin Wolf > > --- > > qapi/block-export.json | 33 + > > block/export/export.c

Re: [PATCH v5 1/5] block/io: fix bdrv_co_block_status_above

2020-08-19 Thread Vladimir Sementsov-Ogievskiy
19.08.2020 14:34, Alberto Garcia wrote: On Wed 19 Aug 2020 11:48:25 AM CEST, Vladimir Sementsov-Ogievskiy wrote: + * The top layer deferred to this layer, and because this layer is + * short, any zeroes that we synthesize beyond EOF behave as if they + * were

Re: [PATCH v7 14/47] stream: Deal with filters

2020-08-19 Thread Max Reitz
On 10.08.20 13:04, Vladimir Sementsov-Ogievskiy wrote: > 10.08.2020 11:12, Max Reitz wrote: >> On 07.08.20 12:29, Vladimir Sementsov-Ogievskiy wrote: [...] >>> But, with our proposed way (freeze only chain up to base_overlay >>> inclusively, and use backing(base_overlay) as final backing), all wi

Re: [PATCH v6 2/7] hw: add 5.2 machine types and 5.1 compat options

2020-08-19 Thread Igor Mammedov
On Tue, 18 Aug 2020 17:11:32 +0200 Cornelia Huck wrote: > On Tue, 18 Aug 2020 15:33:43 +0100 > Stefan Hajnoczi wrote: > > > arm, i386, ppc, and s390x have versioned machine types and associated > > compatibility options. Introduce new ones now that QEMU 5.1 has been > > released. > > > > Signe

Re: [PATCH v6 2/7] hw: add 5.2 machine types and 5.1 compat options

2020-08-19 Thread Igor Mammedov
On Tue, 18 Aug 2020 15:33:43 +0100 Stefan Hajnoczi wrote: > arm, i386, ppc, and s390x have versioned machine types and associated > compatibility options. Introduce new ones now that QEMU 5.1 has been > released. > > Signed-off-by: Stefan Hajnoczi Reviewed-by: Igor Mammedov > --- > include/

Re: [PATCH v7 14/47] stream: Deal with filters

2020-08-19 Thread Vladimir Sementsov-Ogievskiy
19.08.2020 15:39, Max Reitz wrote: On 10.08.20 13:04, Vladimir Sementsov-Ogievskiy wrote: 10.08.2020 11:12, Max Reitz wrote: On 07.08.20 12:29, Vladimir Sementsov-Ogievskiy wrote: [...] But, with our proposed way (freeze only chain up to base_overlay inclusively, and use backing(base_overla

Re: [PATCH v6 2/7] hw: add 5.2 machine types and 5.1 compat options

2020-08-19 Thread Cornelia Huck
On Wed, 19 Aug 2020 14:54:25 +0200 Igor Mammedov wrote: > On Tue, 18 Aug 2020 17:11:32 +0200 > Cornelia Huck wrote: > > > On Tue, 18 Aug 2020 15:33:43 +0100 > > Stefan Hajnoczi wrote: > > > > > arm, i386, ppc, and s390x have versioned machine types and associated > > > compatibility options

Re: [RFC PATCH 0/9] block/curl: Add caching of data downloaded from the remote server

2020-08-19 Thread Stefan Hajnoczi
On Tue, Aug 18, 2020 at 12:08:36PM +0100, David Edmondson wrote: > When using qemu-img to convert an image that is hosted on an HTTP > server to some faster local (or pseudo-local) storage, the overall > performance can be improved by reading data from the HTTP server in > larger blocks and by cach

Re: [PATCH v6 2/7] hw: add 5.2 machine types and 5.1 compat options

2020-08-19 Thread Stefan Hajnoczi
On Tue, Aug 18, 2020 at 05:11:32PM +0200, Cornelia Huck wrote: > On Tue, 18 Aug 2020 15:33:43 +0100 > Stefan Hajnoczi wrote: > > > arm, i386, ppc, and s390x have versioned machine types and associated > > compatibility options. Introduce new ones now that QEMU 5.1 has been > > released. > > > >

Re: [PATCH v6 1/7] Open 5.2 development tree

2020-08-19 Thread Stefan Hajnoczi
On Tue, Aug 18, 2020 at 03:33:42PM +0100, Stefan Hajnoczi wrote: > From: Peter Maydell > > Signed-off-by: Peter Maydell > --- > VERSION | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/VERSION b/VERSION > index 831446cbd2..7d40cb9d36 100644 > --- a/VERSION > +++ b/VERSI

Re: [RFC PATCH 0/9] block/curl: Add caching of data downloaded from the remote server

2020-08-19 Thread David Edmondson
On Wednesday, 2020-08-19 at 15:11:37 +01, Stefan Hajnoczi wrote: > On Tue, Aug 18, 2020 at 12:08:36PM +0100, David Edmondson wrote: >> When using qemu-img to convert an image that is hosted on an HTTP >> server to some faster local (or pseudo-local) storage, the overall >> performance can be impro

Re: [RFC PATCH 19/22] block/export: Move strong user reference to block_exports

2020-08-19 Thread Kevin Wolf
Am 19.08.2020 um 13:56 hat Max Reitz geschrieben: > On 13.08.20 18:29, Kevin Wolf wrote: > > The reference owned by the user/monitor that is created when adding the > > export and dropped when removing it was tied to the 'exports' list in > > nbd/server.c. Every block export will have a user refere

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-19 Thread Alberto Garcia
On Mon 17 Aug 2020 05:53:07 PM CEST, Kevin Wolf wrote: >> > Or are you saying that ZERO_RANGE + pwrite on a sparse file (= >> > cluster allocation) is faster for you than just the pwrite alone (= >> > writing to already allocated cluster)? >> >> Yes, 20% faster in my tests (4KB random writes), but

Re: [PATCH v6 2/7] hw: add 5.2 machine types and 5.1 compat options

2020-08-19 Thread Laszlo Ersek
On 08/19/20 16:12, Stefan Hajnoczi wrote: > On Tue, Aug 18, 2020 at 05:11:32PM +0200, Cornelia Huck wrote: >> On Tue, 18 Aug 2020 15:33:43 +0100 >> Stefan Hajnoczi wrote: >> >>> arm, i386, ppc, and s390x have versioned machine types and associated >>> compatibility options. Introduce new ones now

Re: [PATCH v3 05/12] block: bdrv_mark_request_serialising: split non-waiting function

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:46PM +0300, Vladimir Sementsov-Ogievskiy wrote: > @@ -882,6 +878,20 @@ static bool coroutine_fn > bdrv_wait_serialising_requests(BdrvTrackedRequest *self > return waited; > } > > +bool bdrv_make_request_serialising(BdrvTrackedRequest *req, uint64_t align) > +{

Re: [PATCH v3 06/12] block: introduce BDRV_REQ_NO_WAIT flag

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:47PM +0300, Vladimir Sementsov-Ogievskiy wrote: > diff --git a/block/io.c b/block/io.c > index 96b1b9cf5f..fc6d44d302 100644 > --- a/block/io.c > +++ b/block/io.c > @@ -1911,9 +1911,20 @@ bdrv_co_write_req_prepare(BdrvChild *child, int64_t > offset, uint64_t bytes, >

Re: [PATCH v3] block/nbd: use non-blocking connect: fix vm hang on connect()

2020-08-19 Thread Eric Blake
On 8/12/20 9:52 AM, Vladimir Sementsov-Ogievskiy wrote: This make nbd connection_co to yield during reconnects, so that s/make nbd connection_co to/makes nbd's connection_co/ reconnect doesn't hang up the main thread. This is very important in s/hang up/block/ case of unavailable nbd serv

Re: [PATCH v7 14/47] stream: Deal with filters

2020-08-19 Thread Max Reitz
On 18.08.20 16:28, Kevin Wolf wrote: > Am 25.06.2020 um 17:21 hat Max Reitz geschrieben: >> Because of the (not so recent anymore) changes that make the stream job >> independent of the base node and instead track the node above it, we >> have to split that "bottom" node into two cases: The bottom

Re: [RFC PATCH 19/22] block/export: Move strong user reference to block_exports

2020-08-19 Thread Max Reitz
On 19.08.20 16:23, Kevin Wolf wrote: > Am 19.08.2020 um 13:56 hat Max Reitz geschrieben: >> On 13.08.20 18:29, Kevin Wolf wrote: >>> The reference owned by the user/monitor that is created when adding the >>> export and dropped when removing it was tied to the 'exports' list in >>> nbd/server.c. Ev

Re: [PATCH v3 07/12] block: introduce preallocate filter

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:48PM +0300, Vladimir Sementsov-Ogievskiy wrote: > diff --git a/docs/system/qemu-block-drivers.rst.inc > b/docs/system/qemu-block-drivers.rst.inc > index b052a6d14e..5bfa4f4116 100644 > --- a/docs/system/qemu-block-drivers.rst.inc > +++ b/docs/system/qemu-block-drivers

Re: [PATCH v3 01/12] block: simplify comment to BDRV_REQ_SERIALISING

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:42PM +0300, Vladimir Sementsov-Ogievskiy wrote: > 1. BDRV_REQ_NO_SERIALISING doesn't exist already, don't mention it. > > 2. We are going to add one more user of BDRV_REQ_SERIALISING, so >comment about backup becomes a bit confusing here. The use case in >back

Re: [PATCH v3 04/12] block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:45PM +0300, Vladimir Sementsov-Ogievskiy wrote: > bs is linked in req, so no needs to pass it separately. Most of > tracked-requests API doesn't have bs argument. Actually, after this > patch only tracked_request_begin has it, but it's for purpose. > > While being her

Re: [PATCH v3 03/12] block/io: split out bdrv_find_conflicting_request

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:44PM +0300, Vladimir Sementsov-Ogievskiy wrote: > To be reused in separate. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/io.c | 71 +++--- > 1 file changed, 41 insertions(+), 30 deletions(-) Reviewed-

Re: [PATCH v3 10/12] iotests: add 298 to test new preallocate filter driver

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:51PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/298 | 50 ++ > tests/qemu-iotests/298.out | 6 + > tests/qemu-iotests/group | 1 + > 3 files chang

Re: [PATCH v3 09/12] iotests.py: add filter_img_check

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:50PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Add analog of bash _filter_qemu_img_check to python framework. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/iotests.py | 4 > 1 file changed, 4 insertions(+) Reviewed-by: Stefan Ha

Re: [PATCH v3 08/12] iotests.py: add verify_o_direct helper

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:49PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Add python notrun-helper similar to _check_o_direct for bash tests. > To be used in the following commit. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/iotests.py | 6 ++ > 1 file cha

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-19 Thread Kevin Wolf
Am 19.08.2020 um 16:25 hat Alberto Garcia geschrieben: > On Mon 17 Aug 2020 05:53:07 PM CEST, Kevin Wolf wrote: > >> > Or are you saying that ZERO_RANGE + pwrite on a sparse file (= > >> > cluster allocation) is faster for you than just the pwrite alone (= > >> > writing to already allocated cluste

Re: [PATCH v3 12/12] block/qcow2: automatically insert preallocate filter when on FUSE

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:53PM +0300, Vladimir Sementsov-Ogievskiy wrote: > vstorage has slow allocation, so this patch detect vstorage > (I hope, we don't use other FUSE filesystems) and inserts preallocate > filter between qcow2 node and its file child. > > The following test executes more t

Re: [PATCH v7 14/47] stream: Deal with filters

2020-08-19 Thread Kevin Wolf
Am 19.08.2020 um 16:47 hat Max Reitz geschrieben: > On 18.08.20 16:28, Kevin Wolf wrote: > > Am 25.06.2020 um 17:21 hat Max Reitz geschrieben: > >> Because of the (not so recent anymore) changes that make the stream job > >> independent of the base node and instead track the node above it, we > >>

Re: [PATCH v6 2/7] hw: add 5.2 machine types and 5.1 compat options

2020-08-19 Thread Philippe Mathieu-Daudé
On 8/19/20 4:38 PM, Laszlo Ersek wrote: > On 08/19/20 16:12, Stefan Hajnoczi wrote: >> On Tue, Aug 18, 2020 at 05:11:32PM +0200, Cornelia Huck wrote: >>> On Tue, 18 Aug 2020 15:33:43 +0100 >>> Stefan Hajnoczi wrote: >>> arm, i386, ppc, and s390x have versioned machine types and associated >>>

Re: [PATCH v3 02/12] block/io.c: drop assertion on double waiting for request serialisation

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:43PM +0300, Vladimir Sementsov-Ogievskiy wrote: > The comments states, that on misaligned request we should have already > been waiting. But for bdrv_padding_rmw_read, we called > bdrv_mark_request_serialising with align = request_alignment, and now > we serialise with

Re: [PATCH v3 00/12] preallocate filter

2020-08-19 Thread Stefan Hajnoczi
On Mon, Aug 17, 2020 at 12:15:41PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Here is a filter, which does preallocation on write. Looks quite close to being merged. I have left comments. Stefan signature.asc Description: PGP signature

Re: [PATCH v7 10/47] mirror-top: Support compressed writes

2020-08-19 Thread Max Reitz
On 18.08.20 12:27, Kevin Wolf wrote: > Am 25.06.2020 um 17:21 hat Max Reitz geschrieben: >> Signed-off-by: Max Reitz >> --- >> block/mirror.c | 10 ++ >> 1 file changed, 10 insertions(+) >> >> diff --git a/block/mirror.c b/block/mirror.c >> index e8e8844afc..469acf4600 100644 >> --- a/blo

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-19 Thread Alberto Garcia
On Wed 19 Aug 2020 05:07:11 PM CEST, Kevin Wolf wrote: >> I checked with xfs on my computer. I'm not very familiar with that >> filesystem so I was using the default options and I didn't tune >> anything. >> >> What I got with my tests (using fio): >> >> - Using extent_size_hint didn't make any d

Re: [PATCH v3 02/12] block/io.c: drop assertion on double waiting for request serialisation

2020-08-19 Thread Paolo Bonzini
On 19/08/20 17:28, Stefan Hajnoczi wrote: > On Mon, Aug 17, 2020 at 12:15:43PM +0300, Vladimir Sementsov-Ogievskiy wrote: >> The comments states, that on misaligned request we should have already >> been waiting. But for bdrv_padding_rmw_read, we called >> bdrv_mark_request_serialising with align =

Re: [PATCH v7 27/47] blkverify: Use bdrv_sum_allocated_file_size()

2020-08-19 Thread Max Reitz
On 19.08.20 12:46, Kevin Wolf wrote: > Am 25.06.2020 um 17:21 hat Max Reitz geschrieben: >> blkverify is a filter, so bdrv_get_allocated_file_size()'s default >> implementation will return only the size of its filtered child. >> However, because both of its children are disk images, it makes more >

Re: [PATCH v7 25/47] block: Def. impl.s for get_allocated_file_size

2020-08-19 Thread Max Reitz
On 19.08.20 12:57, Kevin Wolf wrote: > Am 25.06.2020 um 17:21 hat Max Reitz geschrieben: >> If every BlockDriver were to implement bdrv_get_allocated_file_size(), >> there are basically three ways it would be handled: >> (1) For protocol drivers: Figure out the actual allocated file size in >>

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-19 Thread Alberto Garcia
On Wed 19 Aug 2020 05:37:12 PM CEST, Alberto Garcia wrote: > I ran the test again on a newly created filesystem just to make sure, > here are the full results (numbers are IOPS): > > |--+---+---| > | preallocation| ext4 | xfs | > |--+--

Re: [RFC PATCH v3 1/5] block/nvme: Use an array of EventNotifier

2020-08-19 Thread Philippe Mathieu-Daudé
On 8/19/20 10:08 AM, Stefan Hajnoczi wrote: > On Tue, Aug 18, 2020 at 06:45:05PM +0200, Philippe Mathieu-Daudé wrote: >> In preparation of using multiple IRQ (thus multiple eventfds) >> make BDRVNVMeState::irq_notifier an array (for now of a single >> element, the admin queue notifier). >> >> Sign

[RFC PATCH v4 4/5] util/vfio-helpers: Introduce qemu_vfio_pci_init_msix_irqs()

2020-08-19 Thread Philippe Mathieu-Daudé
qemu_vfio_pci_init_irq() allows us to initialize any type of IRQ, but only one. Introduce qemu_vfio_pci_init_msix_irqs() which is specific to MSIX IRQ type, and allow us to use multiple IRQs (thus passing multiple eventfd notifiers). Signed-off-by: Philippe Mathieu-Daudé --- include/qemu/vfio-he

[RFC PATCH v4 5/5] block/nvme: Use qemu_vfio_pci_init_msix_irqs() to initialize our IRQ

2020-08-19 Thread Philippe Mathieu-Daudé
Instead of initializing one MSIX IRQ with the generic qemu_vfio_pci_init_irq() function, use the MSIX specific one which ill allow us to use multiple IRQs. For now we provide an array of a single IRQ. Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 9 +++-- 1 file changed, 7 inserti

Re: [PATCH v7 10/47] mirror-top: Support compressed writes

2020-08-19 Thread Kevin Wolf
Am 19.08.2020 um 17:35 hat Max Reitz geschrieben: > On 18.08.20 12:27, Kevin Wolf wrote: > > Am 25.06.2020 um 17:21 hat Max Reitz geschrieben: > >> Signed-off-by: Max Reitz > >> --- > >> block/mirror.c | 10 ++ > >> 1 file changed, 10 insertions(+) > >> > >> diff --git a/block/mirror.c b/

[RFC PATCH v4 0/5] util/vfio-helpers: Add support for multiple IRQs

2020-08-19 Thread Philippe Mathieu-Daudé
This series intends to setup the VFIO helper to allow binding notifiers on different IRQs. For the NVMe use case, we only care about MSIX interrupts. To not disrupt other users, introduce the qemu_vfio_pci_init_msix_irqs function to initialize multiple MSIX IRQs and attach eventfd to them. Since

[RFC PATCH v4 3/5] util/vfio-helpers: Report error when IOMMU page size is not supported

2020-08-19 Thread Philippe Mathieu-Daudé
This driver uses the host page size to align its memory regions, but this size is not always compatible with the IOMMU. Add a check if the size matches, and bails out with listing the sizes the IOMMU supports. Example on Aarch64: $ qemu-system-aarch64 -M virt -drive if=none,id=nvme0,file=nvme:/

[RFC PATCH v4 1/5] block/nvme: Use an array of EventNotifier

2020-08-19 Thread Philippe Mathieu-Daudé
In preparation of using multiple IRQ (thus multiple eventfds) make BDRVNVMeState::irq_notifier an array (for now of a single element, the admin queue notifier). Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 31 +-- 1 file changed, 21 insertions(+), 10 delet

[RFC PATCH v4 2/5] util/vfio-helpers: Improve reporting unsupported IOMMU type

2020-08-19 Thread Philippe Mathieu-Daudé
Change the confuse "VFIO IOMMU check failed" error message by the explicit "VFIO IOMMU Type1 is not supported" once. Example on POWER: $ qemu-system-ppc64 -drive if=none,id=nvme0,file=nvme://0001:01:00.0/1,format=raw qemu-system-ppc64: -drive if=none,id=nvme0,file=nvme://0001:01:00.0/1,format

Re: [PATCH v7 33/47] mirror: Deal with filters

2020-08-19 Thread Kevin Wolf
Am 25.06.2020 um 17:22 hat Max Reitz geschrieben: > This includes some permission limiting (for example, we only need to > take the RESIZE permission for active commits where the base is smaller > than the top). > > Use this opportunity to rename qmp_drive_mirror()'s "source" BDS to > "target_back

Re: [RFC PATCH v4 4/5] util/vfio-helpers: Introduce qemu_vfio_pci_init_msix_irqs()

2020-08-19 Thread Alex Williamson
On Wed, 19 Aug 2020 18:03:17 +0200 Philippe Mathieu-Daudé wrote: > qemu_vfio_pci_init_irq() allows us to initialize any type of IRQ, > but only one. Introduce qemu_vfio_pci_init_msix_irqs() which is > specific to MSIX IRQ type, and allow us to use multiple IRQs > (thus passing multiple eventfd no

Re: [PATCH v3] block/nbd: use non-blocking connect: fix vm hang on connect()

2020-08-19 Thread Eric Blake
On 8/12/20 9:52 AM, Vladimir Sementsov-Ogievskiy wrote: This make nbd connection_co to yield during reconnects, so that reconnect doesn't hang up the main thread. This is very important in case of unavailable nbd server host: connect() call may take a long time, blocking the main thread (and due

Re: [PATCH v7 35/47] commit: Deal with filters

2020-08-19 Thread Kevin Wolf
Am 25.06.2020 um 17:22 hat Max Reitz geschrieben: > This includes some permission limiting (for example, we only need to > take the RESIZE permission if the base is smaller than the top). > > Signed-off-by: Max Reitz > --- > block/block-backend.c | 9 +++- > block/commit.c

Re: [RFC PATCH 01/22] nbd: Remove unused nbd_export_get_blockdev()

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: Signed-off-by: Kevin Wolf --- include/block/nbd.h | 2 -- nbd/server.c| 5 - 2 files changed, 7 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu

Re: [RFC PATCH 02/22] qapi: Create block-export module

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: Move all block export related types and commands from block-core to the new QAPI module block-export. Signed-off-by: Kevin Wolf --- +++ b/qapi/block-export.json @@ -0,0 +1,172 @@ +## +# == Block device exports +## + +{ 'include': 'sockets.json' } + +

Re: [RFC PATCH 03/22] qapi: Rename BlockExport to BlockExportOptions

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: The name BlockExport will be used for the struct containing the runtime state of block exports, so change the name of export creation options. Signed-off-by: Kevin Wolf --- qapi/block-export.json | 12 ++-- block/monitor/block-hmp-cmds.c

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: We want to have a common set of commands for all types of block exports. Currently, this is only NBD, but we're going to add more types. This patch adds the basic BlockExport and BlockExportDriver structs and a QMP command block-export-add that creates a ne

Re: [RFC PATCH 05/22] qemu-storage-daemon: Use qmp_block_export_add()

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: No reason to duplicate the functionality locally, we can now just reuse the QMP command block-export-add for --export. Signed-off-by: Kevin Wolf --- qemu-storage-daemon.c | 13 + 1 file changed, 1 insertion(+), 12 deletions(-) Reviewed-by

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: Instead of implementing qemu-nbd --offset in the NBD code, just put a raw block node with the requested offset on top of the user image and rely on that doing the job. This does not only simplify the nbd_export_new() interface and bring it closer to the set

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

2020-08-19 Thread Eric Blake
cc: Peter Krempa On 8/13/20 11:29 AM, Kevin Wolf wrote: nbd-server-add tries to be convenient and adds two questionable features that we don't want to share in block-export-add, even for NBD exports: 1. When requesting a writable export of a read-only device, the export is silently downgrad

Re: [RFC PATCH 08/22] nbd: Add max-connections to nbd-server-start

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: This is a QMP equivalent of qemu-nbd's --share option, limiting the maximum number of clients that can attach at the same time. Signed-off-by: Kevin Wolf --- qapi/block-export.json | 10 -- include/block/nbd.h| 3 ++- block

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

2020-08-19 Thread Eric Blake
On 8/17/20 7:56 AM, Max Reitz wrote: On 13.08.20 18:29, Kevin Wolf wrote: qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add. Signed-off-by: Kevin Wolf ---

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: qemu-nbd allows use of writethrough cache modes, which mean that write requests made through NBD will cause a flush before they complete. Expose the same functionality in block-export-add. Signed-off-by: Kevin Wolf --- qapi/block-export.json | 7 ++-

Re: [RFC PATCH 10/22] nbd: Remove NBDExport.close callback

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: The export close callback is unused by the built-in NBD server. qemu-nbd uses it only during shutdown to wait for the unrefed export to actually go away. It can just use nbd_export_close_all() instead and do without the callback. This removes the close call

Re: [RFC PATCH 11/22] qemu-nbd: Use blk_exp_add() to create the export

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: With this change, NBD exports are only created through the BlockExport interface any more. This allows us finally to move things from the NBD s/are only/are now only/; s/any more // layer to the BlockExport layer if they make sense for other export types

Re: [RFC PATCH 12/22] nbd/server: Simplify export shutdown

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: Closing export is somewhat convoluted because nbd_export_close() and nbd_export_put() call each other and the ways they actually end up being nested is not necessarily obvious. You are in a maze of twisty little passages, all alike. Yes, I've always hated

Re: [RFC PATCH 13/22] block/export: Move refcount from NBDExport to BlockExport

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: Having a refcount makes sense for all types of block exports. It is also a prerequisite for keeping a list of all exports at the BlockExport level. Signed-off-by: Kevin Wolf --- +++ b/include/block/export.h @@ -21,14 +21,24 @@ typedef struct BlockExport

Re: [RFC PATCH 15/22] block/export: Move device to BlockExportOptions

2020-08-19 Thread Eric Blake
On 8/13/20 11:29 AM, Kevin Wolf wrote: Every block export needs a block node to export, so move the 'device' option from BlockExportOptionsNbd to BlockExportOptions. To maintain compatibility in nbd-server-add, BlockExportOptionsNbd needs to be wrapped by a new type NbdServerAddOptions that adds

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-19 Thread Brian Foster
On Wed, Aug 19, 2020 at 05:07:11PM +0200, Kevin Wolf wrote: > Am 19.08.2020 um 16:25 hat Alberto Garcia geschrieben: > > On Mon 17 Aug 2020 05:53:07 PM CEST, Kevin Wolf wrote: > > >> > Or are you saying that ZERO_RANGE + pwrite on a sparse file (= > > >> > cluster allocation) is faster for you than

[PATCH v2 02/58] megasas: Rename QOM class cast macros

2020-08-19 Thread Eduardo Habkost
Rename the MEGASAS_DEVICE_CLASS() and MEGASAS_DEVICE_GET_CLASS() macros to be consistent with the MEGASAS() instance cast macro. This will allow us to register the type cast macros using OBJECT_DECLARE_TYPE later. Reviewed-by: Daniel P. Berrangé Signed-off-by: Eduardo Habkost --- Changes v1 ->

[PATCH v2 30/58] ahci: Move QOM macros to header

2020-08-19 Thread Eduardo Habkost
The TYPE_* constants and the typedefs are defined in ahci.h, so we can move the type checking macros there too. This will make future conversion to OBJECT_DECLARE* easier. Signed-off-by: Eduardo Habkost --- Changes series v1 -> v2: new patch in series v2 Cc: John Snow Cc: qemu-block@nongnu.org

[PATCH v2 16/58] throttle-groups: Move ThrottleGroup typedef to header

2020-08-19 Thread Eduardo Habkost
Move typedef closer to the type check macros, to make it easier to convert the code to OBJECT_DEFINE_TYPE() in the future. Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Daniel P. Berrangé Signed-off-by: Eduardo Habkost --- Changes v1 -> v2: none --- Cc: Alberto Garcia Cc: Kevin Wolf Cc: M

[PATCH v2 32/58] ahci: Move QOM macro to header

2020-08-19 Thread Eduardo Habkost
Move the ALLWINNER_AHCI macro close to the TYPE_ALLWINNER_AHCI define. This will make future conversion to OBJECT_DECLARE* easier. Signed-off-by: Eduardo Habkost --- Changes series v1 -> v2: new patch in series v2 Cc: John Snow Cc: qemu-block@nongnu.org Cc: qemu-de...@nongnu.org --- include/h

Re: [PATCH v13 00/11] iotests: Dump QCOW2 dirty bitmaps metadata

2020-08-19 Thread Eric Blake
On 8/14/20 6:56 AM, Andrey Shinkevich wrote: Dear Eric! Vladimir has compeated reviewing this series. I have not received any other responses to it so far. So, is it good for pull request now? Would you please consider taking this series as you did it with the Vladimir's related one? I've

Re: [PATCH v4 1/4] migration: Add block-bitmap-mapping parameter

2020-08-19 Thread Eric Blake
On 8/18/20 8:32 AM, Max Reitz wrote: This migration parameter allows mapping block node names and bitmap names to aliases for the purpose of block dirty bitmap migration. This way, management tools can use different node and bitmap names on the source and destination and pass the mapping of how

Re: [PATCH v4 2/4] iotests.py: Add wait_for_runstate()

2020-08-19 Thread Eric Blake
On 8/18/20 8:32 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 4 1 file changed, 4 insertions(+) Reviewed-by: Eric Blake diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 717b5b652c..ee93cf22db 100644 --- a/tests/qe

Re: [PATCH v4 3/4] iotests.py: Let wait_migration() return on failure

2020-08-19 Thread Eric Blake
On 8/18/20 8:32 AM, Max Reitz wrote: Let wait_migration() return on failure (with the return value indicating whether the migration was completed or has failed), so we can use it for migrations that are expected to fail, too. Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 18 +++

Re: [PATCH v7 0/7] coroutines: generate wrapper code

2020-08-19 Thread Eric Blake
On 7/27/20 7:59 AM, Vladimir Sementsov-Ogievskiy wrote: 27.07.2020 15:48, Stefan Hajnoczi wrote: On Wed, Jun 10, 2020 at 01:03:29PM +0300, Vladimir Sementsov-Ogievskiy wrote: Hi all! The aim of the series is to reduce code-duplication and writing parameters structure-packing by hand around cor

Re: [PATCH v4 4/4] iotests: Test node/bitmap aliases during migration

2020-08-19 Thread Eric Blake
On 8/18/20 8:32 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- tests/qemu-iotests/300 | 595 + tests/qemu-iotests/300.out | 5 + Rather sparse output (I hate debugging those sorts of outputs when the test is failing). tests/qemu-iotests/grou