Re: Suspicious QOM types without instance/class size

2020-08-20 Thread David Gibson
On Thu, Aug 20, 2020 at 05:55:29PM -0400, Eduardo Habkost wrote: > While trying to convert TypeInfo declarations to the new > OBJECT_DECLARE* macros, I've stumbled on a few suspicious cases > where instance_size or class_size is not set, despite having type > checker macros that use a specific type

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-20 Thread Dave Chinner
On Thu, Aug 20, 2020 at 10:03:10PM +0200, Alberto Garcia wrote: > Cc: linux-xfs > > On Wed 19 Aug 2020 07:53:00 PM CEST, Brian Foster wrote: > > In any event, if you're seeing unclear or unexpected performance > > deltas between certain XFS configurations or other fs', I think the > > best thing t

Re: [PATCH v2 0/3] hw/sd: Add Cadence SDHCI emulation

2020-08-20 Thread Bin Meng
Hi Philippe, On Fri, Aug 21, 2020 at 2:04 AM Philippe Mathieu-Daudé wrote: > > Hi Sai Pavan, you said you were interested to test the first 2 > patches. FYI I plan to queue them and send the pull request tomorrow > or Saturday the latest. Have you got a chance to review the v2 of 3rd patch? "hw

Re: [PATCH v4 4/4] iotests: Test node/bitmap aliases during migration

2020-08-20 Thread Eric Blake
On 8/20/20 10:49 AM, Vladimir Sementsov-Ogievskiy wrote: # MYPYPATH=../../python/ mypy 300 300:154: error: Item "None" of "Optional[Match[Any]]" has no attribute "group" Found 1 error in 1 file (checked 1 source file) - the only complain. Suggest a fix: diff --git a/tests/qemu-iotests/300 b/t

Re: [PATCH v5 3/3] iotests: Test node/bitmap aliases during migration

2020-08-20 Thread Eric Blake
On 8/20/20 10:07 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- tests/qemu-iotests/300 | 595 + tests/qemu-iotests/300.out | 5 + tests/qemu-iotests/group | 1 + 3 files changed, 601 insertions(+) create mode 100755 tests/qemu-iotests/300

Re: [PATCH v5 1/3] migration: Add block-bitmap-mapping parameter

2020-08-20 Thread Eric Blake
On 8/20/20 10:07 AM, Max Reitz wrote: This migration parameter allows mapping block node names and bitmap names to aliases for the purpose of block dirty bitmap migration. This way, management tools can use different node and bitmap names on the source and destination and pass the mapping of how

[PATCH v2 4/5] qemu-iotests: Merge FilePaths and FilePath

2020-08-20 Thread Nir Soffer
FilePath creates now one temporary file: with FilePath("a") as a: Or more: with FilePath("a", "b", "c") as (a, b, c): This is also the behavior of the file_path() helper, used by some of the tests. Now we have only 2 helpers for creating temporary files instead of 3. Signed-off-by: Nir

[PATCH v2 5/5] qemu-iotests: Simplify FilePath __init__

2020-08-20 Thread Nir Soffer
Use list comprehension instead of append loop. Signed-off-by: Nir Soffer --- tests/qemu-iotests/iotests.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 1b5cdd493e..7ebd0bcc92 100644 --- a/tests/qemu-i

[PATCH v2 2/5] qemu-iotests: Fix FilePaths docstring

2020-08-20 Thread Nir Soffer
When this class was extracted from FilePath, the docstring was not updated for generating multiple files, and the example usage was referencing unrelated file. Fixes: de263986b5dc Signed-off-by: Nir Soffer --- tests/qemu-iotests/iotests.py | 14 -- 1 file changed, 8 insertions(+), 6

[PATCH v2 0/5] iotest.FilePath fixes and cleanups

2020-08-20 Thread Nir Soffer
Fix some issues introduced when iotests.FilePaths was added and merge it back into FilePath keeping the option to create multiple file names. Changes since v1: - Remove unwanted submodule change [Eric] - Add Fixes: tag v1 was here: https://lists.nongnu.org/archive/html/qemu-block/2020-08/msg00773

[PATCH v2 3/5] qemu-iotests: Support varargs syntax in FilePaths

2020-08-20 Thread Nir Soffer
Accept variable number of names instead of a sequence: with FilePaths("a", "b", "c") as (a, b, c): The disadvantage is that base_dir must be used as kwarg: with FilePaths("a", "b", base_dir=soc_dir) as (sock1, sock2): But this is more clear and calling optional argument as positional ar

[PATCH v2 1/5] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Nir Soffer
If os.remove() fails to remove one of the paths, for example if the file was removed by the test, the cleanup loop would exit silently, without removing the rest of the files. Fixes: de263986b5dc Signed-off-by: Nir Soffer --- tests/qemu-iotests/iotests.py | 8 1 file changed, 4 insertio

Suspicious QOM types without instance/class size

2020-08-20 Thread Eduardo Habkost
While trying to convert TypeInfo declarations to the new OBJECT_DECLARE* macros, I've stumbled on a few suspicious cases where instance_size or class_size is not set, despite having type checker macros that use a specific type. The ones with "WARNING" are abstract types (maybe not serious if subcl

Re: [PATCH] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Nir Soffer
On Fri, Aug 21, 2020 at 12:40 AM Nir Soffer wrote: > > On Fri, Aug 21, 2020 at 12:33 AM Eric Blake wrote: > > > > On 8/20/20 4:29 PM, Eric Blake wrote: > > > On 8/20/20 4:19 PM, Nir Soffer wrote: > > >> If os.remove() fails to remove one of the paths, for example if the file > > >> was removed by

Re: [PATCH] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Nir Soffer
On Fri, Aug 21, 2020 at 12:33 AM Eric Blake wrote: > > On 8/20/20 4:29 PM, Eric Blake wrote: > > On 8/20/20 4:19 PM, Nir Soffer wrote: > >> If os.remove() fails to remove one of the paths, for example if the file > >> was removed by the test, the cleanup loop would exit silently, without > >> remo

Re: [PATCH] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Eric Blake
On 8/20/20 4:29 PM, Eric Blake wrote: On 8/20/20 4:19 PM, Nir Soffer wrote: If os.remove() fails to remove one of the paths, for example if the file was removed by the test, the cleanup loop would exit silently, without removing the rest of the files. Signed-off-by: Nir Soffer ---   dtc   

Re: [PATCH] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Eric Blake
On 8/20/20 4:19 PM, Nir Soffer wrote: If os.remove() fails to remove one of the paths, for example if the file was removed by the test, the cleanup loop would exit silently, without removing the rest of the files. Signed-off-by: Nir Soffer --- dtc | 2 +- tests/qemu

Re: [PATCH] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Nir Soffer
On Fri, Aug 21, 2020 at 12:19 AM Nir Soffer wrote: > > If os.remove() fails to remove one of the paths, for example if the file > was removed by the test, the cleanup loop would exit silently, without > removing the rest of the files. > > Signed-off-by: Nir Soffer > --- > dtc

[PATCH] qemu-iotests: Fix FilePaths cleanup

2020-08-20 Thread Nir Soffer
If os.remove() fails to remove one of the paths, for example if the file was removed by the test, the cleanup loop would exit silently, without removing the rest of the files. Signed-off-by: Nir Soffer --- dtc | 2 +- tests/qemu-iotests/iotests.py | 8 2 files

Re: [PATCH v4 08/10] iotests.py: add verify_o_direct helper

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
21.08.2020 00:00, Nir Soffer wrote: On Thu, Aug 20, 2020 at 9:49 PM Vladimir Sementsov-Ogievskiy wrote: Add python notrun-helper similar to _check_o_direct for bash tests. To be used in the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Stefan Hajnoczi --- test

Re: [PATCH v4 08/10] iotests.py: add verify_o_direct helper

2020-08-20 Thread Nir Soffer
On Thu, Aug 20, 2020 at 9:49 PM Vladimir Sementsov-Ogievskiy wrote: > > Add python notrun-helper similar to _check_o_direct for bash tests. > To be used in the following commit. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > Reviewed-by: Stefan Hajnoczi > --- > tests/qemu-iotests/iotests.py

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-20 Thread Alberto Garcia
Cc: linux-xfs On Wed 19 Aug 2020 07:53:00 PM CEST, Brian Foster wrote: > In any event, if you're seeing unclear or unexpected performance > deltas between certain XFS configurations or other fs', I think the > best thing to do is post a more complete description of the workload, > filesystem/stora

Re: [PATCH v4 00/10] preallocate filter

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
20.08.2020 22:16, no-re...@patchew.org wrote: Patchew URL: https://patchew.org/QEMU/20200820183950.13109-1-vsement...@virtuozzo.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can pro

Re: [PATCH v4 00/10] preallocate filter

2020-08-20 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200820183950.13109-1-vsement...@virtuozzo.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEG

[PATCH v4 09/10] iotests.py: add filter_img_check

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
Add analog of bash _filter_qemu_img_check to python framework. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/iotests.py | 4 1 file changed, 4 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 3

[PATCH v4 08/10] iotests.py: add verify_o_direct helper

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
Add python notrun-helper similar to _check_o_direct for bash tests. To be used in the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/iotests.py | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/qemu-iotests/iotests

[PATCH v4 07/10] block: introduce preallocate filter

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
It's intended to be inserted between format and protocol nodes to preallocate additional space (expanding protocol file) on writes crossing EOF. It improves performance for file-systems with slow allocation. Signed-off-by: Vladimir Sementsov-Ogievskiy --- docs/system/qemu-block-drivers.rst.inc |

[PATCH v4 06/10] block: introduce BDRV_REQ_NO_WAIT flag

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
Add flag to make serialising request no wait: if there are conflicting requests, just return error immediately. It's will be used in upcoming preallocate filter. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 9 - block/io.c| 11 ++- 2 files

[PATCH v4 10/10] iotests: add 298 to test new preallocate filter driver

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Stefan Hajnoczi --- tests/qemu-iotests/298 | 50 ++ tests/qemu-iotests/298.out | 6 + tests/qemu-iotests/group | 1 + 3 files changed, 57 insertions(+) create mode 100644 tests/qemu-iotests/

[PATCH v4 04/10] block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
bs is linked in req, so no needs to pass it separately. Most of tracked-requests API doesn't have bs argument. Actually, after this patch only tracked_request_begin has it, but it's for purpose. While being here, also add a comment about what "_locked" is. Signed-off-by: Vladimir Sementsov-Ogievs

[PATCH v4 01/10] block: simplify comment to BDRV_REQ_SERIALISING

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
1. BDRV_REQ_NO_SERIALISING doesn't exist already, don't mention it. 2. We are going to add one more user of BDRV_REQ_SERIALISING, so comment about backup becomes a bit confusing here. The use case in backup is documented in block/backup.c, so let's just drop duplication here. 3. The fact

[PATCH v4 02/10] block/io.c: drop assertion on double waiting for request serialisation

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
The comments states, that on misaligned request we should have already been waiting. But for bdrv_padding_rmw_read, we called bdrv_mark_request_serialising with align = request_alignment, and now we serialise with align = cluster_size. So we may have to wait again with larger alignment. Note, that

[PATCH v4 05/10] block: bdrv_mark_request_serialising: split non-waiting function

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
We'll need a separate function, which will only "mark" request serialising with specified align but not wait for conflicting requests. So, it will be like old bdrv_mark_request_serialising(), before merging bdrv_wait_serialising_requests_locked() into it. To reduce the possible mess, let's do the

[PATCH v4 03/10] block/io: split out bdrv_find_conflicting_request

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
To be reused in separate. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Stefan Hajnoczi --- block/io.c | 71 +++--- 1 file changed, 41 insertions(+), 30 deletions(-) diff --git a/block/io.c b/block/io.c index b18680a842..5b96715058 100

[PATCH v4 00/10] preallocate filter

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
Hi all! Here is a filter, which does preallocation on write. In Virtuozzo we have to deal with some custom distributed storage solution, where allocation is relatively expensive operation. We have to workaround it in Qemu, so here is a new filter. v4: 01-04: add r-bs 05: add coroutine_fn tag 06:

Re: [PATCH v2 0/3] hw/sd: Add Cadence SDHCI emulation

2020-08-20 Thread Philippe Mathieu-Daudé
Hi Sai Pavan, you said you were interested to test the first 2 patches. FYI I plan to queue them and send the pull request tomorrow or Saturday the latest. On 8/17/20 12:03 PM, Bin Meng wrote: > This series is spun off from the following series as it is hw/sd > centric, so that it can be picked up

[PATCH v5 15/15] block/nvme: Use an array of EventNotifier

2020-08-20 Thread Philippe Mathieu-Daudé
In preparation of using multiple IRQ (thus multiple eventfds) make BDRVNVMeState::irq_notifier an array (for now of a single element, the admin queue notifier). Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 31 +-- 1 file chang

[PATCH v5 11/15] block/nvme: Simplify nvme_init_queue() arguments

2020-08-20 Thread Philippe Mathieu-Daudé
nvme_init_queue() doesn't require BlockDriverState anymore. Replace it by BDRVNVMeState to simplify. Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index f

[PATCH v5 13/15] block/nvme: Simplify nvme_create_queue_pair() arguments

2020-08-20 Thread Philippe Mathieu-Daudé
nvme_create_queue_pair() doesn't require BlockDriverState anymore. Replace it by BDRVNVMeState and AioContext to simplify. Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 13 +++-- 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/block/

[PATCH v5 07/15] block/nvme: Rename local variable

2020-08-20 Thread Philippe Mathieu-Daudé
We are going to modify the code in the next commit. Renaming the 'resp' variable to 'id' first makes the next commit easier to review. No logical changes. Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 19 +-- 1 file changed, 9 insertions(+

[PATCH v5 06/15] block/nvme: Use common error path in nvme_add_io_queue()

2020-08-20 Thread Philippe Mathieu-Daudé
Rearrange nvme_add_io_queue() by using a common error path. This will be proven useful in few commits where we add IRQ notification to the IO queues. Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-

[PATCH v5 09/15] block/nvme: Replace qemu_try_blockalign0 by qemu_try_blockalign/memset

2020-08-20 Thread Philippe Mathieu-Daudé
In the next commit we'll get rid of qemu_try_blockalign(). To ease review, first replace qemu_try_blockalign0() by explicit calls to qemu_try_blockalign() and memset(). Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 16 +--- 1 file changed, 9 i

[PATCH v5 10/15] block/nvme: Replace qemu_try_blockalign(bs) by qemu_try_memalign(pg_sz)

2020-08-20 Thread Philippe Mathieu-Daudé
qemu_try_blockalign() is a generic API that call back to the block driver to return its page alignment. As we call from within the very same driver, we already know to page alignment stored in our state. Remove indirections and use the value from BDRVNVMeState. This change is required to later remo

[PATCH v5 05/15] block/nvme: Improve error message when IO queue creation failed

2020-08-20 Thread Philippe Mathieu-Daudé
Do not use the same error message for different failures. Display a different error whether it is the CQ or the SQ. Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/nvme.c b/block/nv

[PATCH v5 14/15] block/nvme: Extract nvme_poll_queue()

2020-08-20 Thread Philippe Mathieu-Daudé
As we want to do per-queue polling, extract the nvme_poll_queue() method which operates on a single queue. Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 44 +++- 1 file changed, 27 insertions(+), 17 deletions(-) di

[PATCH v5 12/15] block/nvme: Replace BDRV_POLL_WHILE by AIO_WAIT_WHILE

2020-08-20 Thread Philippe Mathieu-Daudé
BDRV_POLL_WHILE() is defined as: #define BDRV_POLL_WHILE(bs, cond) ({ \ BlockDriverState *bs_ = (bs); \ AIO_WAIT_WHILE(bdrv_get_aio_context(bs_), \ cond); }) As we will remove the BlockDriverState use in the next commit, start by using the e

[PATCH v5 04/15] block/nvme: Define INDEX macros to ease code review

2020-08-20 Thread Philippe Mathieu-Daudé
Use definitions instead of '0' or '1' indexes. Also this will be useful when using multi-queues later. Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 33 +++-- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/bloc

[PATCH v5 08/15] block/nvme: Use union of NvmeIdCtrl / NvmeIdNs structures

2020-08-20 Thread Philippe Mathieu-Daudé
We allocate an unique chunk of memory then use it for two different structures. By using an union, we make it clear the data is overlapping (and we can remove the casts). Suggested-by: Stefan Hajnoczi Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 31

[PATCH v5 02/15] block/nvme: Avoid further processing if trace event not enabled

2020-08-20 Thread Philippe Mathieu-Daudé
Avoid further processing if TRACE_NVME_SUBMIT_COMMAND_RAW is not enabled. This is an untested intend of performance optimization. Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/nvme.c b/block/nvme.

[PATCH v5 01/15] block/nvme: Replace magic value by SCALE_MS definition

2020-08-20 Thread Philippe Mathieu-Daudé
Use self-explicit SCALE_MS definition instead of magic value. Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/nvme.c b/block/nvme.c index 374e2689157..2f5e3c2adfa 100644 --- a/block/nvm

[PATCH v5 03/15] block/nvme: Let nvme_create_queue_pair() fail gracefully

2020-08-20 Thread Philippe Mathieu-Daudé
As nvme_create_queue_pair() is allowed to fail, replace the alloc() calls by try_alloc() to avoid aborting QEMU. Reviewed-by: Stefan Hajnoczi Signed-off-by: Philippe Mathieu-Daudé --- block/nvme.c | 12 ++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/block/nvme.c b/b

[PATCH v5 00/15] block/nvme: Various cleanups required to use multiple queues

2020-08-20 Thread Philippe Mathieu-Daudé
Hi Kevin, This series is mostly code rearrangement (cleanups) to be able to split the hardware code from the block driver code, to be able to use multiple queues on the same hardware, or multiple block drivers on the same hardware. All this series is reviewed. Since v4: - added 'block/nvme: Use

Re: [PATCH v5 3/3] iotests: Test node/bitmap aliases during migration

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
20.08.2020 18:07, Max Reitz wrote: Signed-off-by: Max Reitz Sorry, I missed v5 and answered on v4. Still, the only change is s/write/discard/ (good change, I'm for), so please refer to my answer on v4 for other comments. -- Best regards, Vladimir

Re: [PATCH v5 1/3] migration: Add block-bitmap-mapping parameter

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
20.08.2020 18:07, Max Reitz wrote: This migration parameter allows mapping block node names and bitmap names to aliases for the purpose of block dirty bitmap migration. This way, management tools can use different node and bitmap names on the source and destination and pass the mapping of how bi

Re: [PATCH v4 4/4] iotests: Test node/bitmap aliases during migration

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
# MYPYPATH=../../python/ mypy 300 300:154: error: Item "None" of "Optional[Match[Any]]" has no attribute "group" Found 1 error in 1 file (checked 1 source file) - the only complain. Suggest a fix: diff --git a/tests/qemu-iotests/300 b/tests/qemu-iotests/300 index c6d86b1dbc..0241903743 100755 --

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

2020-08-20 Thread Peter Krempa
On Thu, Aug 20, 2020 at 09:41:14 -0500, Eric Blake wrote: > On 8/20/20 6:05 AM, Kevin Wolf wrote: > > > As long as we can keep the compatibility code local to qmp_nbd_*(), I > > don't think it's too bad. In particular because it's already written. > > > > Instead of adjusting libvirt to changes i

[PATCH v5 2/3] iotests.py: Let wait_migration() return on failure

2020-08-20 Thread Max Reitz
Let wait_migration() return on failure (with the return value indicating whether the migration was completed or has failed), so we can use it for migrations that are expected to fail, too. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu

[PATCH v5 3/3] iotests: Test node/bitmap aliases during migration

2020-08-20 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/300 | 595 + tests/qemu-iotests/300.out | 5 + tests/qemu-iotests/group | 1 + 3 files changed, 601 insertions(+) create mode 100755 tests/qemu-iotests/300 create mode 100644 tests/qemu-iotests/300.out

[PATCH v5 0/3] migration: Add block-bitmap-mapping parameter

2020-08-20 Thread Max Reitz
RFC v1: https://lists.nongnu.org/archive/html/qemu-block/2020-05/msg00912.html RFC v2: https://lists.nongnu.org/archive/html/qemu-block/2020-05/msg00915.html v1: https://lists.nongnu.org/archive/html/qemu-devel/2020-06/msg09792.html v2: https://lists.nongnu.org/archive/html/qemu-block/2020-07/msg01

[PATCH v5 1/3] migration: Add block-bitmap-mapping parameter

2020-08-20 Thread Max Reitz
This migration parameter allows mapping block node names and bitmap names to aliases for the purpose of block dirty bitmap migration. This way, management tools can use different node and bitmap names on the source and destination and pass the mapping of how bitmaps are to be transferred to qemu (

Re: [RFC PATCH v4 1/5] block/nvme: Use an array of EventNotifier

2020-08-20 Thread Stefan Hajnoczi
On Wed, Aug 19, 2020 at 06:03:14PM +0200, Philippe Mathieu-Daudé wrote: > In preparation of using multiple IRQ (thus multiple eventfds) > make BDRVNVMeState::irq_notifier an array (for now of a single > element, the admin queue notifier). > > Signed-off-by: Philippe Mathieu-Daudé > --- > block/n

Re: [PATCH v4 2/4] iotests.py: Add wait_for_runstate()

2020-08-20 Thread Max Reitz
On 20.08.20 16:34, Vladimir Sementsov-Ogievskiy wrote: > 20.08.2020 17:23, Dr. David Alan Gilbert wrote: >> * Eric Blake (ebl...@redhat.com) wrote: >>> On 8/18/20 8:32 AM, Max Reitz wrote: Signed-off-by: Max Reitz ---    tests/qemu-iotests/iotests.py | 4    1 file changed,

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

2020-08-20 Thread Eric Blake
On 8/20/20 6:05 AM, Kevin Wolf wrote: As long as we can keep the compatibility code local to qmp_nbd_*(), I don't think it's too bad. In particular because it's already written. Instead of adjusting libvirt to changes in the nbd-* commands, I'd rather have it change over to block-export-*. I wo

Re: [PATCH v4 2/4] iotests.py: Add wait_for_runstate()

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
20.08.2020 17:23, Dr. David Alan Gilbert wrote: * Eric Blake (ebl...@redhat.com) wrote: On 8/18/20 8:32 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 4 1 file changed, 4 insertions(+) Reviewed-by: Eric Blake diff --git a/tests/qemu-iotests/i

Re: [PATCH v4 2/4] iotests.py: Add wait_for_runstate()

2020-08-20 Thread Dr. David Alan Gilbert
* Eric Blake (ebl...@redhat.com) wrote: > On 8/18/20 8:32 AM, Max Reitz wrote: > > Signed-off-by: Max Reitz > > --- > > tests/qemu-iotests/iotests.py | 4 > > 1 file changed, 4 insertions(+) > > Reviewed-by: Eric Blake > > > > > diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-i

Re: [RFC PATCH 13/22] block/export: Move refcount from NBDExport to BlockExport

2020-08-20 Thread Kevin Wolf
Am 19.08.2020 um 22:58 hat Eric Blake geschrieben: > On 8/13/20 11:29 AM, Kevin Wolf wrote: > > Having a refcount makes sense for all types of block exports. It is also > > a prerequisite for keeping a list of all exports at the BlockExport > > level. > > > > Signed-off-by: Kevin Wolf > > --- >

Re: [PATCH v13 00/11] iotests: Dump QCOW2 dirty bitmaps metadata

2020-08-20 Thread Andrey Shinkevich
On 20.08.2020 03:49, Eric Blake wrote: On 8/14/20 6:56 AM, Andrey Shinkevich wrote: Dear Eric! Vladimir has compeated reviewing this series. I have not received any other responses to it so far. So, is it good for pull request now? Would you please consider taking this series as you did it

Re: [PATCH v4 4/4] iotests: Test node/bitmap aliases during migration

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
20.08.2020 16:17, Max Reitz wrote: On 20.08.20 03:58, Eric Blake wrote: On 8/18/20 8:32 AM, Max Reitz wrote: Signed-off-by: Max Reitz ---   tests/qemu-iotests/300 | 595 +   tests/qemu-iotests/300.out |   5 + Rather sparse output (I hate debugging tho

Re: [PATCH v7 35/47] commit: Deal with filters

2020-08-20 Thread Kevin Wolf
Am 20.08.2020 um 13:27 hat Max Reitz geschrieben: > On 19.08.20 19:58, Kevin Wolf wrote: > > Am 25.06.2020 um 17:22 hat Max Reitz geschrieben: > >> This includes some permission limiting (for example, we only need to > >> take the RESIZE permission if the base is smaller than the top). > >> > >> Si

Re: [PATCH v4 3/4] iotests.py: Let wait_migration() return on failure

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
18.08.2020 16:32, Max Reitz wrote: Let wait_migration() return on failure (with the return value indicating whether the migration was completed or has failed), so we can use it for migrations that are expected to fail, too. Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 18 +

Re: [PATCH v4 2/4] iotests.py: Add wait_for_runstate()

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
18.08.2020 16:32, Max Reitz wrote: Signed-off-by: Max Reitz Reviewed-by: Vladimir Sementsov-Ogievskiy -- Best regards, Vladimir

Re: [PATCH v4 1/4] migration: Add block-bitmap-mapping parameter

2020-08-20 Thread Max Reitz
On 20.08.20 14:58, Vladimir Sementsov-Ogievskiy wrote: > 18.08.2020 16:32, Max Reitz wrote: >> This migration parameter allows mapping block node names and bitmap >> names to aliases for the purpose of block dirty bitmap migration. >> >> This way, management tools can use different node and bitmap

Re: [PATCH v4 4/4] iotests: Test node/bitmap aliases during migration

2020-08-20 Thread Max Reitz
On 20.08.20 03:58, Eric Blake wrote: > On 8/18/20 8:32 AM, Max Reitz wrote: >> Signed-off-by: Max Reitz >> --- >>   tests/qemu-iotests/300 | 595 + >>   tests/qemu-iotests/300.out |   5 + > > Rather sparse output (I hate debugging those sorts of outputs when

Re: [PATCH v4 1/4] migration: Add block-bitmap-mapping parameter

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
18.08.2020 16:32, Max Reitz wrote: This migration parameter allows mapping block node names and bitmap names to aliases for the purpose of block dirty bitmap migration. This way, management tools can use different node and bitmap names on the source and destination and pass the mapping of how bi

Re: [PATCH v4 1/4] migration: Add block-bitmap-mapping parameter

2020-08-20 Thread Max Reitz
On 20.08.20 03:17, Eric Blake wrote: > On 8/18/20 8:32 AM, Max Reitz wrote: >> This migration parameter allows mapping block node names and bitmap >> names to aliases for the purpose of block dirty bitmap migration. >> >> This way, management tools can use different node and bitmap names on >> the

Re: [PATCH v7 14/47] stream: Deal with filters

2020-08-20 Thread Max Reitz
On 20.08.20 12:49, Vladimir Sementsov-Ogievskiy wrote: > 20.08.2020 12:22, Max Reitz wrote: >> On 20.08.20 10:31, Max Reitz wrote: >> >> [...] >> >>> So all in all, I believe the biggest surprise about what’s written into >>> the top layer isn’t that it may be a json:{} filename, but the filename >

Re: [PATCH v7 35/47] commit: Deal with filters

2020-08-20 Thread Max Reitz
On 19.08.20 19:58, Kevin Wolf wrote: > Am 25.06.2020 um 17:22 hat Max Reitz geschrieben: >> This includes some permission limiting (for example, we only need to >> take the RESIZE permission if the base is smaller than the top). >> >> Signed-off-by: Max Reitz >> --- >> block/block-backend.c

Re: [RFC PATCH 08/22] nbd: Add max-connections to nbd-server-start

2020-08-20 Thread Kevin Wolf
Am 19.08.2020 um 22:00 hat Eric Blake geschrieben: > On 8/13/20 11:29 AM, Kevin Wolf wrote: > > This is a QMP equivalent of qemu-nbd's --share option, limiting the > > maximum number of clients that can attach at the same time. > > > > Signed-off-by: Kevin Wolf > > --- > > qapi/block-export.jso

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

2020-08-20 Thread Kevin Wolf
Am 19.08.2020 um 21:50 hat Eric Blake geschrieben: > cc: Peter Krempa > > On 8/13/20 11:29 AM, Kevin Wolf wrote: > > nbd-server-add tries to be convenient and adds two questionable > > features that we don't want to share in block-export-add, even for NBD > > exports: > > > > 1. When requesting a

Re: [PATCH v7 14/47] stream: Deal with filters

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
20.08.2020 12:22, Max Reitz wrote: On 20.08.20 10:31, Max Reitz wrote: [...] So all in all, I believe the biggest surprise about what’s written into the top layer isn’t that it may be a json:{} filename, but the filename of a node that maybe doesn’t even exist anymore? (Oh, no, please don’t t

Re: [PATCH v3] block/nbd: use non-blocking connect: fix vm hang on connect()

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
19.08.2020 20:52, Eric Blake wrote: On 8/12/20 9:52 AM, Vladimir Sementsov-Ogievskiy wrote: This make nbd connection_co to yield during reconnects, so that reconnect doesn't hang up the main thread. This is very important in case of unavailable nbd server host: connect() call may take a long tim

Re: [PATCH v7 33/47] mirror: Deal with filters

2020-08-20 Thread Max Reitz
On 19.08.20 18:50, Kevin Wolf wrote: > Am 25.06.2020 um 17:22 hat Max Reitz geschrieben: >> This includes some permission limiting (for example, we only need to >> take the RESIZE permission for active commits where the base is smaller >> than the top). >> >> Use this opportunity to rename qmp_driv

Re: [PATCH v3] block/nbd: use non-blocking connect: fix vm hang on connect()

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
19.08.2020 17:46, Eric Blake wrote: On 8/12/20 9:52 AM, Vladimir Sementsov-Ogievskiy wrote: This make nbd connection_co to yield during reconnects, so that s/make nbd connection_co to/makes nbd's connection_co/ reconnect doesn't hang up the main thread. This is very important in s/hang up/

Re: [PATCH v3 12/12] block/qcow2: automatically insert preallocate filter when on FUSE

2020-08-20 Thread Vladimir Sementsov-Ogievskiy
19.08.2020 18:15, Stefan Hajnoczi wrote: On Mon, Aug 17, 2020 at 12:15:53PM +0300, Vladimir Sementsov-Ogievskiy wrote: vstorage has slow allocation, so this patch detect vstorage (I hope, we don't use other FUSE filesystems) and inserts preallocate filter between qcow2 node and its file child.

Re: [PATCH v7 14/47] stream: Deal with filters

2020-08-20 Thread Max Reitz
On 20.08.20 10:31, Max Reitz wrote: [...] > So all in all, I believe the biggest surprise about what’s written into > the top layer isn’t that it may be a json:{} filename, but the filename > of a node that maybe doesn’t even exist anymore? (Oh, no, please don’t > tell me you can delete it and g

Re: [PULL 0/2] Block patches for 5.1.0-rc4

2020-08-20 Thread Peter Maydell
On Tue, 11 Aug 2020 at 10:35, Max Reitz wrote: > > Hi, > > There is a bug in the backup job that breaks backups from images whose > size is not aligned to the job's cluster size (i.e., qemu crashes > because of a failed assertion). If this bug makes it into the release, > it would be a regression

Re: [PATCH v7 14/47] stream: Deal with filters

2020-08-20 Thread Max Reitz
On 19.08.20 17:16, Kevin Wolf wrote: > Am 19.08.2020 um 16:47 hat Max Reitz geschrieben: >> On 18.08.20 16:28, Kevin Wolf wrote: >>> Am 25.06.2020 um 17:21 hat Max Reitz geschrieben: Because of the (not so recent anymore) changes that make the stream job independent of the base node and i

Re: [PATCH 2/3] block: add logging facility for long standing IO requests

2020-08-20 Thread David Edmondson
On Monday, 2020-08-10 at 13:14:46 +03, Denis V. Lunev wrote: > +strftime(buf, sizeof(buf), "%m-%d %H:%M:%S", "%F %T" would include the year, which can be useful. > + localtime_r(&start_time_host_s, &t)); > + > +bs = blk_bs(blk_stats2blk(stats)); > +qemu_log("long %s[%ld]

Re: [PATCH v8 0/6] block: seriously improve savevm/loadvm performance

2020-08-20 Thread Denis V. Lunev
On 7/9/20 4:26 PM, Denis V. Lunev wrote: > This series do standard basic things: > - it creates intermediate buffer for all writes from QEMU migration code > to QCOW2 image, > - this buffer is sent to disk asynchronously, allowing several writes to > run in parallel. > > In general, migration c

Re: [PATCH v7 4/7] scripts: add coroutine-wrapper.py

2020-08-20 Thread Paolo Bonzini
On 10/06/20 12:03, Vladimir Sementsov-Ogievskiy wrote: > +{struct_name} s = {{ > +.poll_state.bs = {bs}, > +.poll_state.in_progress = true, > + > +{ func.gen_block('.{name} = {name},') } > +}}; > + > +s.poll_state.co = qemu_coroutine_creat

Re: [PATCH v2 for 5.2 0/3] block: add logging facility for long standing IO requests

2020-08-20 Thread Denis V. Lunev
On 8/12/20 5:00 PM, Stefan Hajnoczi wrote: > On Mon, Aug 10, 2020 at 01:14:44PM +0300, Denis V. Lunev wrote: >> There are severe delays with IO requests processing if QEMU is running in >> virtual machine or over software defined storage. Such delays potentially >> results in unpredictable guest be

Re: [PATCH v7 0/7] coroutines: generate wrapper code

2020-08-20 Thread Paolo Bonzini
On 20/08/20 03:33, Eric Blake wrote: >>> >> >> OK, will do. Thanks for taking a look! > > As this series touched Makefile to add a generated .c, you'll also need > to rebase that part to apply on top of Paolo's meson conversion (cc'ing > him if you need help figuring it out) It should be trivial