Clarification regarding new qemu-img convert --target-is-zero flag

2020-06-09 Thread Sam Eiderman
Hi, 168468fe19c8 ("qemu-img: Add --target-is-zero to convert") has added a nice functionality for cloud scenarios: * Create a virtual disk * Convert a sparse image (qcow2, vmdk) to the virtual disk using --target-is-zero * Use the virtual disk This saves many unnecessary writes - a qcow2 with

Re: [PATCH v6 4/7] scripts: add coroutine-wrapper.py

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
10.06.2020 00:21, Eric Blake wrote: On 6/9/20 9:13 AM, Vladimir Sementsov-Ogievskiy wrote: We have a very frequent pattern of creating coroutine from function with several arguments:    - create structure to pack parameters    - create _entry function to call original function taking

Re: [PATCH v2 0/7] virtio: enable VIRTIO_F_RING_PACKED for all devices

2020-06-09 Thread Michael S. Tsirkin
On Tue, Jun 09, 2020 at 06:02:11PM +0100, Stefan Hajnoczi wrote: > v2: > * Add libvhost-user VHOST_USER_GET_FEATURES patch to start reporting feature >bits correctly (already reviewed by Marc-André Lureau) > * Rephrase doc comments and drop vhost_get_default_features() [Jason] > * Simplify

Re: [PATCH v2 3/7] docs: document non-net VHOST_USER_GET_FEATURES behavior

2020-06-09 Thread Michael S. Tsirkin
On Tue, Jun 09, 2020 at 06:02:14PM +0100, Stefan Hajnoczi wrote: > QEMU enabled several feature bits for non-net devices without allowing > the device backend to control them. This only works when the device > backend implements support for those features. It won't work for new > features like the

[PULL 35/56] vhost-user-blk: delay vhost_user_blk_disconnect

2020-06-09 Thread Michael S. Tsirkin
From: Dima Stepanov A socket write during vhost-user communication may trigger a disconnect event, calling vhost_user_blk_disconnect() and clearing all the vhost_dev structures holding data that vhost-user functions expect to remain valid to roll back initialization correctly. Delay the cleanup

Re: [PATCH v2 4/7] vhost: involve device backends in feature negotiation

2020-06-09 Thread Michael S. Tsirkin
On Wed, Jun 10, 2020 at 11:21:50AM +0800, Jason Wang wrote: > > On 2020/6/10 上午2:07, Michael S. Tsirkin wrote: > > > +/* > > > + * Default vhost_get_features() feature bits for existing device types > > > that do > > > + * not define their own. > > > + * > > > + * This is a workaround for

Re: [PATCH v2 4/7] vhost: involve device backends in feature negotiation

2020-06-09 Thread Jason Wang
On 2020/6/10 上午2:07, Michael S. Tsirkin wrote: +/* + * Default vhost_get_features() feature bits for existing device types that do + * not define their own. + * + * This is a workaround for existing device types, do not use this in new vhost + * device types. Explicitly define a list of

[PULL 1/3] iotests: 194: wait for migration completion on target too

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy It is possible, that shutdown on target occurs earlier than migration finish. In this case we crash in bdrv_release_dirty_bitmap_locked() on assertion "assert(!bdrv_dirty_bitmap_busy(bitmap));" as we do have busy bitmap, as bitmap migration is ongoing. We'll

[PULL 3/3] block: Call attention to truncation of long NBD exports

2020-06-09 Thread Eric Blake
Commit 93676c88 relaxed our NBD client code to request export names up to the NBD protocol maximum of 4096 bytes without NUL terminator, even though the block layer can't store anything longer than 4096 bytes including NUL terminator for display to the user. Since this means there are some export

[PULL 2/3] nbd/server: Avoid long error message assertions CVE-2020-10761

2020-06-09 Thread Eric Blake
Ever since commit 36683283 (v2.8), the server code asserts that error strings sent to the client are well-formed per the protocol by not exceeding the maximum string length of 4096. At the time the server first started sending error messages, the assertion could not be triggered, because messages

Re: [PATCH v2 08/20] qemu-storage-daemon: Add --object option

2020-06-09 Thread Coiby Xu
On Mon, Feb 24, 2020 at 03:29:56PM +0100, Kevin Wolf wrote: Add a command line option to create user-creatable QOM objects. Signed-off-by: Kevin Wolf --- qemu-storage-daemon.c | 47 +++ Makefile.objs | 2 +- qom/Makefile.objs | 1 + 3 files

Re: [PATCH v6 4/7] scripts: add coroutine-wrapper.py

2020-06-09 Thread Eric Blake
On 6/9/20 9:13 AM, Vladimir Sementsov-Ogievskiy wrote: We have a very frequent pattern of creating coroutine from function with several arguments: - create structure to pack parameters - create _entry function to call original function taking parameters from struct - do different

[PULL 13/15] qcow2: QcowHeaderExtension print names for extension magics

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Suggested-by: Andrey Shinkevich Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20200606081806.23897-13-vsement...@virtuozzo.com> Reviewed-by: Eric Blake Signed-off-by: Eric Blake --- tests/qemu-iotests/031.out | 22 +++---

[PATCH] iotests: Add copyright line in qcow2.py

2020-06-09 Thread Eric Blake
The file qcow2.py was originally contributed in 2012 by Kevin Wolf, but was not given traditional boilerplate headers at the time. The missing license was just rectified (commit 16306a7b39) using the project-default GPLv2+, but as Vladimir is not at Red Hat, he did not add a Copyright line. All

[PULL 14/15] qcow2_format.py: dump bitmaps header extension

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Add class for bitmap extension and dump its fields. Further work is to dump bitmap directory. Test new functionality inside 291 iotest. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id:

[PULL 09/15] qcow2_format.py: separate generic functionality of structure classes

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy We are going to introduce more Qcow2 structure types, defined like QcowHeader. Move generic functionality into base class to be reused for further structure classes. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id:

[PULL 08/15] qcow2_format.py: use strings to specify c-type of struct fields

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy We are going to move field-parsing to super-class, this will be simpler with simple string specifiers instead of variables. For some reason, python doesn't allow the definition of ctypes variable in the class alongside fields: it would not be available then

[PULL 12/15] qcow2_format: refactor QcowHeaderExtension as a subclass of Qcow2Struct

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Only two fields we can parse by generic code, but that is better than nothing. Keep further refactoring of variable-length fields for another day. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id:

[PULL 06/15] qcow2_format.py: use tuples instead of lists for fields

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy No need in lists: it's a constant variable. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id: <20200606081806.23897-6-vsement...@virtuozzo.com> Signed-off-by: Eric Blake --- tests/qemu-iotests/qcow2_format.py | 40

[PULL 07/15] qcow2_format.py: use modern string formatting

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Use .format and f-strings instead of old %style. Also, the file uses both '' and "" quotes, for consistency let's use '', except for cases when we need '' inside the string (use "" to avoid extra escaping). Signed-off-by: Vladimir Sementsov-Ogievskiy

[PULL 15/15] iotests: Fix 291 across more file systems

2020-06-09 Thread Eric Blake
Depending on the granularity of holes and amount of metadata consumed by a file, the 'disk size:' number of 'qemu-img info' is not reliable. Adjust our test to use a different set of filters to avoid spurious failures. Reported-by: Kevin Wolf Fixes: cf2d1203dc Signed-off-by: Eric Blake

[PULL 10/15] qcow2_format.py: add field-formatting class

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Allow formatter class in structure definition instead of hacking with 'mask'. This will simplify further introduction of new formatters. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id:

[PULL 03/15] qcow2.py: add licensing blurb

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Add classic heading, which is missing here. Keep copyright place empty, prior authors may add a line later. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20200606081806.23897-3-vsement...@virtuozzo.com> Reviewed-by: Eric Blake [eblake: tweak

[PULL 11/15] qcow2_format.py: QcowHeaderExtension: add dump method

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Obviously, for-loop body in dump_extensions should be the dump method of extension. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id: <20200606081806.23897-11-vsement...@virtuozzo.com> Signed-off-by: Eric Blake ---

[PULL 04/15] qcow2.py: move qcow2 format classes to separate module

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy We are going to enhance qcow2 format parsing by adding more structure classes. Let's split format parsing from utility code. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20200606081806.23897-4-vsement...@virtuozzo.com> Reviewed-by: Eric Blake

[PULL 02/15] qcow2.py: python style fixes

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Fix flake8 complaints. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20200606081806.23897-2-vsement...@virtuozzo.com> Tested-by: Eric Blake Reviewed-by: Eric Blake [eblake: commit message improved] Signed-off-by: Eric Blake ---

[PULL 05/15] qcow2_format.py: drop new line printing at end of dump()

2020-06-09 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy This will simplify further conversion. To compensate, print this empty line directly in cmd_dump_header(). Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Andrey Shinkevich Message-Id: <20200606081806.23897-5-vsement...@virtuozzo.com> Signed-off-by:

Re: [PATCH] iotests: Fix 291 across more file systems

2020-06-09 Thread Eric Blake
On 6/9/20 8:32 AM, Kevin Wolf wrote: Am 08.06.2020 um 21:56 hat Eric Blake geschrieben: Depending on the granularity of holes and amount of metadata consumed by a file, the 'disk size:' number of 'qemu-img info' is not reliable. Adjust our test to use a different set of filters to avoid

Re: [PATCH v6 5/7] block: generate coroutine-wrapper code

2020-06-09 Thread Eric Blake
On 6/9/20 9:13 AM, Vladimir Sementsov-Ogievskiy wrote: Use code generation implemented in previous commit to generated coroutine wrappers in block.c and block/io.c Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/coroutines.h| 7 +- include/block/block.h | 17 ++-- block.c

[PATCH v7 20/22] hw/block/nvme: Verify msix_vector_use() returned value

2020-06-09 Thread Klaus Jensen
From: Philippe Mathieu-Daudé msix_vector_use() returns -EINVAL on error. Assert it won't. Signed-off-by: Philippe Mathieu-Daudé Message-Id: <20200602155709.9776-1-phi...@redhat.com> Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff

[PATCH v7 21/22] hw/block/nvme: add msix_qsize parameter

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Decouple the requested maximum number of ioqpairs (param max_ioqpairs) from the number of MSI-X interrupt vectors by introducing a new msix_qsize parameter and initialize MSI-X with that. This allows emulating a device that has fewer vectors than I/O queue pairs and also

[PATCH v7 18/22] hw/block/nvme: do cmb/pmr init as part of pci init

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Maxim Levitsky Message-Id: <20200514044611.734782-20-...@irrelevant.dk> Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff

[PATCH v7 19/22] hw/block/nvme: factor out controller identify setup

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id: <20200514044611.734782-21-...@irrelevant.dk> Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 49

[PATCH v7 15/22] hw/block/nvme: factor out pci setup

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id: <20200514044611.734782-17-...@irrelevant.dk> Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 30 ++ 1 file

[PATCH v7 22/22] hw/block/nvme: verify msix_init_exclusive_bar() return value

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Pass an Error to msix_init_exclusive_bar() and check it. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index acc6dbc900e2..2a2e43f681f9 100644 ---

[PATCH v7 12/22] hw/block/nvme: factor out block backend setup

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id: <20200514044611.734782-14-...@irrelevant.dk> Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 13 ++--- 1 file changed, 10

[PATCH v7 14/22] hw/block/nvme: factor out namespace setup

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id: <20200514044611.734782-16-...@irrelevant.dk> Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 46

[PATCH v7 11/22] hw/block/nvme: factor out device state setup

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id: <20200514044611.734782-13-...@irrelevant.dk> Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 22 +- 1 file changed, 13

[PATCH v7 10/22] hw/block/nvme: factor out property/constraint checks

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id: <20200514044611.734782-12-...@irrelevant.dk> Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 48

Re: [PATCH v5 12/13] qcow2: QcowHeaderExtension print names for extension magics

2020-06-09 Thread Eric Blake
On 6/6/20 3:18 AM, Vladimir Sementsov-Ogievskiy wrote: Suggested-by: Andrey Shinkevich Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/031.out | 22 +++--- tests/qemu-iotests/036.out | 4 ++-- tests/qemu-iotests/061.out | 14

[PATCH v7 02/22] hw/block/nvme: rename trace events to pci_nvme

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Change the prefix of all nvme device related trace events to 'pci_nvme' to not clash with trace events from the nvme block driver. Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id:

[PATCH v7 16/22] hw/block/nvme: factor out cmb setup

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id: <20200514044611.734782-18-...@irrelevant.dk> Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 43 --- 1

[PATCH v7 17/22] hw/block/nvme: factor out pmr setup

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Maxim Levitsky Message-Id: <20200514044611.734782-19-...@irrelevant.dk> Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 95 ++--- 1 file changed, 51

[PATCH v7 09/22] hw/block/nvme: remove redundant cmbloc/cmbsz members

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id: <20200514044611.734782-11-...@irrelevant.dk> Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 7 ++- hw/block/nvme.h | 2 -- 2 files

[PATCH v7 07/22] hw/block/nvme: fix pin-based interrupt behavior

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen First, since the device only supports MSI-X or pin-based interrupt, if MSI-X is not enabled, it should not accept interrupt vectors different from 0 when creating completion queues. Secondly, the irq_status NvmeCtrl member is meant to be compared to the INTMS register, so it

[PATCH v7 13/22] hw/block/nvme: add namespace helpers

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Introduce some small helpers to make the next patches easier on the eye. Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id: <20200514044611.734782-15-...@irrelevant.dk> Signed-off-by: Kevin Wolf

[PATCH v7 00/22] nvme: small fixes, refactoring and cleanups

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Hi all, As per our discussion about how to amend the bug I introduced in "hw/block/nvme: allow use of any valid msix vector", this is a respin without that patch. Kevin, it applies cleanly on top of your block tree with all current hw/block/bnvme patches removed. Changes

[PATCH v7 05/22] hw/block/nvme: use constants in identify

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Maxim Levitsky Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Keith Busch Message-Id: <20200514044611.734782-6-...@irrelevant.dk> Signed-off-by: Kevin Wolf --- hw/block/nvme.c | 8 include/block/nvme.h | 8

[PATCH v7 08/22] hw/block/nvme: add max_ioqpairs device parameter

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen The num_queues device paramater has a slightly confusing meaning because it accounts for the admin queue pair which is not really optional. Secondly, it is really a maximum value of queues allowed. Add a new max_ioqpairs parameter that only accounts for I/O queue pairs, but

[PATCH v7 03/22] hw/block/nvme: remove superfluous breaks

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen These break statements was left over when commit 3036a626e9ef ("nvme: add Get/Set Feature Timestamp support") was merged. Signed-off-by: Klaus Jensen Reviewed-by: Maxim Levitsky Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Keith Busch Message-Id:

[PATCH v7 04/22] hw/block/nvme: move device parameters to separate struct

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Move device configuration parameters to separate struct to make it explicit what is configurable and what is set internally. Signed-off-by: Klaus Jensen Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Message-Id:

[PATCH v7 06/22] hw/block/nvme: refactor nvme_addr_read

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Pull the controller memory buffer check to its own function. The check will be used on its own in later patches. Signed-off-by: Klaus Jensen Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Maxim Levitsky Reviewed-by: Keith Busch Message-Id:

[PATCH v7 01/22] hw/block/nvme: fix pci doorbell size calculation

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen The size of the BAR is 0x1000 (main registers) + 8 bytes for each queue. Currently, the size of the BAR is calculated like so: n->reg_size = pow2ceil(0x1004 + 2 * (n->num_queues + 1) * 4); Since the 'num_queues' parameter already accounts for the admin queue, this should

[PATCH v2] rbd: Use RBD fast-diff for querying actual allocation

2020-06-09 Thread Yi Li
Since Ceph version Infernalis (9.2.0) the new fast-diff mechanism of RBD allows for querying actual rbd image usage. Prior to this version there was no easy and fast way to query how much allocation a RBD image had inside a Ceph cluster. To use the fast-diff feature it needs to be enabled per

Re: [PATCH 1/2] nbd/server: Avoid long error message assertions CVE-2020-10761

2020-06-09 Thread Eric Blake
On 6/8/20 1:26 PM, Eric Blake wrote: Ever since commit 36683283 (v2.8), the server code asserts that error strings sent to the client are well-formed per the protocol by not exceeding the maximum string length of 4096. At the time the server first started sending error messages, the assertion

Re: [PATCH 0/1] hw/block/nvme: fix assert on invalid irq vector

2020-06-09 Thread Klaus Jensen
On Jun 9 17:32, Kevin Wolf wrote: > Am 09.06.2020 um 16:18 hat Philippe Mathieu-Daudé geschrieben: > > On 6/9/20 4:14 PM, Kevin Wolf wrote: > > > Am 09.06.2020 um 13:46 hat Klaus Jensen geschrieben: > > >> On Jun 9 13:17, Philippe Mathieu-Daudé wrote: > > >>> On 6/9/20 11:45 AM, Klaus Jensen

[PATCH v2 1/2] nvme: indicate CMB support through controller capabilities register

2020-06-09 Thread Andrzej Jakowski
This patch sets CMBS bit in controller capabilities register when user configures NVMe driver with CMB support, so capabilites are correctly reported to guest OS. Signed-off-by: Andrzej Jakowski Reviewed-by: Klaus Jensen --- hw/block/nvme.c | 2 +- include/block/nvme.h | 6 +- 2 files

[PATCH v2] nvme: allow cmb and pmr emulation on same device

2020-06-09 Thread Andrzej Jakowski
This patch series does following: - Fixes problem where CMBS bit was not set in controller capabilities register, so support for CMB was not correctly advertised to guest. This is resend of patch that has been submitted and reviewed by Klaus [1] - Introduces BAR4 sharing between MSI-X

[PATCH v2 2/2] nvme: allow cmb and pmr to be enabled on same device

2020-06-09 Thread Andrzej Jakowski
So far it was not possible to have CMB and PMR emulated on the same device, because BAR2 was used exclusively either of PMR or CMB. This patch places CMB at BAR4 offset so it not conflicts with MSI-X vectors. Signed-off-by: Andrzej Jakowski --- hw/block/nvme.c | 120

Re: [PATCH v2 4/7] vhost: involve device backends in feature negotiation

2020-06-09 Thread Michael S. Tsirkin
On Tue, Jun 09, 2020 at 06:02:15PM +0100, Stefan Hajnoczi wrote: > Many vhost devices in QEMU currently do not involve the device backend > in feature negotiation. This seems fine at first glance when no > device-specific feature bits are defined (virtio-net has many but some > devices have none).

Re: [PATCH] qemu-img: Fix doc typo for 'bitmap' subcommand

2020-06-09 Thread Laurent Vivier
Le 29/05/2020 à 16:45, Eric Blake a écrit : > Prefer a consistent naming for the --merge argument. > > Fixes: 3b51ab4bf > Signed-off-by: Eric Blake > --- > > I'm happy for this to go in through my bitmaps queue or through the > trivial tree, whichever picks it up first. > >

[PATCH v2 6/7] vhost-scsi: add VIRTIO_F_VERSION_1 and VIRTIO_F_RING_PACKED

2020-06-09 Thread Stefan Hajnoczi
Let vhost-scsi and vhost-user-scsi device backends determine whether VIRTIO 1.0 and packed virtqueues are supported. It doesn't make sense to handle these feature bits in QEMU since the device backend needs to support them if we want to use them. Signed-off-by: Stefan Hajnoczi Reviewed-by:

[PATCH v2 5/7] vhost-user-blk: add VIRTIO_F_RING_PACKED feature bit

2020-06-09 Thread Stefan Hajnoczi
Vhost devices have a list of feature bits that the device backend is allowed to control. The VIRTIO_F_RING_PACKED feature is a feature that must be negotiated through all the way to the device backend. Add it so the device backend can declare whether or not it supports the packed ring layout.

[PATCH v2 1/7] tests/libqos: mask out VIRTIO_F_RING_PACKED for now

2020-06-09 Thread Stefan Hajnoczi
The libqos VIRTIO code does not implement the packed virtqueue layout yet. Mask out the feature bit for now because tests have a habit of enabling all device feature bits and we don't want packed virtqueues to be enabled. Later patches will enable VIRTIO_F_RING_PACKED so prepare libqos now.

[PATCH v2 4/7] vhost: involve device backends in feature negotiation

2020-06-09 Thread Stefan Hajnoczi
Many vhost devices in QEMU currently do not involve the device backend in feature negotiation. This seems fine at first glance when no device-specific feature bits are defined (virtio-net has many but some devices have none). Unfortunately this causes problems when QEMU's virtqueue implementation

[PATCH v2 7/7] virtio: enable VIRTIO_F_RING_PACKED for all devices

2020-06-09 Thread Stefan Hajnoczi
The packed virtqueue layout was introduced in VIRTIO 1.1. It is a single ring instead of a split avail/used ring design. There are CPU cache advantages to this layout and it is also suited better to hardware implementation. The vhost-net backend has already supported packed virtqueues for some

[PATCH v2 3/7] docs: document non-net VHOST_USER_GET_FEATURES behavior

2020-06-09 Thread Stefan Hajnoczi
QEMU enabled several feature bits for non-net devices without allowing the device backend to control them. This only works when the device backend implements support for those features. It won't work for new features like the packed virtqueue layout, where proper feature negotiation will be

[PATCH v2 2/7] libvhost-user: advertise vring features

2020-06-09 Thread Stefan Hajnoczi
libvhost-user implements several vring features without advertising them. There is no way for the vhost-user master to detect support for these features. Things more or less work today because QEMU assumes the vhost-user backend always implements certain feature bits like VIRTIO_RING_F_EVENT_IDX.

[PATCH v2 0/7] virtio: enable VIRTIO_F_RING_PACKED for all devices

2020-06-09 Thread Stefan Hajnoczi
v2: * Add libvhost-user VHOST_USER_GET_FEATURES patch to start reporting feature bits correctly (already reviewed by Marc-André Lureau) * Rephrase doc comments and drop vhost_get_default_features() [Jason] * Simplify hw/core/machine.c compat props by setting packed=off on the virtio-dev

Re: [PATCH v6 2/7] block/io: refactor coroutine wrappers

2020-06-09 Thread Eric Blake
On 6/9/20 9:13 AM, Vladimir Sementsov-Ogievskiy wrote: Most of our coroutine wrappers already follow this convention: We have 'coroutine_fn bdrv_co_()' as the core function, and a wrapper 'bdrv_()' which does parameters packing and call bdrv_run_co(). The only outsiders are the bdrv_prwv_co

Re: [PATCH] qcow2: Reduce write_zeroes size in handle_alloc_space()

2020-06-09 Thread Eric Blake
On 6/9/20 10:18 AM, Kevin Wolf wrote: -    ret = bdrv_co_pwrite_zeroes(s->data_file, m->alloc_offset, -    m->nb_clusters * s->cluster_size, +    ret = bdrv_co_pwrite_zeroes(s->data_file, start, len,  

Re: [PATCH v4 1/5] virtio-pci: add virtio_pci_optimal_num_queues() helper

2020-06-09 Thread Michael S. Tsirkin
On Thu, May 28, 2020 at 05:35:55PM +0200, Cornelia Huck wrote: > On Wed, 27 May 2020 11:29:21 +0100 > Stefan Hajnoczi wrote: > > > Multi-queue devices achieve the best performance when each vCPU has a > > dedicated queue. This ensures that virtqueue used notifications are > > handled on the same

Re: [PATCH v4 5/5] vhost-user-blk: default num_queues to -smp N

2020-06-09 Thread Michael S. Tsirkin
On Sat, May 30, 2020 at 10:42:05PM -0400, Raphael Norwitz wrote: > I'm happy with the code but as David pointed out with virtio-scsi, we > should probably add a comment about virtio_pci_optimal_num_queues() > capping the number of VQs here too. Stefan could you add this tweak and repost pls?

Re: [PATCH 0/1] hw/block/nvme: fix assert on invalid irq vector

2020-06-09 Thread Kevin Wolf
Am 09.06.2020 um 16:18 hat Philippe Mathieu-Daudé geschrieben: > On 6/9/20 4:14 PM, Kevin Wolf wrote: > > Am 09.06.2020 um 13:46 hat Klaus Jensen geschrieben: > >> On Jun 9 13:17, Philippe Mathieu-Daudé wrote: > >>> On 6/9/20 11:45 AM, Klaus Jensen wrote: > From: Klaus Jensen > >

Re: [PATCH] qcow2: Reduce write_zeroes size in handle_alloc_space()

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
09.06.2020 18:18, Kevin Wolf wrote: Am 09.06.2020 um 16:46 hat Eric Blake geschrieben: On 6/9/20 9:28 AM, Vladimir Sementsov-Ogievskiy wrote: 09.06.2020 17:08, Kevin Wolf wrote: Since commit c8bb23cbdbe, handle_alloc_space() is called for newly allocated clusters to efficiently initialise the

Re: [PATCH] qcow2: Reduce write_zeroes size in handle_alloc_space()

2020-06-09 Thread Kevin Wolf
Am 09.06.2020 um 16:46 hat Eric Blake geschrieben: > On 6/9/20 9:28 AM, Vladimir Sementsov-Ogievskiy wrote: > > 09.06.2020 17:08, Kevin Wolf wrote: > > > Since commit c8bb23cbdbe, handle_alloc_space() is called for newly > > > allocated clusters to efficiently initialise the COW areas with zeros

Re: [PATCH RFC v2 0/5] block: add block-dirty-bitmap-populate job

2020-06-09 Thread Peter Krempa
On Mon, May 18, 2020 at 16:52:45 +0200, Peter Krempa wrote: > On Wed, May 13, 2020 at 23:49:17 -0400, John Snow wrote: > > Hi, > > > > This is a new (very small) block job that writes a pattern into a > > bitmap. The only pattern implemented is the top allocation information. > > > > This can be

Re: [PATCH] qcow2: Reduce write_zeroes size in handle_alloc_space()

2020-06-09 Thread Eric Blake
On 6/9/20 9:28 AM, Vladimir Sementsov-Ogievskiy wrote: 09.06.2020 17:08, Kevin Wolf wrote: Since commit c8bb23cbdbe, handle_alloc_space() is called for newly allocated clusters to efficiently initialise the COW areas with zeros if necessary. It skips the whole operation if both start_cow nor

Re: [PATCH] qcow2: Reduce write_zeroes size in handle_alloc_space()

2020-06-09 Thread Eric Blake
On 6/9/20 9:08 AM, Kevin Wolf wrote: Since commit c8bb23cbdbe, handle_alloc_space() is called for newly allocated clusters to efficiently initialise the COW areas with zeros if necessary. It skips the whole operation if both start_cow nor end_cow s/nor/and/ are empty. However, it requests

Re: [PATCH v6 00/16] acpi: i386 tweaks

2020-06-09 Thread Michael S. Tsirkin
On Fri, May 15, 2020 at 05:04:05PM +0200, Gerd Hoffmann wrote: > First batch of microvm patches, some generic acpi stuff. > Split the acpi-build.c monster, specifically split the > pc and q35 and pci bits into a separate file which we > can skip building at some point in the future. Applied

Re: [PATCH v6 08/16] acpi: move aml builder code for floppy device

2020-06-09 Thread Michael S. Tsirkin
On Fri, May 15, 2020 at 05:04:13PM +0200, Gerd Hoffmann wrote: > Signed-off-by: Gerd Hoffmann > Reviewed-by: Igor Mammedov > --- > hw/block/fdc.c | 83 > hw/i386/acpi-build.c | 83 > stubs/cmos.c

Re: [PATCH] qcow2: Reduce write_zeroes size in handle_alloc_space()

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
09.06.2020 17:08, Kevin Wolf wrote: Since commit c8bb23cbdbe, handle_alloc_space() is called for newly allocated clusters to efficiently initialise the COW areas with zeros if necessary. It skips the whole operation if both start_cow nor end_cow are empty. However, it requests zeroing the whole

Re: [PATCH 0/1] hw/block/nvme: fix assert on invalid irq vector

2020-06-09 Thread Philippe Mathieu-Daudé
On 6/9/20 4:14 PM, Kevin Wolf wrote: > Am 09.06.2020 um 13:46 hat Klaus Jensen geschrieben: >> On Jun 9 13:17, Philippe Mathieu-Daudé wrote: >>> On 6/9/20 11:45 AM, Klaus Jensen wrote: From: Klaus Jensen I goofed up with commit c09794fe40e3 ("hw/block/nvme: allow use of any

Re: [PATCH 0/1] hw/block/nvme: fix assert on invalid irq vector

2020-06-09 Thread Kevin Wolf
Am 09.06.2020 um 13:46 hat Klaus Jensen geschrieben: > On Jun 9 13:17, Philippe Mathieu-Daudé wrote: > > On 6/9/20 11:45 AM, Klaus Jensen wrote: > > > From: Klaus Jensen > > > > > > I goofed up with commit c09794fe40e3 ("hw/block/nvme: allow use of any > > > valid msix vector"). > > > >

[PATCH v6 6/7] block: drop bdrv_prwv

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
Now that we are not maintaining boilerplate code for coroutine wrappers, there is no more sense in keeping the extra indirection layer of bdrv_prwv(). Let's drop it and instead generate pure bdrv_preadv() and bdrv_pwritev(). Currently, bdrv_pwritev() and bdrv_preadv() are returning bytes on

[PATCH v6 4/7] scripts: add coroutine-wrapper.py

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
We have a very frequent pattern of creating coroutine from function with several arguments: - create structure to pack parameters - create _entry function to call original function taking parameters from struct - do different magic to handle completion: set ret to NOT_DONE or

[PATCH v6 5/7] block: generate coroutine-wrapper code

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
Use code generation implemented in previous commit to generated coroutine wrappers in block.c and block/io.c Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/coroutines.h| 7 +- include/block/block.h | 17 ++-- block.c | 73 --- block/io.c| 212

[PATCH v6 7/7] block/io: refactor save/load vmstate

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
Like for read/write in a previous commit, drop extra indirection layer, generate directly bdrv_readv_vmstate() and bdrv_writev_vmstate(). Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake --- block/coroutines.h| 10 +++ include/block/block.h | 6 ++-- block/io.c

[PATCH v6 2/7] block/io: refactor coroutine wrappers

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
Most of our coroutine wrappers already follow this convention: We have 'coroutine_fn bdrv_co_()' as the core function, and a wrapper 'bdrv_()' which does parameters packing and call bdrv_run_co(). The only outsiders are the bdrv_prwv_co and bdrv_common_block_status_above wrappers. Let's refactor

[PATCH v6 0/7] coroutines: generate wrapper code

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
Hi all! The aim of the series is to reduce code-duplication and writing parameters structure-packing by hand around coroutine function wrappers. Benefits: - no code duplication - less indirection v6: rebase on master (mostly because of introducing bdrv_run_co()) 01: grammar/wording in commit

[PATCH v6 1/7] block: return error-code from bdrv_invalidate_cache

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
This is the only coroutine wrapper from block.c and block/io.c which doesn't return a value, so let's convert it to the common behavior, to simplify moving to generated coroutine wrappers in a further commit. Also, bdrv_invalidate_cache is a void function, returning error only through **errp

[PATCH v6 3/7] block: declare some coroutine functions in block/coroutines.h

2020-06-09 Thread Vladimir Sementsov-Ogievskiy
We are going to keep coroutine-wrappers code (structure-packing parameters, BDRV_POLL wrapper functions) in separate auto-generated files. So, we'll need a header with declaration of original _co_ functions, for those which are static now. As well, we'll need declarations for wrapper functions. Do

[PATCH] qcow2: Reduce write_zeroes size in handle_alloc_space()

2020-06-09 Thread Kevin Wolf
Since commit c8bb23cbdbe, handle_alloc_space() is called for newly allocated clusters to efficiently initialise the COW areas with zeros if necessary. It skips the whole operation if both start_cow nor end_cow are empty. However, it requests zeroing the whole request size (possibly multiple

Re: [PATCH] rbd: Use RBD fast-diff for querying actual allocation

2020-06-09 Thread 李义
On 6/9/20, Jason Dillaman wrote: > On Tue, Jun 9, 2020 at 3:31 AM Yi Li wrote: >> >> Since Ceph version Infernalis (9.2.0) the new fast-diff mechanism >> of RBD allows for querying actual rbd image usage. >> >> Prior to this version there was no easy and fast way to query how >> much allocation

Re: [PATCH] iotests: Fix 291 across more file systems

2020-06-09 Thread Kevin Wolf
Am 08.06.2020 um 21:56 hat Eric Blake geschrieben: > Depending on the granularity of holes and amount of metadata consumed > by a file, the 'disk size:' number of 'qemu-img info' is not reliable. > Adjust our test to use a different set of filters to avoid spurious > failures. > > Reported-by:

Re: [PATCH] rbd: Use RBD fast-diff for querying actual allocation

2020-06-09 Thread 李义
CC qemu-block and rbd maintainers On 6/9/20, Yi Li wrote: > Since Ceph version Infernalis (9.2.0) the new fast-diff mechanism > of RBD allows for querying actual rbd image usage. > > Prior to this version there was no easy and fast way to query how > much allocation a RBD image had inside a Ceph

Re: [PATCH 0/1] hw/block/nvme: fix assert on invalid irq vector

2020-06-09 Thread Klaus Jensen
On Jun 9 13:17, Philippe Mathieu-Daudé wrote: > On 6/9/20 11:45 AM, Klaus Jensen wrote: > > From: Klaus Jensen > > > > I goofed up with commit c09794fe40e3 ("hw/block/nvme: allow use of any > > valid msix vector"). > > Kevin, since your queue isn't merged, can you directly squash the fix? The

Re: [PATCH 0/1] hw/block/nvme: fix assert on invalid irq vector

2020-06-09 Thread Philippe Mathieu-Daudé
On 6/9/20 11:45 AM, Klaus Jensen wrote: > From: Klaus Jensen > > I goofed up with commit c09794fe40e3 ("hw/block/nvme: allow use of any > valid msix vector"). Kevin, since your queue isn't merged, can you directly squash the fix? > > This fixes the goof by adding a new msix_qsize parameter.

Re: [PATCH v1 2/2] nvme: allow cmb and pmr to be enabled on same device

2020-06-09 Thread Klaus Jensen
On Jun 8 12:44, Andrzej Jakowski wrote: > On 6/8/20 1:08 AM, Klaus Jensen wrote: > > On Jun 5 11:10, Andrzej Jakowski wrote: > >> So far it was not possible to have CMB and PMR emulated on the same > >> device, because BAR2 was used exclusively either of PMR or CMB. This > >> patch places CMB at

[PATCH 1/2] hw/block/nvme: add msix_qsize parameter

2020-06-09 Thread Klaus Jensen
From: Klaus Jensen Commit 4eb9f5217a16 ("hw/block/nvme: allow use of any valid msix vector") erronously allows any valid MSI-X vector to be used when creating completion queues. This is bad because MSI-X is initialized with max_ioqpairs + 1 vectors, which defaults to 64 + 1. And since commit

  1   2   >