[PATCH v6 09/10] block: Add support to warn on backing file change without format

2020-07-06 Thread Eric Blake
For now, this is a mechanical addition; all callers pass false. But the next patch will use it to improve 'qemu-img rebase -u' when selecting a backing file with no format. Signed-off-by: Eric Blake Reviewed-by: Peter Krempa Reviewed-by: Ján Tomko --- include/block/block.h | 4 ++-- block.c

[PATCH v6 10/10] qemu-img: Deprecate use of -b without -F

2020-07-06 Thread Eric Blake
Creating an image that requires format probing of the backing image is potentially unsafe (we've had several CVEs over the years based on probes leaking information to the guest on a subsequent boot, although these days tools like libvirt are aware of the issue enough to prevent the worst

[PATCH v6 06/10] block: Error if backing file fails during creation without -u

2020-07-06 Thread Eric Blake
Back in commit 6e6e55f5 (Jul 2017, v2.10), we tweaked the code to warn if the backing file could not be opened but the user gave a size, unless the user also passes the -u option to bypass the open of the backing file. As one common reason for failure to open the backing file is when there is

[PATCH v6 01/10] qemu-img: Flush stdout before before potential stderr messages

2020-07-06 Thread Eric Blake
During 'qemu-img create ... 2>&1', if --quiet is not in force, we can end up with buffered I/O in stdout that was produced before failure, but which appears in output after failure. This is confusing; the fix is to flush stdout prior to attempting anything that might produce an error message.

[PATCH v6 02/10] block: Finish deprecation of 'qemu-img convert -n -o'

2020-07-06 Thread Eric Blake
It's been two releases since we started warning; time to make the combination an error as promised. There was no iotest coverage, so add some. While touching the documentation, tweak another section heading for consistent style. Signed-off-by: Eric Blake --- docs/system/deprecated.rst | 18

[PATCH v6 04/10] vmdk: Add trivial backing_fmt support

2020-07-06 Thread Eric Blake
vmdk already requires that if backing_file is present, that it be another vmdk image (see vmdk_co_do_create). Meanwhile, we want to move towards always being explicit about the backing format for other drivers where it matters. So for convenience, make qemu-img create -F vmdk work, while

[PATCH v6 07/10] qcow2: Deprecate use of qemu-img amend to change backing file

2020-07-06 Thread Eric Blake
The use of 'qemu-img amend' to change qcow2 backing files is not tested very well. In particular, our implementation has a bug where if a new backing file is provided without a format, then the prior format is blindly reused, even if this results in data corruption, but this is not caught by

[PATCH v6 05/10] qcow: Tolerate backing_fmt=

2020-07-06 Thread Eric Blake
qcow has no space in the metadata to store a backing format, and there are existing qcow images backed both by raw or by other formats (usually qcow) images, reliant on probing to tell the difference. On the bright side, because we probe every time, raw files are marked as probed and we thus

[PATCH v6 03/10] sheepdog: Add trivial backing_fmt support

2020-07-06 Thread Eric Blake
Sheepdog already requires that if backing_file is present, that it be another sheepdog image (see sd_co_create). Meanwhile, we want to move towards always being explicit about the backing format for other drivers where it matters. So for convenience, make qemu-img create -F sheepdog work, while

[PATCH v6 00/10] Tighten qemu-img rules on missing backing format

2020-07-06 Thread Eric Blake
v5 was here: https://lists.gnu.org/archive/html/qemu-devel/2020-04/msg00679.html In v6: - add a few more patches - change qcow semantics based on several iterations of mailing list debates on what behavior is easiest to support - add iotesting that a probed raw file cannot be committed into -

Re: [PATCH v3 29/44] qom: Make functions taking Error ** return bool, not 0/-1

2020-07-06 Thread Markus Armbruster
Vladimir Sementsov-Ogievskiy writes: > On 06.07.2020 11:09, Markus Armbruster wrote: >> Just for consistency. Also fix the example in object_set_props()'s >> documentation. >> >> Signed-off-by: Markus Armbruster >> Reviewed-by: Eric Blake > > Reviewed-by: Vladimir Sementsov-Ogievskiy > >

Re: [PATCH v3 06/44] qemu-option: Check return value instead of @err where convenient

2020-07-06 Thread Markus Armbruster
Greg Kurz writes: > On Mon, 6 Jul 2020 10:09:12 +0200 > Markus Armbruster wrote: > >> Convert uses like >> >> opts = qemu_opts_create(..., ); >> if (err) { >> ... >> } >> >> to >> >> opts = qemu_opts_create(..., ); > > The patch doesn't strictly do that since it also

[PATCH] MAINTAINERS: update nvme entry

2020-07-06 Thread Keith Busch
The nvme emulated device development pace has increased recently. Klaus has offered to co-maintain, and since we have many new contributions coming through, we're adding a repository to accumulate and test new features. Cc: Klaus Jensen Signed-off-by: Keith Busch --- MAINTAINERS | 2 ++ 1

[PATCH 0/3] qemu: implementation of transient option for qcow2 file

2020-07-06 Thread Masayoshi Mizuma
Hello, This patchset tries to implement transient option for qcow2 file. It gets user available to set to the domain xml file like as: Any changes which the Guest does to the disk is dropped when the Guest is shutdowned. Masayoshi Mizuma (3): qemu:

[PATCH 2/3] testutilsqemu: Assign qemu-img path to driver->qemuImgBinary

2020-07-06 Thread Masayoshi Mizuma
From: Masayoshi Mizuma Assign qemu-img command file path to driver->qemuImgBinary so that the unit tests can use qemu-img command. Signed-off-by: Masayoshi Mizuma --- tests/testutilsqemu.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/testutilsqemu.c

[PATCH 1/3] qemu: implementation of transient option for qcow2 file

2020-07-06 Thread Masayoshi Mizuma
From: Masayoshi Mizuma Here is the implementation of transient option for qcow2 file. This gets available directive in domain xml file like as: The internal procedure is as follows. When the qemu command line options are built, a new qcow2 image is

[PATCH 3/3] qemublocktest: add test of transient option for qcow2 disk

2020-07-06 Thread Masayoshi Mizuma
From: Masayoshi Mizuma Add a unit test for transient option for qcow2 file. Signed-off-by: Masayoshi Mizuma --- tests/qemublocktest.c | 10 ++ .../xml2json/qcow2-transient-srconly.json | 9 +

Re: [PATCH v3 00/11] hw/sd/sdcard: Fix CVE-2020-13253 & cleanups

2020-07-06 Thread Philippe Mathieu-Daudé
On 7/6/20 6:31 PM, Alistair Francis wrote: > On Mon, Jun 8, 2020 at 10:48 AM Philippe Mathieu-Daudé > wrote: >> >> Hi Alistair, >> >> On 6/5/20 12:22 PM, Philippe Mathieu-Daudé wrote: >>> Patches 2 & 3 fix CVE-2020-13253. >>> The rest are (accumulated) cleanups. >>> >>> Supersedes:

Re: [PATCH v3 30/44] qdev: Make functions taking Error ** return bool, not void

2020-07-06 Thread Vladimir Sementsov-Ogievskiy
On 06.07.2020 11:09, Markus Armbruster wrote: See recent commit "error: Document Error API usage rules" for rationale. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy

Re: [PATCH v7 17/17] hw/sd/sdcard: Simplify realize() a bit

2020-07-06 Thread Alistair Francis
On Tue, Jun 30, 2020 at 6:46 AM Philippe Mathieu-Daudé wrote: > > We don't need to check if sd->blk is set twice. > > Reviewed-by: Peter Maydell > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Alistair Francis Alistair > --- > hw/sd/sd.c | 10 +- > 1 file changed, 5

Re: [PATCH v3 29/44] qom: Make functions taking Error ** return bool, not 0/-1

2020-07-06 Thread Vladimir Sementsov-Ogievskiy
On 06.07.2020 11:09, Markus Armbruster wrote: Just for consistency. Also fix the example in object_set_props()'s documentation. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Note, that object_set_props is unused and may be dropped.

Re: [PATCH v7 16/17] hw/sd/sdcard: Display offset in read/write_data() trace events

2020-07-06 Thread Alistair Francis
On Tue, Jun 30, 2020 at 6:44 AM Philippe Mathieu-Daudé wrote: > > Having 'base address' and 'relative offset' displayed > separately is more helpful than the absolute address. > > Reviewed-by: Peter Maydell > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Alistair Francis Alistair > ---

Re: [PATCH v3 00/11] hw/sd/sdcard: Fix CVE-2020-13253 & cleanups

2020-07-06 Thread Alistair Francis
On Mon, Jun 8, 2020 at 10:48 AM Philippe Mathieu-Daudé wrote: > > Hi Alistair, > > On 6/5/20 12:22 PM, Philippe Mathieu-Daudé wrote: > > Patches 2 & 3 fix CVE-2020-13253. > > The rest are (accumulated) cleanups. > > > > Supersedes: <20200604182502.24228-1-f4...@amsat.org> > > > > Philippe

Re: [PATCH v7 14/17] hw/sd/sdcard: Make iolen unsigned

2020-07-06 Thread Alistair Francis
On Tue, Jun 30, 2020 at 6:51 AM Philippe Mathieu-Daudé wrote: > > From: Philippe Mathieu-Daudé > > I/O request length can not be negative. > > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Alistair Francis Alistair > --- > v4: Use uint32_t (pm215) > --- > hw/sd/sd.c | 2 +- >

Re: [PATCH v7 12/17] hw/sd/sdcard: Simplify cmd_valid_while_locked()

2020-07-06 Thread Alistair Francis
On Tue, Jun 30, 2020 at 6:44 AM Philippe Mathieu-Daudé wrote: > > cmd_valid_while_locked() only needs to read SDRequest->cmd, > pass it directly and make it const. > > Reviewed-by: Peter Maydell > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Alistair Francis Alistair > --- >

Re: [PATCH v3 28/44] qom: Use returned bool to check for failure, manual part

2020-07-06 Thread Vladimir Sementsov-Ogievskiy
On 06.07.2020 11:09, Markus Armbruster wrote: The previous commit used Coccinelle to convert from checking the Error object to checking the return value. Convert a few more manually. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy

Re: [PATCH v7 11/17] hw/sd/sdcard: Update the SDState documentation

2020-07-06 Thread Alistair Francis
On Tue, Jun 30, 2020 at 6:45 AM Philippe Mathieu-Daudé wrote: > > Add more descriptive comments to keep a clear separation > between static property vs runtime changeable. > > Suggested-by: Peter Maydell > Reviewed-by: Peter Maydell > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by:

Re: [PATCH v7 06/17] hw/sd/sdcard: Restrict Class 6 commands to SCSD cards

2020-07-06 Thread Alistair Francis
On Tue, Jun 30, 2020 at 6:44 AM Philippe Mathieu-Daudé wrote: > > Only SCSD cards support Class 6 (Block Oriented Write Protection) > commands. > > "SD Specifications Part 1 Physical Layer Simplified Spec. v3.01" > > 4.3.14 Command Functional Difference in Card Capacity Types > > * Write

Re: [PATCH v7 05/17] hw/sd/sdcard: Do not switch to ReceivingData if address is invalid

2020-07-06 Thread Alistair Francis
On Tue, Jun 30, 2020 at 6:42 AM Philippe Mathieu-Daudé wrote: > > Only move the state machine to ReceivingData if there is no > pending error. This avoids later OOB access while processing > commands queued. > > "SD Specifications Part 1 Physical Layer Simplified Spec. v3.01" > > 4.3.3 Data

Re: [PATCH v7 02/17] hw/sd/sdcard: Update coding style to make checkpatch.pl happy

2020-07-06 Thread Alistair Francis
On Tue, Jun 30, 2020 at 6:40 AM Philippe Mathieu-Daudé wrote: > > From: Philippe Mathieu-Daudé > > To make the next commit easier to review, clean this code first. > > Reviewed-by: Peter Maydell > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Alistair Francis Alistair > --- >

Re: [PATCH] trivial: Remove trailing whitespaces

2020-07-06 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200706162300.1084753-1-dinec...@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [PATCH] trivial: Remove trailing whitespaces Type: series Message-id:

[PATCH] trivial: Remove trailing whitespaces

2020-07-06 Thread Christophe de Dinechin
There are a number of unnecessary trailing whitespaces that have accumulated over time in the source code. They cause stray changes in patches if you use tools that automatically remove them. Tested by doing a `git diff -w` after the change. This could probably be turned into a pre-commit hook.

Re: [PATCH 0/6] migration: bring savevm/loadvm/delvm over to QMP

2020-07-06 Thread Kevin Wolf
Am 06.07.2020 um 18:03 hat Daniel P. Berrangé geschrieben: > On Mon, Jul 06, 2020 at 05:50:11PM +0200, Kevin Wolf wrote: > > Am 06.07.2020 um 17:29 hat Daniel P. Berrangé geschrieben: > > > On Mon, Jul 06, 2020 at 05:27:01PM +0200, Kevin Wolf wrote: > > > > Am 03.07.2020 um 19:29 hat Denis V.

Re: [PATCH 0/6] migration: bring savevm/loadvm/delvm over to QMP

2020-07-06 Thread Daniel P . Berrangé
On Mon, Jul 06, 2020 at 07:10:16PM +0300, Denis V. Lunev wrote: > On 7/6/20 7:03 PM, Daniel P. Berrangé wrote: > > On Mon, Jul 06, 2020 at 05:50:11PM +0200, Kevin Wolf wrote: > >> Am 06.07.2020 um 17:29 hat Daniel P. Berrangé geschrieben: > >>> On Mon, Jul 06, 2020 at 05:27:01PM +0200, Kevin Wolf

Re: [PATCH 2/6] migration: introduce savevm, loadvm, delvm QMP commands

2020-07-06 Thread Kevin Wolf
Am 03.07.2020 um 18:02 hat Daniel P. Berrangé geschrieben: > On Fri, Jul 03, 2020 at 04:49:33PM +0100, Dr. David Alan Gilbert wrote: > > * Daniel P. Berrangé (berra...@redhat.com) wrote: > > > On Thu, Jul 02, 2020 at 01:12:52PM -0500, Eric Blake wrote: > > > > On 7/2/20 12:57 PM, Daniel P.

Re: [PATCH 0/6] migration: bring savevm/loadvm/delvm over to QMP

2020-07-06 Thread Denis V. Lunev
On 7/6/20 7:03 PM, Daniel P. Berrangé wrote: > On Mon, Jul 06, 2020 at 05:50:11PM +0200, Kevin Wolf wrote: >> Am 06.07.2020 um 17:29 hat Daniel P. Berrangé geschrieben: >>> On Mon, Jul 06, 2020 at 05:27:01PM +0200, Kevin Wolf wrote: Am 03.07.2020 um 19:29 hat Denis V. Lunev geschrieben: >

Re: [PATCH v3 27/44] qom: Use returned bool to check for failure, Coccinelle part

2020-07-06 Thread Vladimir Sementsov-Ogievskiy
On 06.07.2020 11:09, Markus Armbruster wrote: The previous commit enables conversion of foo(..., ); if (err) { ... } to if (!foo(..., errp)) { ... } for QOM functions that now return true / false on success / error. Coccinelle script: @@

Re: [PATCH 0/6] migration: bring savevm/loadvm/delvm over to QMP

2020-07-06 Thread Denis V. Lunev
On 7/6/20 5:28 PM, Daniel P. Berrangé wrote: > On Fri, Jul 03, 2020 at 08:29:08PM +0300, Denis V. Lunev wrote: >> On 7/3/20 8:22 PM, Daniel P. Berrangé wrote: >>> On Fri, Jul 03, 2020 at 08:15:44PM +0300, Denis V. Lunev wrote: On 7/2/20 8:57 PM, Daniel P. Berrangé wrote: > When QMP was

Re: [PATCH 0/6] migration: bring savevm/loadvm/delvm over to QMP

2020-07-06 Thread Daniel P . Berrangé
On Mon, Jul 06, 2020 at 05:50:11PM +0200, Kevin Wolf wrote: > Am 06.07.2020 um 17:29 hat Daniel P. Berrangé geschrieben: > > On Mon, Jul 06, 2020 at 05:27:01PM +0200, Kevin Wolf wrote: > > > Am 03.07.2020 um 19:29 hat Denis V. Lunev geschrieben: > > > > On 7/3/20 8:22 PM, Daniel P. Berrangé

Re: [PATCH v3 06/44] qemu-option: Check return value instead of @err where convenient

2020-07-06 Thread Greg Kurz
On Mon, 6 Jul 2020 10:09:12 +0200 Markus Armbruster wrote: > Convert uses like > > opts = qemu_opts_create(..., ); > if (err) { > ... > } > > to > > opts = qemu_opts_create(..., ); The patch doesn't strictly do that since it also converts to errp. This is okay

Re: [PATCH 5/6] migration: support excluding block devs in QMP snapshot commands

2020-07-06 Thread Kevin Wolf
Am 02.07.2020 um 19:57 hat Daniel P. Berrangé geschrieben: > This wires up support for a new "exclude" parameter to the QMP commands > for snapshots (savevm, loadvm, delvm). This parameter accepts a list of > block driver state node names. > > One use case for this would be a VM using OVMF

Re: [PATCH 0/6] migration: bring savevm/loadvm/delvm over to QMP

2020-07-06 Thread Kevin Wolf
Am 06.07.2020 um 17:29 hat Daniel P. Berrangé geschrieben: > On Mon, Jul 06, 2020 at 05:27:01PM +0200, Kevin Wolf wrote: > > Am 03.07.2020 um 19:29 hat Denis V. Lunev geschrieben: > > > On 7/3/20 8:22 PM, Daniel P. Berrangé wrote: > > > > On Fri, Jul 03, 2020 at 08:15:44PM +0300, Denis V.

Re: [PATCH v3 26/44] qom: Make functions taking Error ** return bool, not void

2020-07-06 Thread Vladimir Sementsov-Ogievskiy
06.07.2020 11:09, Markus Armbruster wrote: See recent commit "error: Document Error API usage rules" for rationale. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy -- Best regards, Vladimir

Re: [PATCH 0/6] migration: bring savevm/loadvm/delvm over to QMP

2020-07-06 Thread Daniel P . Berrangé
On Mon, Jul 06, 2020 at 05:27:01PM +0200, Kevin Wolf wrote: > Am 03.07.2020 um 19:29 hat Denis V. Lunev geschrieben: > > On 7/3/20 8:22 PM, Daniel P. Berrangé wrote: > > > On Fri, Jul 03, 2020 at 08:15:44PM +0300, Denis V. Lunev wrote: > > >> On 7/2/20 8:57 PM, Daniel P. Berrangé wrote: > >

Re: [PATCH 0/6] migration: bring savevm/loadvm/delvm over to QMP

2020-07-06 Thread Kevin Wolf
Am 03.07.2020 um 19:29 hat Denis V. Lunev geschrieben: > On 7/3/20 8:22 PM, Daniel P. Berrangé wrote: > > On Fri, Jul 03, 2020 at 08:15:44PM +0300, Denis V. Lunev wrote: > >> On 7/2/20 8:57 PM, Daniel P. Berrangé wrote: > >>> When QMP was first introduced some 10+ years ago now, the snapshot >

Re: [PATCH v3 25/44] qom: Put name parameter before value / visitor parameter

2020-07-06 Thread Vladimir Sementsov-Ogievskiy
06.07.2020 11:09, Markus Armbruster wrote: The object_property_set_FOO() setters take property name and value in an unusual order: void object_property_set_FOO(Object *obj, FOO_TYPE value, const char *name, Error **errp) Having to pass value before name

Re: [PATCH v3 04/44] macio: Tidy up error handling in macio_newworld_realize()

2020-07-06 Thread Greg Kurz
On Mon, 6 Jul 2020 10:09:10 +0200 Markus Armbruster wrote: > macio_newworld_realize() effectively ignores ns->gpio realization > errors, leaking the Error object. Fortunately, macio_gpio_realize() > can't actually fail. Tidy up. > > Cc: Mark Cave-Ayland > Cc: David Gibson > Signed-off-by:

Re: [PATCH v3 03/44] qdev: Use returned bool to check for qdev_realize() etc. failure

2020-07-06 Thread Greg Kurz
On Mon, 06 Jul 2020 13:35:19 +0200 Markus Armbruster wrote: > Greg Kurz writes: > > > On Mon, 6 Jul 2020 10:09:09 +0200 > > Markus Armbruster wrote: > > > >> Convert > >> > >> foo(..., ); > >> if (err) { > >> ... > >> } > >> > >> to > >> > >> if (!foo(..., )) { >

Re: [PATCH v3 01/44] error: Improve examples in error.h's big comment

2020-07-06 Thread Markus Armbruster
Markus Armbruster writes: > Show errp instead of where is actually unusual. Add a > missing declaration. Add a second error pileup example. > > Signed-off-by: Markus Armbruster > Reviewed-by: Eric Blake > Reviewed-by: Vladimir Sementsov-Ogievskiy > Reviewed-by: Greg Kurz > --- >

Re: [PATCH 0/6] migration: bring savevm/loadvm/delvm over to QMP

2020-07-06 Thread Daniel P . Berrangé
On Fri, Jul 03, 2020 at 08:29:08PM +0300, Denis V. Lunev wrote: > On 7/3/20 8:22 PM, Daniel P. Berrangé wrote: > > On Fri, Jul 03, 2020 at 08:15:44PM +0300, Denis V. Lunev wrote: > >> On 7/2/20 8:57 PM, Daniel P. Berrangé wrote: > >>> When QMP was first introduced some 10+ years ago now, the

Re: [PATCH v4 1/5] virtio-pci: add virtio_pci_optimal_num_queues() helper

2020-07-06 Thread Cornelia Huck
On Mon, 6 Jul 2020 14:25:20 +0100 Stefan Hajnoczi wrote: > On Thu, May 28, 2020 at 05:35:55PM +0200, Cornelia Huck wrote: > > On Wed, 27 May 2020 11:29:21 +0100 > > Stefan Hajnoczi wrote: > > > > > Multi-queue devices achieve the best performance when each vCPU has a > > > dedicated queue.

Re: [PATCH v9 34/34] iotests: Add tests for qcow2 images with extended L2 entries

2020-07-06 Thread Eric Blake
On 7/3/20 8:06 AM, Alberto Garcia wrote: +printf -v expected_bitmap "%llu" $expected_bitmap # Convert to unsigned Does the length modifier “ll” actually do anything? + +printf "L2 entry #%d: 0x%016lx %016lx\n" "$entry_no" "$entry" "$bitmap" Or the “l” here? Actually they don't

[PATCH v5 1/5] virtio-pci: add virtio_pci_optimal_num_queues() helper

2020-07-06 Thread Stefan Hajnoczi
Multi-queue devices achieve the best performance when each vCPU has a dedicated queue. This ensures that virtqueue used notifications are handled on the same vCPU that submitted virtqueue buffers. When another vCPU handles the the notification an IPI will be necessary to wake the submission vCPU

[PATCH v5 4/5] virtio-blk: default num_queues to -smp N

2020-07-06 Thread Stefan Hajnoczi
Automatically size the number of virtio-blk-pci request virtqueues to match the number of vCPUs. Other transports continue to default to 1 request virtqueue. A 1:1 virtqueue:vCPU mapping ensures that completion interrupts are handled on the same vCPU that submitted the request. No IPI is

[PATCH v5 2/5] virtio-scsi: introduce a constant for fixed virtqueues

2020-07-06 Thread Stefan Hajnoczi
The event and control virtqueues are always present, regardless of the multi-queue configuration. Define a constant so that virtqueue number calculations are easier to read. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck Reviewed-by: Pankaj Gupta Reviewed-by: Philippe Mathieu-Daudé

[PATCH v5 5/5] vhost-user-blk: default num_queues to -smp N

2020-07-06 Thread Stefan Hajnoczi
Automatically size the number of request virtqueues to match the number of vCPUs. This ensures that completion interrupts are handled on the same vCPU that submitted the request. No IPI is necessary to complete an I/O request and performance is improved. The maximum number of MSI-X vectors and

[PATCH v5 3/5] virtio-scsi: default num_queues to -smp N

2020-07-06 Thread Stefan Hajnoczi
Automatically size the number of virtio-scsi-pci, vhost-scsi-pci, and vhost-user-scsi-pci request virtqueues to match the number of vCPUs. Other transports continue to default to 1 request virtqueue. A 1:1 virtqueue:vCPU mapping ensures that completion interrupts are handled on the same vCPU that

[PATCH v5 0/5] virtio-pci: enable blk and scsi multi-queue by default

2020-07-06 Thread Stefan Hajnoczi
v4: * Sorry for the long delay. I considered replacing this series with a simpler approach. Real hardware ships with a fixed number of queues (e.g. 128). The equivalent can be done in QEMU too. That way we don't need to magically size num_queues. In the end I decided against this

Re: [PATCH v3 00/44] Less clumsy error checking

2020-07-06 Thread Eric Blake
On 7/6/20 3:12 AM, Markus Armbruster wrote: diff between v2 rebased and v3, with hunks that change only whitespace dropped: Thanks, that's useful. Overall, the series looks ready to go from my perspective. It looks like the changes from v2 were minimal enough that I don't have any R-b to

[RFC PATCH 2/2] block/block-backend: Let blk_attach_dev() provide helpful error

2020-07-06 Thread Philippe Mathieu-Daudé
Let blk_attach_dev() take an Error* object to return helpful information. Adapt the callers. $ qemu-system-arm -M n800 qemu-system-arm: sd_init failed: cannot attach blk 'sd0' to device 'sd-card' because it is already attached by device 'omap2-mmc' Drive 'sd0' is already in use because it

[PATCH 1/2] block/block-backend: Trace blk_attach_dev()

2020-07-06 Thread Philippe Mathieu-Daudé
Add a trace event to follow devices attaching block drives: $ qemu-system-arm -M n800 -trace blk_\* 9513@1594040428.738162:blk_attach_dev attaching blk 'sd0' to device 'omap2-mmc' 9513@1594040428.738189:blk_attach_dev attaching blk 'sd0' to device 'sd-card' qemu-system-arm: sd_init

[PATCH 0/2] block/block-backend: Let blk_attach_dev() provide helpful error

2020-07-06 Thread Philippe Mathieu-Daudé
A pair of patches which helps me debug an issue with block drive already attached. Suggestions to correctly/better use the Error API welcome, in particular in qdev-properties-system::set_drive_helper(). Philippe Mathieu-Daudé (2): block/block-backend: Trace blk_attach_dev()

Re: [PATCH v4 1/5] virtio-pci: add virtio_pci_optimal_num_queues() helper

2020-07-06 Thread Stefan Hajnoczi
On Thu, May 28, 2020 at 05:35:55PM +0200, Cornelia Huck wrote: > On Wed, 27 May 2020 11:29:21 +0100 > Stefan Hajnoczi wrote: > > > Multi-queue devices achieve the best performance when each vCPU has a > > dedicated queue. This ensures that virtqueue used notifications are > > handled on the same

Re: [PATCH v3 16/16] block/nvme: Use per-queuepair IRQ notifier and AIO context

2020-07-06 Thread Philippe Mathieu-Daudé
On 7/6/20 2:04 PM, Stefan Hajnoczi wrote: > On Sat, Jul 04, 2020 at 11:30:51PM +0200, Philippe Mathieu-Daudé wrote: >> @@ -683,6 +676,17 @@ static bool nvme_add_io_queue(BlockDriverState *bs, >> Error **errp) >> s->queues = g_renew(NVMeQueuePair *, s->queues, n + 1); >> s->queues[n] =

Re: [PATCH v3 16/16] block/nvme: Use per-queuepair IRQ notifier and AIO context

2020-07-06 Thread Stefan Hajnoczi
On Sat, Jul 04, 2020 at 11:30:51PM +0200, Philippe Mathieu-Daudé wrote: > @@ -683,6 +676,17 @@ static bool nvme_add_io_queue(BlockDriverState *bs, > Error **errp) > s->queues = g_renew(NVMeQueuePair *, s->queues, n + 1); > s->queues[n] = q; > s->nr_queues++; > + > +ret =

Re: [PATCH v3 15/16] block/nvme: Move nvme_poll_cb() earlier

2020-07-06 Thread Stefan Hajnoczi
On Sat, Jul 04, 2020 at 11:30:50PM +0200, Philippe Mathieu-Daudé wrote: > We are going to use this callback in nvme_add_io_queue() > in the next commit. To avoid forward-declaring it, move > it before. No logical change. > > Signed-off-by: Philippe Mathieu-Daudé > --- > block/nvme.c | 18

Re: [PATCH v3 14/16] block/nvme: Extract nvme_poll_queue()

2020-07-06 Thread Stefan Hajnoczi
On Sat, Jul 04, 2020 at 11:30:49PM +0200, Philippe Mathieu-Daudé wrote: > As we want to do per-queue polling, extract the nvme_poll_queue() > method which operates on a single queue. > > Signed-off-by: Philippe Mathieu-Daudé > --- > Stefan better double check here! > --- > block/nvme.c | 44

Re: [PATCH v3 06/16] block/nvme: Use common error path in nvme_add_io_queue()

2020-07-06 Thread Stefan Hajnoczi
On Sat, Jul 04, 2020 at 11:30:41PM +0200, Philippe Mathieu-Daudé wrote: > Rearrange nvme_add_io_queue() by using a common error path. > This will be proven useful in few commits where we add IRQ > notification to the IO queues. > > Signed-off-by: Philippe Mathieu-Daudé > --- > block/nvme.c | 9

Re: [PATCH v3 03/44] qdev: Use returned bool to check for qdev_realize() etc. failure

2020-07-06 Thread Markus Armbruster
Greg Kurz writes: > On Mon, 6 Jul 2020 10:09:09 +0200 > Markus Armbruster wrote: > >> Convert >> >> foo(..., ); >> if (err) { >> ... >> } >> >> to >> >> if (!foo(..., )) { >> ... >> } >> >> for qdev_realize(), qdev_realize_and_unref(), qbus_realize() and

Re: [PATCH v3 24/44] qom: Use return values to check for error where that's simpler

2020-07-06 Thread Vladimir Sementsov-Ogievskiy
06.07.2020 11:09, Markus Armbruster wrote: When using the Error object to check for error, we need to receive it into a local variable, then propagate() it to @errp. Using the return value permits allows receiving it straight to @errp. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake

Re: [PATCH v3 05/16] block/nvme: Improve error message when IO queue creation failed

2020-07-06 Thread Stefan Hajnoczi
On Sat, Jul 04, 2020 at 11:30:40PM +0200, Philippe Mathieu-Daudé wrote: > Do not use the same error message for different failures. > Display a different error whether it is the CQ or the SQ. > > Signed-off-by: Philippe Mathieu-Daudé > --- > block/nvme.c | 4 ++-- > 1 file changed, 2

Re: [PULL 00/31] Block patches

2020-07-06 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200706100432.2301919-1-mre...@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [PULL 00/31] Block patches Type: series Message-id: 20200706100432.2301919-1-mre...@redhat.com === TEST

[PULL 31/31] qed: Simplify backing reads

2020-07-06 Thread Max Reitz
From: Eric Blake The other four drivers that support backing files (qcow, qcow2, parallels, vmdk) all rely on the block layer to populate zeroes when reading beyond EOF of a short backing file. We can simplify the qed code by doing likewise. Signed-off-by: Eric Blake Reviewed-by: Vladimir

[PULL 30/31] block: drop unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy Currently this field only set by qed and qcow2. But in fact, all backing-supporting formats (parallels, qcow, qcow2, qed, vmdk) share these semantics: on unallocated blocks, if there is no backing file they just memset the buffer with zeroes. So, document this

Re: [PATCH v2 4/7] vhost: involve device backends in feature negotiation

2020-07-06 Thread Stefan Hajnoczi
On Tue, Jun 09, 2020 at 02:07:44PM -0400, Michael S. Tsirkin wrote: > On Tue, Jun 09, 2020 at 06:02:15PM +0100, Stefan Hajnoczi wrote: > > Many vhost devices in QEMU currently do not involve the device backend > > in feature negotiation. This seems fine at first glance when no > > device-specific

[PULL 23/31] block: inline bdrv_unallocated_blocks_are_zero()

2020-07-06 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy The function has only one user: bdrv_co_block_status(). Inline it to simplify reviewing of the following patches, which will finally drop unallocated_blocks_are_zero field too. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id:

[PULL 27/31] block/iscsi: drop unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy We set bdi->unallocated_blocks_are_zero = iscsilun->lbprz, but iscsi_co_block_status doesn't return 0 in case of iscsilun->lbprz, it returns ZERO when appropriate. So actually unallocated_blocks_are_zero is useless (it doesn't affect the only user of the field:

[PULL 20/31] block/qcow2: implement blockdev-amend

2020-07-06 Thread Max Reitz
From: Maxim Levitsky Currently the implementation only supports amending the encryption options, unlike the qemu-img version Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Reviewed-by: Max Reitz Message-Id: <20200608094030.670121-14-mlevi...@redhat.com> Signed-off-by: Max

[PULL 21/31] iotests: add tests for blockdev-amend

2020-07-06 Thread Max Reitz
From: Maxim Levitsky This commit adds two tests that cover the new blockdev-amend functionality of luks and qcow2 driver Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé [mreitz: Let 295 verify that LUKS works; drop 295 and 296 from the auto group] Signed-off-by: Max

Re: [PATCH v3 03/44] qdev: Use returned bool to check for qdev_realize() etc. failure

2020-07-06 Thread Greg Kurz
On Mon, 6 Jul 2020 10:09:09 +0200 Markus Armbruster wrote: > Convert > > foo(..., ); > if (err) { > ... > } > > to > > if (!foo(..., )) { > ... > } > > for qdev_realize(), qdev_realize_and_unref(), qbus_realize() and their > wrappers

[PULL 28/31] block/file-posix: drop unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy raw_co_block_status() in block/file-posix.c never returns 0, so unallocated_blocks_are_zero is useless (it doesn't affect the only user of the field: bdrv_co_block_status()). Drop it. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake

[PULL 15/31] block/crypto: implement the encryption key management

2020-07-06 Thread Max Reitz
From: Maxim Levitsky This implements the encryption key management using the generic code in qcrypto layer and exposes it to the user via qemu-img This code adds another 'write_func' because the initialization write_func works directly on the underlying file, and amend works on instance of luks

[PULL 25/31] block/vpc: return ZERO block-status when appropriate

2020-07-06 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy In case when get_image_offset() returns -1, we do zero out the corresponding chunk of qiov. So, this should be reported as ZERO. Note that this changes visible output of "qemu-img map --output=json" and "qemu-io -c map" commands. For qemu-img map, the change

[PULL 29/31] block/vhdx: drop unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy vhdx doesn't have .bdrv_co_block_status handler, so DATA|ALLOCATED is always assumed for it in bdrv_co_block_status(). unallocated_blocks_are_zero is useless (it doesn't affect the only user of the field: bdrv_co_block_status()), drop it. Signed-off-by:

[PULL 14/31] block/crypto: rename two functions

2020-07-06 Thread Max Reitz
From: Maxim Levitsky rename the write_func to create_write_func, and init_func to create_init_func. This is preparation for other write_func that will be used to update the encryption keys. No functional changes Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Message-Id:

[PULL 26/31] block/crypto: drop unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy It's false by default, no needs to set it. We are going to drop this variable at all, so drop it now here, it doesn't hurt. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id: <20200528094405.145708-6-vsement...@virtuozzo.com>

[PULL 22/31] qemu-img: convert: don't use unallocated_blocks_are_zero

2020-07-06 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy qemu-img convert wants to distinguish ZERO which comes from short backing files. unallocated_blocks_are_zero field of bdi is unrelated: space after EOF is always considered to be zero anyway. So, just make post_backing_zero true in case of short backing file.

[PULL 12/31] block/amend: separate amend and create options for qemu-img

2020-07-06 Thread Max Reitz
From: Maxim Levitsky Some options are only useful for creation (or hard to be amended, like cluster size for qcow2), while some other options are only useful for amend, like upcoming keyslot management options for luks Since currently only qcow2 supports amend, move all its options to a common

[PULL 24/31] block/vdi: return ZERO block-status when appropriate

2020-07-06 Thread Max Reitz
From: Vladimir Sementsov-Ogievskiy In case of !VDI_IS_ALLOCATED[], we do zero out the corresponding chunk of qiov. So, this should be reported as ZERO. Note that this changes visible output of "qemu-img map --output=json" and "qemu-io -c map" commands. For qemu-img map, the change is obvious:

[PULL 19/31] block/crypto: implement blockdev-amend

2020-07-06 Thread Max Reitz
From: Maxim Levitsky Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Reviewed-by: Max Reitz Message-Id: <20200608094030.670121-13-mlevi...@redhat.com> Signed-off-by: Max Reitz --- qapi/block-core.json | 14 - block/crypto.c | 72

[PULL 16/31] block/qcow2: extend qemu-img amend interface with crypto options

2020-07-06 Thread Max Reitz
From: Maxim Levitsky Now that we have all the infrastructure in place, wire it in the qcow2 driver and expose this to the user. Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Reviewed-by: Max Reitz Message-Id: <20200608094030.670121-9-mlevi...@redhat.com> Signed-off-by: Max

[PULL 18/31] block/core: add generic infrastructure for x-blockdev-amend qmp command

2020-07-06 Thread Max Reitz
From: Maxim Levitsky blockdev-amend will be used similiar to blockdev-create to allow on the fly changes of the structure of the format based block devices. Current plan is to first support encryption keyslot management for luks based formats (raw and embedded in qcow2) Signed-off-by: Maxim

[PULL 13/31] block/amend: refactor qcow2 amend options

2020-07-06 Thread Max Reitz
From: Maxim Levitsky Some qcow2 create options can't be used for amend. Remove them from the qcow2 create options and add generic logic to detect such options in qemu-img Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé [mreitz: Dropped some iotests reference output hunks that

[PULL 08/31] iotests: Check whether luks works

2020-07-06 Thread Max Reitz
Whenever running an iotest for the luks format, we should check whether luks actually really works. Tests that try to create luks-encrypted qcow2 images should do the same. Signed-off-by: Max Reitz Message-Id: <20200625125548.870061-7-mre...@redhat.com> Reviewed-by: Maxim Levitsky ---

[PULL 07/31] iotests.py: Add (verify|has)_working_luks()

2020-07-06 Thread Max Reitz
Similar to _require_working_luks for bash tests, these functions can be used to check whether our luks driver can actually create images. Signed-off-by: Max Reitz Message-Id: <20200625125548.870061-6-mre...@redhat.com> Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/iotests.py | 39

[PULL 17/31] iotests: qemu-img tests for luks key management

2020-07-06 Thread Max Reitz
From: Maxim Levitsky This commit adds two tests, which test the new amend interface of both luks raw images and qcow2 luks encrypted images. Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé [mreitz: Let 293 verify that LUKS works; drop $(seq) usage from 293; drop 293 and

[PULL 06/31] iotests.py: Add qemu_img_pipe_and_status()

2020-07-06 Thread Max Reitz
This function will be used by the next patch, which intends to check both the exit code and qemu-img's output. Signed-off-by: Max Reitz Message-Id: <20200625125548.870061-5-mre...@redhat.com> Reviewed-by: Maxim Levitsky [mreitz: Rebased on 49438972b8c2e] Signed-off-by: Max Reitz ---

[PULL 10/31] qcrypto/luks: implement encryption key management

2020-07-06 Thread Max Reitz
From: Maxim Levitsky Next few patches will expose that functionality to the user. Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Message-Id: <20200608094030.670121-3-mlevi...@redhat.com> Signed-off-by: Max Reitz --- qapi/crypto.json| 59 ++- crypto/block-luks.c | 416

[PULL 09/31] qcrypto/core: add generic infrastructure for crypto options amendment

2020-07-06 Thread Max Reitz
From: Maxim Levitsky This will be used first to implement luks keyslot management. block_crypto_amend_opts_init will be used to convert qemu-img cmdline to QCryptoBlockAmendOptions Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé Message-Id:

[PULL 04/31] iotests: filter few more luks specific create options

2020-07-06 Thread Max Reitz
From: Maxim Levitsky This allows more tests to be able to have same output on both qcow2 luks encrypted images and raw luks images Signed-off-by: Maxim Levitsky Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky Message-Id: <20200625125548.870061-3-mre...@redhat.com> ---

  1   2   >