Re: [Qemu-block] [Libguestfs] [nbdkit PATCH 3/3] plugins: Add .can_fast_zero hook

On 8/23/19 9:40 AM, Eric Blake wrote: > Allow plugins to affect the handling of the new NBD_CMD_FLAG_FAST_ZERO > flag, then update affected plugins. In particular, in-memory plugins > are always fast; the full plugin is better served by omitting .zero > and relying on .pwrite fallback; nbd takes

Re: [Qemu-block] [Qemu-devel] [PATCH v5 0/4] virtio/block: handle zoned backing devices

Patchew URL: https://patchew.org/QEMU/20190823194927.23278-1-dmitry.fomic...@wdc.com/ Hi, This series seems to have some coding style problems. See output below for more information: Type: series Subject: [Qemu-devel] [PATCH v5 0/4] virtio/block: handle zoned backing devices Message-id:

Re: [Qemu-block] [Qemu-devel] [RFC PATCH 17/17] build: Correct non-common common-obj-* to obj-*

Thanks for splitting this. The good news is that most of these devices actually need _not_ be target-endian. Comments are inline. Paolo On 23/08/19 20:56, Tony Nguyen wrote: > diff --git a/hw/audio/Makefile.objs b/hw/audio/Makefile.objs > index 63db383709..13133b412d 100644 > ---

Re: [Qemu-block] [Qemu-devel] [PATCH v4 0/4] virtio/block: handle zoned backing devices

Patchew URL: https://patchew.org/QEMU/20190823192226.21154-1-dmitry.fomic...@wdc.com/ Hi, This series seems to have some coding style problems. See output below for more information: Type: series Subject: [Qemu-devel] [PATCH v4 0/4] virtio/block: handle zoned backing devices Message-id:

[Qemu-block] [PATCH v5 3/4] block/ide/scsi: Set BLK_PERM_SUPPORT_ZONED

Added a new boolean argument to blkconf_apply_backend_options() to let the common block code know whether the chosen block backend can handle zoned block devices or not. blkconf_apply_backend_options() then sets BLK_PERM_SUPPORT_ZONED permission accordingly. The raw code can then use this

[Qemu-block] [PATCH v5 4/4] raw: Don't open ZBDs if backend can't handle them

Abort opening a zoned device as a raw file in case the chosen block backend driver lacks proper support for this type of storage. Signed-off-by: Dmitry Fomichev --- block/file-posix.c | 14 ++ hw/block/fdc.c | 2 +- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git

[Qemu-block] [Qemu-devel] [PATCH 2/9] exec: Replace DEVICE_NATIVE_ENDIAN with MO_TE

Simplify endianness comparisons with consistent use of the more expressive MemOp. Suggested-by: Richard Henderson Signed-off-by: Tony Nguyen Reviewed-by: Richard Henderson Acked-by: David Gibson --- exec.c | 8 +++ hw/adc/stm32f2xx_adc.c | 2 +-

[Qemu-block] [PATCH v5 1/4] block: Add zoned device model property

This commit adds Zoned Device Model (as defined in T10 ZBC and T13 ZAC standards) as a block driver property, along with some useful access functions. A new backend driver permission, BLK_PERM_SUPPORT_ZONED, is also introduced. Only the drivers having this permission will be allowed to open zoned

[Qemu-block] [PATCH v5 2/4] raw: Recognize zoned backing devices

The purpose of this patch is to recognize a zoned block device (ZBD) when it is opened as a raw file. The new code initializes the zoned model propery introduced by the previous commit. This commit is Linux-specific as it gets the Zoned Block Device Model value (none/host-managed/host-aware) from

[Qemu-block] [PATCH v5 0/4] virtio/block: handle zoned backing devices

Ping... Any objections to merging this patchset? Ask me if you are not sure how to validate these patches without having the hardware :) Currently, attaching zoned block devices (i.e., storage devices compliant to ZAC/ZBC standards) using several virtio methods doesn't work properly as zoned

[Qemu-block] [PATCH v4 2/4] raw: Recognize zoned backing devices

The purpose of this patch is to recognize a zoned block device (ZBD) when it is opened as a raw file. The new code initializes the zoned model propery introduced by the previous commit. This commit is Linux-specific as it gets the Zoned Block Device Model value (none/host-managed/host-aware) from

[Qemu-block] [PATCH v4 3/4] block/ide/scsi: Set BLK_PERM_SUPPORT_ZONED

Added a new boolean argument to blkconf_apply_backend_options() to let the common block code know whether the chosen block backend can handle zoned block devices or not. blkconf_apply_backend_options() then sets BLK_PERM_SUPPORT_ZONED permission accordingly. The raw code can then use this

[Qemu-block] [PATCH v4 1/4] block: Add zoned device model property

This commit adds Zoned Device Model (as defined in T10 ZBC and T13 ZAC standards) as a block driver property, along with some useful access functions. A new backend driver permission, BLK_PERM_SUPPORT_ZONED, is also introduced. Only the drivers having this permission will be allowed to open zoned

Re: [Qemu-block] [PATCH 1/1] protocol: Add NBD_CMD_FLAG_FAST_ZERO

On 8/23/19 1:48 PM, Wouter Verhelst wrote: > On Fri, Aug 23, 2019 at 09:34:26AM -0500, Eric Blake wrote: >> +- bit 4, `NBD_CMD_FLAG_FAST_ZERO`; valid during >> + `NBD_CMD_WRITE_ZEROES`. If set, but the server cannot perform the >> + write zeroes any faster than it would for an equivalent >> +

[Qemu-block] [Qemu-devel] [RFC PATCH 17/17] build: Correct non-common common-obj-* to obj-*

Device realizing code with MemorRegionOps endianness as DEVICE_NATIVE_ENDIAN is not common code. Corrected devices were identified by making the declaration of DEVICE_NATIVE_ENDIAN conditional upon NEED_CPU_H and then listing what failed to compile. Signed-off-by: Tony Nguyen ---

[Qemu-block] [PATCH v4 2/4] raw: Recognize zoned backing devices

The purpose of this patch is to recognize a zoned block device (ZBD) when it is opened as a raw file. The new code initializes the zoned model propery introduced by the previous commit. This commit is Linux-specific as it gets the Zoned Block Device Model value (none/host-managed/host-aware) from

[Qemu-block] [PATCH v4 1/4] block: Add zoned device model property

This commit adds Zoned Device Model (as defined in T10 ZBC and T13 ZAC standards) as a block driver property, along with some useful access functions. A new backend driver permission, BLK_PERM_SUPPORT_ZONED, is also introduced. Only the drivers having this permission will be allowed to open zoned

[Qemu-block] [PATCH v4 0/4] virtio/block: handle zoned backing devices

Ping... Any objections to merging this patchset? Ask me if you are not sure how to validate these patches without having the hardware :) Currently, attaching zoned block devices (i.e., storage devices compliant to ZAC/ZBC standards) using several virtio methods doesn't work properly as zoned

[Qemu-block] [PATCH v4 4/4] raw: Don't open ZBDs if backend can't handle them

Abort opening a zoned device as a raw file in case the chosen block backend driver lacks proper support for this type of storage. Signed-off-by: Dmitry Fomichev --- block/file-posix.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/block/file-posix.c b/block/file-posix.c

Re: [Qemu-block] [PATCH 1/1] protocol: Add NBD_CMD_FLAG_FAST_ZERO

On Fri, Aug 23, 2019 at 09:34:26AM -0500, Eric Blake wrote: > +- bit 4, `NBD_CMD_FLAG_FAST_ZERO`; valid during > + `NBD_CMD_WRITE_ZEROES`. If set, but the server cannot perform the > + write zeroes any faster than it would for an equivalent > + `NBD_CMD_WRITE`, One way of fulfilling the letter

[Qemu-block] [PATCH 2/2] iotests: Test blockdev-create for vpc

Signed-off-by: Max Reitz --- tests/qemu-iotests/266 | 182 + tests/qemu-iotests/266.out | 107 ++ tests/qemu-iotests/group | 1 + 3 files changed, 290 insertions(+) create mode 100755 tests/qemu-iotests/266 create mode 100644

[Qemu-block] [Qemu-devel] [RFC PATCH 02/17] hw/block: Declare device little or big endian

For each device declared with DEVICE_NATIVE_ENDIAN, find the set of targets from the set of target/hw/*/device.o. If the set of targets are all little or all big endian, re-declare the device endianness as DEVICE_LITTLE_ENDIAN or DEVICE_BIG_ENDIAN respectively. This *naive* deduction may result

[Qemu-block] [PATCH 1/2] block: Let blockdev-create return 0 on success

Block drivers should let their .bdrv_co_create() implementation return a non-negative value to indicate success. However, jobs should return exactly 0. Thus, we need to translate positive return values to 0 in blockdev_create_run(). Cc: qemu-sta...@nongnu.org Signed-off-by: Max Reitz ---

[Qemu-block] [PATCH 0/2] block: Let blockdev-create return 0 on success

Jobs are expected to return 0 on success. .bdrv_co_create() on the other hand is a block layer function, and as such returns a non-negative value on success. blockdev_create_run() should translate between the two (patch 1). Without patch 1, blockdev-create is likely to fail for VPC images.

Re: [Qemu-block] [PATCH] block: posix: Always allocate the first block

On 23.08.19 18:48, Nir Soffer wrote: > On Fri, Aug 23, 2019 at 4:58 PM Max Reitz > wrote: [...] > If you have a format layer that truncates the image to a fixed size and > does not write anything into the first block itself (say because it uses > a footer),

Re: [Qemu-block] [PATCH] block: posix: Always allocate the first block

On 23.08.19 18:30, Nir Soffer wrote: > On Fri, Aug 23, 2019 at 4:58 PM Max Reitz > wrote: > [...] > Exactly.  But if posix_fallocate() works, it should have allocated the > first block. > > > Only if the file system does not support fallocate().

Re: [Qemu-block] [PATCH] block: posix: Always allocate the first block

On Fri, Aug 23, 2019 at 4:58 PM Max Reitz wrote: > On 22.08.19 21:01, Nir Soffer wrote: > > On Thu, Aug 22, 2019 at 9:11 PM Max Reitz > > wrote: > > > > On 22.08.19 18:39, Nir Soffer wrote: > > > On Thu, Aug 22, 2019 at 5:28 PM Max Reitz >

Re: [Qemu-block] [Qemu-devel] [PATCH 5/5] nbd: Tolerate more errors to structured reply request

[dropping non-qemu lists] On 8/23/19 9:37 AM, Eric Blake wrote: > A server may have a reason to reject a request for structured replies, > beyond just not recognizing them as a valid request. It doesn't hurt > us to continue talking to such a server; otherwise 'qemu-nbd --list' > of such a

Re: [Qemu-block] [PATCH] block: posix: Always allocate the first block

On Fri, Aug 23, 2019 at 4:58 PM Max Reitz wrote: > On 22.08.19 21:01, Nir Soffer wrote: > ... > > > > @@ -1794,6 +1815,8 @@ static int handle_aiocb_truncate(void > > *opaque) > > > > /* posix_fallocate() doesn't set errno. */ > > > >

Re: [Qemu-block] [Qemu-devel] [PATCH v2 00/12] block: qiov_offset parameter for io

On Thu, Aug 22, 2019 at 05:59:43PM +, Vladimir Sementsov-Ogievskiy wrote: > 22.08.2019 20:39, Vladimir Sementsov-Ogievskiy wrote: > > 22.08.2019 20:24, Vladimir Sementsov-Ogievskiy wrote: > >> 22.08.2019 18:50, Stefan Hajnoczi wrote: > >>> On Tue, Jun 04, 2019 at 07:15:02PM +0300, Vladimir

Re: [Qemu-block] [PATCH v5 6/6] iotests: extend sleeping time under Valgrind

16.08.2019 4:01, John Snow wrote: > > > On 7/19/19 12:30 PM, Andrey Shinkevich wrote: >> To synchronize the time when QEMU is running longer under the Valgrind, >> increase the sleeping time in the test 247. >> >> Signed-off-by: Andrey Shinkevich >> Reviewed-by: Vladimir Sementsov-Ogievskiy >>

Re: [Qemu-block] [Qemu-devel] cross-project patches: Add NBD Fast Zero support

23.08.2019 17:30, Eric Blake wrote: > This is a cover letter to a series of patches being proposed in tandem > to four different projects: I always knew you were great, but that breaks all the records > - nbd: Document a new NBD_CMD_FLAG_FAST_ZERO command flag > - qemu: Implement the flag for

[Qemu-block] [nbdkit PATCH 0/3] nbdkit support for new NBD fast zero

See the cross-post cover letter for details: https://www.redhat.com/archives/libguestfs/2019-August/msg00322.html Notably, this series did NOT add fast zero support to the file plugin yet; there, I probably need to do more testing and/or kernel source code reading to learn whether to mark

[Qemu-block] [nbdkit PATCH 2/3] filters: Add .can_fast_zero hook

Allow filters to affect the handling of the new NBD_CMD_FLAG_FAST_ZERO flag, then update affected filters. In particular, the log filter logs the state of the new flag (requires a tweak to expected output in test-fua.sh), the delay filter gains a bool parameter delay-fast-zero, several filters

[Qemu-block] [PATCH 4/5] nbd: Implement server use of NBD FAST_ZERO

The server side is fairly straightforward: we can always advertise support for detection of fast zero, and implement it by mapping the request to the block layer BDRV_REQ_NO_FALLBACK. Signed-off-by: Eric Blake --- Again, this may be worth merging with the previous patch. --- nbd/server.c | 8

[Qemu-block] [PATCH 5/5] nbd: Tolerate more errors to structured reply request

A server may have a reason to reject a request for structured replies, beyond just not recognizing them as a valid request. It doesn't hurt us to continue talking to such a server; otherwise 'qemu-nbd --list' of such a server fails to display all possible details about the export. Encountered

[Qemu-block] [nbdkit PATCH 1/3] server: Add internal support for NBDKIT_FLAG_FAST_ZERO

Qemu was able to demonstrate that knowing whether a zero operation is fast is useful when copying from one image to another: there is a choice between bulk pre-zeroing and then revisiting the data sections (fewer transactions, but depends on the zeroing to be fast), vs. visiting every portion of

[Qemu-block] [libnbd PATCH 1/1] api: Add support for FAST_ZERO flag

Qemu was able to demonstrate that knowing whether a zero operation is fast is useful when copying from one image to another: there is a choice between bulk pre-zeroing and then revisiting the data sections (fewer transactions, but depends on the zeroing to be fast), vs. visiting every portion of

[Qemu-block] [libnbd PATCH 0/1] libnbd support for new fast zero

See the cross-post cover letter for details: https://www.redhat.com/archives/libguestfs/2019-August/msg00322.html Eric Blake (1): api: Add support for FAST_ZERO flag lib/nbd-protocol.h | 2 ++ generator/generator| 29 +++-- lib/flags.c| 12

[Qemu-block] [PATCH 1/5] nbd: Improve per-export flag handling in server

When creating a read-only image, we are still advertising support for TRIM and WRITE_ZEROES to the client, even though the client should not be issuing those commands. But seeing this requires looking across multiple functions: All callers to nbd_export_new() passed a single flag based solely on

[Qemu-block] [PATCH 1/1] protocol: Add NBD_CMD_FLAG_FAST_ZERO

While it may be counterintuitive at first, the introduction of NBD_CMD_WRITE_ZEROES and NBD_CMD_BLOCK_STATUS has caused a performance regression in qemu [1], when copying a sparse file. When the destination file must contain the same contents as the source, but it is not known in advance whether

[Qemu-block] [nbdkit PATCH 3/3] plugins: Add .can_fast_zero hook

Allow plugins to affect the handling of the new NBD_CMD_FLAG_FAST_ZERO flag, then update affected plugins. In particular, in-memory plugins are always fast; the full plugin is better served by omitting .zero and relying on .pwrite fallback; nbd takes advantage of libnbd extensions proposed in

[Qemu-block] [PATCH 0/1] NBD protocol change to add fast zero support

See the cross-post cover letter for details: https://www.redhat.com/archives/libguestfs/2019-August/msg00322.html Eric Blake (1): protocol: Add NBD_CMD_FLAG_FAST_ZERO doc/proto.md | 50 +- 1 file changed, 49 insertions(+), 1 deletion(-) --

[Qemu-block] [PATCH 2/5] nbd: Prepare for NBD_CMD_FLAG_FAST_ZERO

Commit fe0480d6 and friends added BDRV_REQ_NO_FALLBACK as a way to avoid wasting time on a preliminary write-zero request that will later be rewritten by actual data, if it is known that the write-zero request will use a slow fallback; but in doing so, could not optimize for NBD. The NBD

[Qemu-block] [PATCH 3/5] nbd: Implement client use of NBD FAST_ZERO

The client side is fairly straightforward: if the server advertised fast zero support, then we can map that to BDRV_REQ_NO_FALLBACK support. A server that advertises FAST_ZERO but not WRITE_ZEROES is technically broken, but we can ignore that situation as it does not change our behavior.

[Qemu-block] [PATCH v2] job: drop job_drain

In job_finish_sync job_enter should be enough for a job to make some progress and draining is a wrong tool for it. So use job_enter directly here and drop job_drain with all related staff not used more. Suggested-by: Kevin Wolf Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: drop drain

[Qemu-block] cross-project patches: Add NBD Fast Zero support

This is a cover letter to a series of patches being proposed in tandem to four different projects: - nbd: Document a new NBD_CMD_FLAG_FAST_ZERO command flag - qemu: Implement the flag for both clients and server - libnbd: Implement the flag for clients - nbdkit: Implement the flag for servers,

Re: [Qemu-block] [PATCH] block: posix: Always allocate the first block

On 22.08.19 21:01, Nir Soffer wrote: > On Thu, Aug 22, 2019 at 9:11 PM Max Reitz > wrote: > > On 22.08.19 18:39, Nir Soffer wrote: > > On Thu, Aug 22, 2019 at 5:28 PM Max Reitz > >

[Qemu-block] [PATCH v2] iotests: Check for enabled drivers before testing them

It is possible to enable only a subset of the block drivers with the "--block-drv-rw-whitelist" option of the "configure" script. All other drivers are marked as unusable (or only included as read-only with the "--block-drv-ro-whitelist" option). If an iotest is now using such a disabled block

[Qemu-block] [PATCH 2/2] iotests: Test reverse sub-cluster qcow2 writes

This exercises the regression introduced in commit 50ba5b2d994853b38fed10e0841b119da0f8b8e5. On my machine, it has close to a 50 % false-negative rate, but that should still be sufficient to test the fix. Signed-off-by: Max Reitz --- tests/qemu-iotests/265 | 67

[Qemu-block] [PATCH 1/2] block/file-posix: Reduce xfsctl() use

This patch removes xfs_write_zeroes() and xfs_discard(). Both functions have been added just before the same feature was present through fallocate(): - fallocate() has supported PUNCH_HOLE for XFS since Linux 2.6.38 (March 2011); xfs_discard() was added in December 2010. - fallocate() has

[Qemu-block] [PATCH 0/2] block/file-posix: Reduce xfsctl() use

Hi, As suggested by Paolo, this series drops xfsctl() calls where we have working fallocate() alternatives. (And thus replaces “block/file-posix: Fix xfs_write_zeroes()”.) Unfortunately, we also use xfsctl() to inquire the request alignment for O_DIRECT, and this is the only way we currently

Re: [Qemu-block] [PATCH] block: workaround for unaligned byte range in fallocate()

On 23/08/2019 00:09, Eric Blake wrote: > On 8/22/19 1:31 PM, Andrey Shinkevich wrote: >> Revert the commit 118f99442d 'block/io.c: fix for the allocation failure' >> and make better error handling for the file systems that do not support > > s/make/use/ > >> fallocate() for the unaligned byte

Re: [Qemu-block] [Qemu-devel] [PATCH] tests/check-block: Skip iotests when sanitizers are enabled

On 8/23/19 11:53 AM, Peter Maydell wrote: > On Fri, 23 Aug 2019 at 10:35, Thomas Huth wrote: >> >> On 8/23/19 11:04 AM, Peter Maydell wrote: >>> On Fri, 23 Aug 2019 at 09:43, Thomas Huth wrote: The sanitizers (especially the address sanitizer from Clang) are sometimes printing out

Re: [Qemu-block] [Qemu-devel] [PATCH] tests/check-block: Skip iotests when sanitizers are enabled

On Fri, 23 Aug 2019 at 10:35, Thomas Huth wrote: > > On 8/23/19 11:04 AM, Peter Maydell wrote: > > On Fri, 23 Aug 2019 at 09:43, Thomas Huth wrote: > >> > >> The sanitizers (especially the address sanitizer from Clang) are > >> sometimes printing out warnings or false positives - this spoils >

Re: [Qemu-block] [Qemu-devel] [PATCH] tests/check-block: Skip iotests when sanitizers are enabled

On 8/23/19 11:04 AM, Peter Maydell wrote: > On Fri, 23 Aug 2019 at 09:43, Thomas Huth wrote: >> >> The sanitizers (especially the address sanitizer from Clang) are >> sometimes printing out warnings or false positives - this spoils >> the output of the iotests, causing some of the tests to fail.

Re: [Qemu-block] [PATCH 2/2] qapi: deprecate implicit filters

14.08.2019 13:07, Vladimir Sementsov-Ogievskiy wrote: > To get rid of implicit filters related workarounds in future let's > deprecate them now. Interesting, could we deprecate implicit filter without deprecation of unnecessity of parameter? As actually, it's good when this parameter is not

Re: [Qemu-block] [Qemu-devel] [PATCH] tests/check-block: Skip iotests when sanitizers are enabled

On Fri, 23 Aug 2019 at 09:43, Thomas Huth wrote: > > The sanitizers (especially the address sanitizer from Clang) are > sometimes printing out warnings or false positives - this spoils > the output of the iotests, causing some of the tests to fail. > Thus let's skip the automatic iotests during

Re: [Qemu-block] [Qemu-devel] [PULL 2/3] tests: Run the iotests during "make check" again

On 8/23/19 9:12 AM, Thomas Huth wrote: > On 8/23/19 12:34 AM, Paolo Bonzini wrote: >> On 17/08/19 10:54, Thomas Huth wrote: >>> People often forget to run the iotests before submitting patches or pull >>> requests - this is likely due to the fact that we do not run the tests >>> during our

[Qemu-block] [PATCH] tests/check-block: Skip iotests when sanitizers are enabled

The sanitizers (especially the address sanitizer from Clang) are sometimes printing out warnings or false positives - this spoils the output of the iotests, causing some of the tests to fail. Thus let's skip the automatic iotests during "make check" when the user configured QEMU with

Re: [Qemu-block] [Qemu-devel] [PATCH 0/2] block/file-posix: Fix xfs_write_zeroes()

On 22/8/2019 7:26 PM, Max Reitz wrote: > Lukàš ran over a nasty regression in our xfs_write_zeroes() function > (sorry, my fault) made apparent by a recent patch from Anton that makes > qcow2 images heavily exercise the offending code path. > > This series fixes the bug and adds a test to prevent

Re: [Qemu-block] [Qemu-devel] [PATCH 16/16] nvme: support multiple namespaces

On Thu, Aug 22, 2019 at 02:18:05PM +0100, Ross Lagerwall wrote: > On 7/5/19 8:23 AM, Klaus Birkelund Jensen wrote: > > I tried this patch series by installing Windows with a single NVME > controller having two namespaces. QEMU crashed in get_feature / > NVME_VOLATILE_WRITE_CACHE because req->ns

Re: [Qemu-block] [Qemu-devel] [PULL 2/3] tests: Run the iotests during "make check" again

On 8/23/19 12:34 AM, Paolo Bonzini wrote: > On 17/08/19 10:54, Thomas Huth wrote: >> People often forget to run the iotests before submitting patches or pull >> requests - this is likely due to the fact that we do not run the tests >> during our mandatory "make check" tests yet. Now that we've got

Re: [Qemu-block] [Qemu-devel] [PATCH 0/2] block/file-posix: Fix xfs_write_zeroes()

Dne 22. 08. 19 v 19:09 Max Reitz napsal(a): > On 22.08.19 18:53, Paolo Bonzini wrote: >> On 22/08/19 18:26, Max Reitz wrote: >>> Lukàš ran over a nasty regression in our xfs_write_zeroes() function >>> (sorry, my fault) made apparent by a recent patch from Anton that makes >>> qcow2 images heavily