Re: [Qemu-block] [PATCH] nvme: add Get/Set Feature Timestamp support

On Fri, May 17, 2019 at 07:35:04AM +0200, Klaus Birkelund wrote: > Hi Kenneth, > > On Thu, May 16, 2019 at 05:24:47PM -0600, Heitke, Kenneth wrote: > > Hi Klaus, thank you for you review. I have one comment inline > > > > On 5/14/2019 12:02 AM, Klaus Birkelund wrote: > > > On Fri, Apr 05, 2019 at

Re: [Qemu-block] [PATCH] nvme: add Get/Set Feature Timestamp support

Hi Kenneth, On Thu, May 16, 2019 at 05:24:47PM -0600, Heitke, Kenneth wrote: > Hi Klaus, thank you for you review. I have one comment inline > > On 5/14/2019 12:02 AM, Klaus Birkelund wrote: > > On Fri, Apr 05, 2019 at 03:41:17PM -0600, Kenneth Heitke wrote: > > > Signed-off-by: Kenneth Heitke >

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] qapi: support external bitmaps in block-dirty-bitmap-merge

On 5/16/19 8:27 AM, Vladimir Sementsov-Ogievskiy wrote: > Add new optional parameter making possible to merge bitmaps from > different nodes. It is needed to maintain external snapshots during > incremental backup chain history. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > qapi/blo

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/4] blockdev-backup: don't check aio_context too early

Happy Friday: ping! Max: this series corrects some things that were causing some of the pitfalls that made me nervous about the context I wrote about in 219. On 5/10/19 3:03 PM, John Snow wrote: > See patch one's commit message for justification. > > v2: added patch 4, with iotest framework adju

Re: [Qemu-block] [PATCH] iotests: Fix intermittent failure in 219

On 5/16/19 12:11 PM, Max Reitz wrote: > In 219, we wait for the job to make progress before we emit its status. > This makes the output reliable. > > Unfortunately, there is a bug: We do not wait for any more progress if > the job has already reached its total-progress. Right after the job has

Re: [Qemu-block] [PATCH] nvme: add Get/Set Feature Timestamp support

Hi Klaus, thank you for you review. I have one comment inline On 5/14/2019 12:02 AM, Klaus Birkelund wrote: On Fri, Apr 05, 2019 at 03:41:17PM -0600, Kenneth Heitke wrote: Signed-off-by: Kenneth Heitke --- hw/block/nvme.c | 120 +- hw/block/nvme

Re: [Qemu-block] [PATCH v1] [RFC] qcow2: add compression type feature

On 5/16/19 9:48 AM, Denis Plotnikov wrote: > The patch adds some preparation parts for incompatible compression type > feature into QCOW2 header that indicates that *all* compressed clusters > must be (de)compressed using a certain compression type. > > It is implied that the compression type i

Re: [Qemu-block] [Qemu-devel] [PATCH v2] migration/dirty-bitmaps: change bitmap enumeration method

On 5/16/19 6:12 AM, Vladimir Sementsov-Ogievskiy wrote: > 14.05.2019 23:19, John Snow wrote: >> Shift from looking at every root BDS to *every* BDS. This will migrate >> bitmaps that are attached to blockdev created nodes instead of just ones >> attached to emulated storage devices. >> >> Note t

Re: [Qemu-block] [PATCH] iotests: Fix intermittent failure in 219

On 16.05.19 18:11, Max Reitz wrote: > In 219, we wait for the job to make progress before we emit its status. > This makes the output reliable. > > Unfortunately, there is a bug: We do not wait for any more progress if > the job has already reached its total-progress. Right after the job has > be

[Qemu-block] [PATCH] iotests: Fix intermittent failure in 219

In 219, we wait for the job to make progress before we emit its status. This makes the output reliable. Unfortunately, there is a bug: We do not wait for any more progress if the job has already reached its total-progress. Right after the job has been started, it is possible that total-progress i

Re: [Qemu-block] [PATCH v8 3/9] block: add empty account cookie type

On 16/5/2019 6:34 PM, Vladimir Sementsov-Ogievskiy wrote: > 16.05.2019 17:33, Anton Nefedov wrote: >> This adds some protection from accounting uninitialized cookie. >> That is, block_acct_failed/done without previous block_acct_start; >> in that case, cookie probably holds values from previous ope

Re: [Qemu-block] [PATCH v8 3/9] block: add empty account cookie type

16.05.2019 17:33, Anton Nefedov wrote: > This adds some protection from accounting uninitialized cookie. > That is, block_acct_failed/done without previous block_acct_start; > in that case, cookie probably holds values from previous operation. > > (Note: it might also be uninitialized holding garb

Re: [Qemu-block] [PATCH v8 8/9] file-posix: account discard operations

16.05.2019 17:33, Anton Nefedov wrote: > This will help to identify how many of the user-issued discard operations > (accounted on a device level) have actually suceeded down on the host file > (even though the numbers will not be exactly the same if non-raw format > driver is used (e.g. qcow2 send

Re: [Qemu-block] [PATCH v4 6/7] iotests: Test qemu-img convert --salvage

On 16.05.19 16:40, Vladimir Sementsov-Ogievskiy wrote: > 07.05.2019 23:35, Max Reitz wrote: >> This test converts a simple image to another, but blkdebug injects >> block_status and read faults at some offsets. The resulting image >> should be the same as the input image, except that sectors that

Re: [Qemu-block] [PATCH v14 1/1] qcow2: skip writing zero buffers to empty COW areas

16.05.2019 17:27, Anton Nefedov wrote: > If COW areas of the newly allocated clusters are zeroes on the backing > image, efficient bdrv_write_zeroes(flags=BDRV_REQ_NO_FALLBACK) can be > used on the whole cluster instead of writing explicit zero buffers later > in perform_cow(). > > iotest 060: > w

Re: [Qemu-block] [PATCH v4 6/7] iotests: Test qemu-img convert --salvage

07.05.2019 23:35, Max Reitz wrote: > This test converts a simple image to another, but blkdebug injects > block_status and read faults at some offsets. The resulting image > should be the same as the input image, except that sectors that could > not be read have to be 0. > > Signed-off-by: Max Re

[Qemu-block] [PATCH v3] iotests: Filter 175's allocation information

It is possible for an empty file to take up blocks on a filesystem, for example: $ qemu-img create -f raw test.img 1G Formatting 'test.img', fmt=raw size=1073741824 $ mkfs.ext4 -I 128 -q test.img $ mkdir test-mount $ sudo mount -o loop test.img test-mount $ sudo touch test-mount/test-file $ stat -

Re: [Qemu-block] [PATCH v1] [RFC] qcow2: add compression type feature

On 5/16/19 8:48 AM, Denis Plotnikov wrote: > The patch adds some preparation parts for incompatible compression type > feature into QCOW2 header that indicates that *all* compressed clusters > must be (de)compressed using a certain compression type. > > It is implied that the compression type is s

[Qemu-block] [PATCH v8 5/9] scsi: store unmap offset and nb_sectors in request struct

it allows to report it in the error handler Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia --- hw/scsi/scsi-disk.c | 12 +--- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c inde

Re: [Qemu-block] [PATCH 4/4] iotests: Make 245 faster and more reliable

On Wed 15 May 2019 10:15:03 PM CEST, Max Reitz wrote: > Sometimes, 245 fails for me because some stream job has already finished > while the test expects it to still be active. (With -c none, it fails > basically every time.) The most reliable way to fix this is to simply > set auto_finalize=false

[Qemu-block] [PATCH v8 4/9] ide: account UNMAP (TRIM) operations

Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy --- hw/ide/core.c | 12 1 file changed, 12 insertions(+) diff --git a/hw/ide/core.c b/hw/ide/core.c index 6afadf894f..3a7ac93777 100644 --- a/hw/ide/core.c +++ b/hw/ide/core.c @@ -441,6 +441,14 @@ static void id

[Qemu-block] [PATCH v8 6/9] scsi: move unmap error checking to the complete callback

This will help to account the operation in the following commit. The difference is that we don't call scsi_disk_req_check_error() before the 1st discard iteration anymore. That function also checks if the request is cancelled, however it shouldn't get canceled until it yields in blk_aio() function

[Qemu-block] [PATCH v8 8/9] file-posix: account discard operations

This will help to identify how many of the user-issued discard operations (accounted on a device level) have actually suceeded down on the host file (even though the numbers will not be exactly the same if non-raw format driver is used (e.g. qcow2 sending metadata discards)). Note that these numbe

[Qemu-block] [PATCH v2] iotest 134: test cluster-misaligned encrypted write

COW (even empty/zero) areas require encryption too Signed-off-by: Anton Nefedov Reviewed-by: Eric Blake Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia --- ..apparently v1 ended up in a weird base64 that would not easily git-am. Resending. used to be a part of 'qcow2: cluster space preall

[Qemu-block] [PATCH v8 7/9] scsi: account unmap operations

Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy --- hw/scsi/scsi-disk.c | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index 6eff496b54..5c77981d60 100644 --- a/hw/scsi/scsi-disk.c +++ b/hw/scsi/sc

[Qemu-block] [PATCH v8 2/9] qapi: add unmap to BlockDeviceStats

Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Eric Blake --- qapi/block-core.json | 29 +++-- include/block/accounting.h | 1 + block/qapi.c | 6 ++ tests/qemu-iotests/227.out |

[Qemu-block] [PATCH v8 9/9] qapi: query-blockstat: add driver specific file-posix stats

A block driver can provide a callback to report driver-specific statistics. file-posix driver now reports discard statistics Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy Acked-by: Markus Armbruster --- qapi/block-core.json | 38 ++

[Qemu-block] [PATCH v8 0/9] discard blockstats

..apparently v7 ended up in a weird base64 that would not easily git-am. Resending. hi, yet another take for this patch series; please kindly consider these for 4.1 Just a few cosmetic comments were received for v6 so this is mostly a rebase+ping. new in v7: - general rebase - sin

[Qemu-block] [PATCH v8 1/9] qapi: group BlockDeviceStats fields

Make the stat fields definition slightly more readable. Also reorder total_time_ns stats read-write-flush as done elsewhere. Cosmetic change only. Signed-off-by: Anton Nefedov Reviewed-by: Alberto Garcia Reviewed-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 26 +++---

[Qemu-block] [PATCH v8 3/9] block: add empty account cookie type

This adds some protection from accounting uninitialized cookie. That is, block_acct_failed/done without previous block_acct_start; in that case, cookie probably holds values from previous operation. (Note: it might also be uninitialized holding garbage value and there is still "< BLOCK_MAX_IOTYPE

[Qemu-block] [PATCH v14 1/1] qcow2: skip writing zero buffers to empty COW areas

If COW areas of the newly allocated clusters are zeroes on the backing image, efficient bdrv_write_zeroes(flags=BDRV_REQ_NO_FALLBACK) can be used on the whole cluster instead of writing explicit zero buffers later in perform_cow(). iotest 060: write to the discarded cluster does not trigger COW an

[Qemu-block] [PATCH v14 0/1] qcow2: cluster space preallocation

..apparently v13 ended up in a weird base64 that would not easily git-am. Resending. hi, this used to be a large 10-patches series, but now thanks to Kevin's BDRV_REQ_NO_FALLBACK series (http://lists.nongnu.org/archive/html/qemu-devel/2019-03/msg06389.html) the core block layer functionalit

[Qemu-block] [PATCH v1] [RFC] qcow2: add compression type feature

The patch adds some preparation parts for incompatible compression type feature into QCOW2 header that indicates that *all* compressed clusters must be (de)compressed using a certain compression type. It is implied that the compression type is set on the image creation and can be changed only late

Re: [Qemu-block] [PATCH v2] iotests: Filter 175's allocation information

On 16.05.19 15:06, Nir Soffer wrote: > On Thu, May 16, 2019 at 2:03 PM Max Reitz > wrote: > > On 13.05.19 17:52, Max Reitz wrote: > > It is possible for an empty file to take up blocks on a filesystem. > > Make iotest 175 take this into account. > > >

Re: [Qemu-block] [PATCH 3/4] iotests.py: Fix VM.run_job

On Wed 15 May 2019 10:15:02 PM CEST, Max Reitz wrote: > log() is in the current module, there is no need to prefix it. In fact, > doing so may make VM.run_job() unusable in tests that never use > iotests.log() themselves. > > Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH 2/4] iotests.py: Let assert_qmp() accept an array

On Wed 15 May 2019 10:15:01 PM CEST, Max Reitz wrote: > Sometimes we cannot tell which error message qemu will emit, and we do > not care. With this change, we can then just pass an array of all > possible messages to assert_qmp() and it will choose the right one. > > Signed-off-by: Max Reitz Re

Re: [Qemu-block] [PATCH v2] iotests: Filter 175's allocation information

On Thu, May 16, 2019 at 2:03 PM Max Reitz wrote: > On 13.05.19 17:52, Max Reitz wrote: > > It is possible for an empty file to take up blocks on a filesystem. > > Make iotest 175 take this into account. > > > > Reported-by: Thomas Huth > > Signed-off-by: Max Reitz > > --- > > v2: [Nir] > > - Us

[Qemu-block] [PATCH 1/2] qapi: support external bitmaps in block-dirty-bitmap-merge

Add new optional parameter making possible to merge bitmaps from different nodes. It is needed to maintain external snapshots during incremental backup chain history. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 13 ++--- block/dirty-bitmap.c | 9 ++--- blo

Re: [Qemu-block] [Qemu-devel] [PATCH 0/9] Assembly coroutine backend and x86 CET support

On Wed, 15 May 2019 at 10:48, Stefan Hajnoczi wrote: > > On Thu, May 09, 2019 at 02:44:39PM +0100, Peter Maydell wrote: > > On Sun, 5 May 2019 at 16:41, Alex Bennée wrote: > > > > > > > > > Paolo Bonzini writes: > > > > > > > *** BLURB HERE *** > > > > > > I assume there was going to be a bit mo

Re: [Qemu-block] [PATCH 1/4] block: Improve "Block node is read-only" message

On Wed 15 May 2019 10:15:00 PM CEST, Max Reitz wrote: > This message does not make any sense when it appears as the response to > making an R/W node read-only. We should detect that case and emit a > different message, then. > > Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia Berto

[Qemu-block] [PATCH 0/2] bitmaps: merge bitmaps from different nodes

Hi all! We need to copy bitmaps to new top node on external snapshot, to not break incremental backup chain. The only thing to do is to allow block-dirty-bitmap-merge to work with different nodes, here it is. Vladimir Sementsov-Ogievskiy (2): qapi: support external bitmaps in block-dirty-bitma

[Qemu-block] [PATCH 2/2] iotests: test external snapshot with bitmap copying

This test shows that external snapshots and incremental backups are friends. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/254 | 52 + tests/qemu-iotests/254.out | 53 ++ tests/qemu-iotests/group |

[Qemu-block] [PULL 09/37] vhost-user-blk: Add support to reconnect backend

From: Xie Yongji Since we now support the message VHOST_USER_GET_INFLIGHT_FD and VHOST_USER_SET_INFLIGHT_FD. The backend is able to restart safely because it can track inflight I/O in shared memory. This patch allows qemu to reconnect the backend after connection closed. Signed-off-by: Xie Yongj

[Qemu-block] [PULL 06/37] vhost-user-blk: Use started flag in vhost_user_blk_set_status()

From: Xie Yongji Use started flag in vhost_user_blk_set_status() to decide if starting vhost-user backend or not. Signed-off-by: Xie Yongji Signed-off-by: Zhang Yu Message-Id: <20190320112646.3712-4-xieyon...@baidu.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- h

[Qemu-block] [PULL 08/37] vhost-user-blk: Add return value for vhost_user_blk_start()

From: Xie Yongji Add a return value for vhost_user_blk_start() to check whether we start vhost-user backend successfully or not. Signed-off-by: Xie Yongji Signed-off-by: Zhang Yu Message-Id: <20190320112646.3712-6-xieyon...@baidu.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S.

[Qemu-block] [PULL 07/37] vhost-user-blk: Only start vhost-user backend with the first kick

From: Xie Yongji We should only start vhost-user backend at the first kick for virtio 1.0 transitional devices. Signed-off-by: Xie Yongji Signed-off-by: Zhang Yu Message-Id: <20190320112646.3712-5-xieyon...@baidu.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- hw/

Re: [Qemu-block] [PATCH v3 1/3] block: introducing 'bdrv_co_delete_file' interface

Hi John, On 5/13/19 4:47 PM, John Snow wrote: It looks like this one has gone un-noticed for a little while. On 3/26/19 5:17 PM, Daniel Henrique Barboza wrote: Adding to Block Drivers the capability of being able to clean up its created files can be useful in certain situations. For the LUKS d

Re: [Qemu-block] [PATCH v2] iotests: Filter 175's allocation information

On 13.05.19 17:52, Max Reitz wrote: > It is possible for an empty file to take up blocks on a filesystem. > Make iotest 175 take this into account. > > Reported-by: Thomas Huth > Signed-off-by: Max Reitz > --- > v2: [Nir] > - Use a function for filtering > - s/empty_blocks/extra_blocks/ > --- >

Re: [Qemu-block] [PATCH] [RFC] qcow2: add compression type feature

On 16.05.2019 13:40, Max Reitz wrote: > On 16.05.19 09:50, Denis Plotnikov wrote: >> >> >> On 29.04.2019 1:32, Max Reitz wrote: >>> On 05.02.19 10:08, Denis Plotnikov wrote: The patch adds some preparation parts for incompatible compression type feature into QCOW2 header that indicates

Re: [Qemu-block] [PATCH] [RFC] qcow2: add compression type feature

On 16.05.19 09:50, Denis Plotnikov wrote: > > > On 29.04.2019 1:32, Max Reitz wrote: >> On 05.02.19 10:08, Denis Plotnikov wrote: >>> The patch adds some preparation parts for incompatible compression type >>> feature into QCOW2 header that indicates that *all* compressed clusters >>> must be (de

Re: [Qemu-block] [PATCH v2] migration/dirty-bitmaps: change bitmap enumeration method

14.05.2019 23:19, John Snow wrote: > Shift from looking at every root BDS to *every* BDS. This will migrate > bitmaps that are attached to blockdev created nodes instead of just ones > attached to emulated storage devices. > > Note that this will not migrate anonymous or internal-use bitmaps, as >

Re: [Qemu-block] [PATCH] [RFC] qcow2: add compression type feature

On 29.04.2019 1:32, Max Reitz wrote: > On 05.02.19 10:08, Denis Plotnikov wrote: >> The patch adds some preparation parts for incompatible compression type >> feature into QCOW2 header that indicates that *all* compressed clusters >> must be (de)compressed using a certain compression type. >> >>