Re: [PATCH v2 1/6] block/block-copy: rename in-flight requests to tasks

2020-04-28 Thread Max Reitz
On 25.03.20 14:46, Vladimir Sementsov-Ogievskiy wrote: > We are going to use aio-task-pool API and extend in-flight request > structure to be a successor of AioTask, so rename things appropriately. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/block-copy.c | 99 ++

Re: [PATCH v2 2/6] block/block-copy: alloc task on each iteration

2020-04-28 Thread Max Reitz
On 25.03.20 14:46, Vladimir Sementsov-Ogievskiy wrote: > We are going to use aio-task-pool API, so tasks will be handled in > parallel. We need therefore separate allocated task on each iteration. > Introduce this logic now. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/block-cop

Re: [PATCH v20 3/4] qcow2: add zstd cluster compression

2020-04-28 Thread Denis Plotnikov
On 28.04.2020 09:16, Max Reitz wrote: On 27.04.20 21:26, Denis Plotnikov wrote: On 27.04.2020 15:35, Max Reitz wrote: On 21.04.20 10:11, Denis Plotnikov wrote: zstd significantly reduces cluster compression time. It provides better compression performance maintaining the same level of the

Re: [PATCH v2 3/6] block/block-copy: add state pointer to BlockCopyTask

2020-04-28 Thread Max Reitz
On 25.03.20 14:46, Vladimir Sementsov-Ogievskiy wrote: > We are going to use aio-task-pool API, so we'll need state pointer in > BlockCopyTask anyway. Add it now and use where possible. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/block-copy.c | 30 --

Re: [PATCH v2 5/6] block/block-copy: move block_copy_task_create down

2020-04-28 Thread Max Reitz
On 25.03.20 14:46, Vladimir Sementsov-Ogievskiy wrote: > Simple movement without any change. It's needed for the following > patch, as this function will need to use some staff which is currently *stuff > below it. Wouldn’t it be simpler to just declare block_copy_task_entry()? Max > Signed-of

Re: [PATCH v2 1/6] block/block-copy: rename in-flight requests to tasks

2020-04-28 Thread Vladimir Sementsov-Ogievskiy
28.04.2020 10:30, Max Reitz wrote: On 25.03.20 14:46, Vladimir Sementsov-Ogievskiy wrote: We are going to use aio-task-pool API and extend in-flight request structure to be a successor of AioTask, so rename things appropriately. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/block-cop

Re: [PATCH v2 5/6] block/block-copy: move block_copy_task_create down

2020-04-28 Thread Vladimir Sementsov-Ogievskiy
28.04.2020 12:06, Max Reitz wrote: On 25.03.20 14:46, Vladimir Sementsov-Ogievskiy wrote: Simple movement without any change. It's needed for the following patch, as this function will need to use some staff which is currently *stuff below it. Wouldn’t it be simpler to just declare block_c

Re: [PATCH v2 4/6] block/block-copy: move task size initial calculation to _task_create

2020-04-28 Thread Max Reitz
On 25.03.20 14:46, Vladimir Sementsov-Ogievskiy wrote: > Comment "Called only on full-dirty region" without corresponding > assertion is a very unsafe thing. Not sure whether it’s that unsafe for a static function with a single caller, but, well. > Adding assertion means call > bdrv_dirty_bitmap_

Re: [PATCH v2 4/6] block/block-copy: move task size initial calculation to _task_create

2020-04-28 Thread Vladimir Sementsov-Ogievskiy
28.04.2020 11:52, Max Reitz wrote: On 25.03.20 14:46, Vladimir Sementsov-Ogievskiy wrote: Comment "Called only on full-dirty region" without corresponding assertion is a very unsafe thing. Not sure whether it’s that unsafe for a static function with a single caller, but, well. Adding asserti

backing chain & block status & filters

2020-04-28 Thread Vladimir Sementsov-Ogievskiy
Hi! I wanted to resend my "[PATCH 0/4] fix & merge block_status_above and is_allocated_above", and returned to all the inconsistencies about block-status. I keep in mind Max's series about child-access functions, and Andrey's work about using COR filter in block-stream, which depends on Max's

Re: [PATCH v2 6/6] block/block-copy: use aio-task-pool API

2020-04-28 Thread Max Reitz
On 25.03.20 14:46, Vladimir Sementsov-Ogievskiy wrote: > Run block_copy iterations in parallel in aio tasks. > > Changes: > - BlockCopyTask becomes aio task structure. Add zeroes field to pass > it to block_copy_do_copy > - add call state - it's a state of one call of block_copy(), shared

Re: [PATCH v2 5/6] block/block-copy: move block_copy_task_create down

2020-04-28 Thread Max Reitz
On 28.04.20 11:17, Vladimir Sementsov-Ogievskiy wrote: > 28.04.2020 12:06, Max Reitz wrote: >> On 25.03.20 14:46, Vladimir Sementsov-Ogievskiy wrote: >>> Simple movement without any change. It's needed for the following >>> patch, as this function will need to use some staff which is currently >> >

Re: [PATCH v20 3/4] qcow2: add zstd cluster compression

2020-04-28 Thread Max Reitz
On 28.04.20 09:23, Denis Plotnikov wrote: > > > On 28.04.2020 09:16, Max Reitz wrote: >> On 27.04.20 21:26, Denis Plotnikov wrote: >>> >>> On 27.04.2020 15:35, Max Reitz wrote: On 21.04.20 10:11, Denis Plotnikov wrote: > zstd significantly reduces cluster compression time. > It provi

Re: backing chain & block status & filters

2020-04-28 Thread Max Reitz
On 28.04.20 10:55, Vladimir Sementsov-Ogievskiy wrote: > Hi! > > I wanted to resend my "[PATCH 0/4] fix & merge block_status_above and > is_allocated_above", and returned to all the inconsistencies about > block-status. I keep in mind Max's series about child-access functions, > and Andrey's work

Re: fdatasync semantics and block device backup

2020-04-28 Thread Kevin Wolf
Hi Bryan, first of all, for your next question, please don't reply to a message in an unrelated thread, but start a new email. This will give you a lot more visibility because people generally use a threaded email view and will decide whether to read an email or not depending on whether the topic

Re: backing chain & block status & filters

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 13:08 hat Max Reitz geschrieben: > On 28.04.20 10:55, Vladimir Sementsov-Ogievskiy wrote: > > Hi! > > > > I wanted to resend my "[PATCH 0/4] fix & merge block_status_above and > > is_allocated_above", and returned to all the inconsistencies about > > block-status. I keep in mind M

Re: [PATCH v20 4/4] iotests: 287: add qcow2 compression type test

2020-04-28 Thread Denis Plotnikov
On 27.04.2020 16:29, Max Reitz wrote: On 21.04.20 10:11, Denis Plotnikov wrote: The test checks fulfilling qcow2 requirements for the compression type feature and zstd compression type operability. Signed-off-by: Denis Plotnikov --- tests/qemu-iotests/287 | 146 +++

Re: [PATCH v10 00/14] iotests: use python logging

2020-04-28 Thread Max Reitz
On 31.03.20 02:00, John Snow wrote: > This series uses python logging to enable output conditionally on > iotests.log(). We unify an initialization call (which also enables > debugging output for those tests with -d) and then make the switch > inside of iotests. > > It will help alleviate the need

Re: [PATCH v10 00/14] iotests: use python logging

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 13:46 hat Max Reitz geschrieben: > On 31.03.20 02:00, John Snow wrote: > > This series uses python logging to enable output conditionally on > > iotests.log(). We unify an initialization call (which also enables > > debugging output for those tests with -d) and then make the switch

Re: [PATCH v3 1/3] block: Add blk_new_with_bs() helper

2020-04-28 Thread Eric Blake
On 4/28/20 1:34 AM, Max Reitz wrote: block_crypto_co_create_generic(BlockDriverState *bs,     PreallocMode prealloc,     Error **errp)   { -    int ret; +    int ret = -EPERM; I’m not sure I’m a fan of this, bec

Re: [PATCH v20 4/4] iotests: 287: add qcow2 compression type test

2020-04-28 Thread Max Reitz
On 28.04.20 13:41, Denis Plotnikov wrote: > > > On 27.04.2020 16:29, Max Reitz wrote: >> On 21.04.20 10:11, Denis Plotnikov wrote: >>> The test checks fulfilling qcow2 requirements for the compression >>> type feature and zstd compression type operability. >>> >>> Signed-off-by: Denis Plotnikov

Re: [PATCH v3 0/3] qcow2: Allow resize of images with internal snapshots

2020-04-28 Thread Max Reitz
On 24.04.20 21:09, Eric Blake wrote: > In v3: > - patch 1: fix error returns [patchew, Max], R-b dropped > - patch 2,3: unchanged, so add R-b > > Eric Blake (3): > block: Add blk_new_with_bs() helper > qcow2: Allow resize of images with internal snapshots > qcow2: Tweak comment about bitmaps

Re: [PATCH v20 4/4] iotests: 287: add qcow2 compression type test

2020-04-28 Thread Eric Blake
On 4/28/20 7:55 AM, Max Reitz wrote: +# This tests qocw2-specific low-level functionality +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux This test doesn’t work with compat=0.10 (because we can’t store a non-default compression type there) or data_file (because those don’t sup

Re: [PATCH v2 02/14] qcrypto/luks: implement encryption key management

2020-04-28 Thread Daniel P . Berrangé
On Sun, Mar 08, 2020 at 05:18:51PM +0200, Maxim Levitsky wrote: > Next few patches will expose that functionality > to the user. > > Signed-off-by: Maxim Levitsky > --- > crypto/block-luks.c | 398 +++- > qapi/crypto.json| 61 ++- > 2 files change

[PATCH 2/4] block: Use bdrv_make_empty() where possible

2020-04-28 Thread Max Reitz
Signed-off-by: Max Reitz --- block/replication.c | 6 ++ block/vvfat.c | 4 +--- 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/block/replication.c b/block/replication.c index da013c2041..cc6a40d577 100644 --- a/block/replication.c +++ b/block/replication.c @@ -331,9 +33

[PATCH 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

2020-04-28 Thread Max Reitz
Branch: https://github.com/XanClic/qemu.git fix-bdrv_make_empty-v1 Branch: https://git.xanclic.moe/XanClic/qemu.git fix-bdrv_make_empty-v1 Hi, Right now, there is no centralized bdrv_make_empty() function. Not only is it bad style to call BlockDriver methods directly, it is also wrong, unless th

[PATCH 3/4] block: Add blk_make_empty()

2020-04-28 Thread Max Reitz
Two callers of BlockDriver.bdrv_make_empty() remain that should not call this method directly. Both do not have access to a BdrvChild, but they can use a BlockBackend, so we add this function that lets them use it. Signed-off-by: Max Reitz --- include/sysemu/block-backend.h | 2 ++ block/block-

[PATCH 1/4] block: Add bdrv_make_empty()

2020-04-28 Thread Max Reitz
Right now, all users of bdrv_make_empty() call the BlockDriver method directly. That is not only bad style, it is also wrong, unless the caller has a BdrvChild with a WRITE permission. Introduce bdrv_make_empty() that verifies that it does. Signed-off-by: Max Reitz --- include/block/block.h |

[PATCH 4/4] block: Use blk_make_empty() after commits

2020-04-28 Thread Max Reitz
bdrv_commit() already has a BlockBackend pointing to the BDS that we want to empty, it just has the wrong permissions. qemu-img commit has no BlockBackend pointing to the old backing file yet, but introducing one is simple. After this commit, bdrv_make_empty() is the only remaining caller of Bloc

Re: [PATCH v20 4/4] iotests: 287: add qcow2 compression type test

2020-04-28 Thread Denis Plotnikov
On 28.04.2020 16:01, Eric Blake wrote: On 4/28/20 7:55 AM, Max Reitz wrote: +# This tests qocw2-specific low-level functionality +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux This test doesn’t work with compat=0.10 (because we can’t store a non-default compression type t

[PATCH v21 4/4] iotests: 287: add qcow2 compression type test

2020-04-28 Thread Denis Plotnikov
The test checks fulfilling qcow2 requirements for the compression type feature and zstd compression type operability. Signed-off-by: Denis Plotnikov Reviewed-by: Vladimir Sementsov-Ogievskiy Tested-by: Vladimir Sementsov-Ogievskiy --- slirp | 2 +- tests/qemu-iotests/287

[PATCH v21 2/4] qcow2: rework the cluster compression routine

2020-04-28 Thread Denis Plotnikov
The patch enables processing the image compression type defined for the image and chooses an appropriate method for image clusters (de)compression. Signed-off-by: Denis Plotnikov Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz --- block/qcow2-threa

[PATCH v21 3/4] qcow2: add zstd cluster compression

2020-04-28 Thread Denis Plotnikov
zstd significantly reduces cluster compression time. It provides better compression performance maintaining the same level of the compression ratio in comparison with zlib, which, at the moment, is the only compression method available. The performance test results: Test compresses and decompresse

[PATCH v21 0/4] implement zstd cluster compression method

2020-04-28 Thread Denis Plotnikov
v21: 03: * remove the loop on compression [Max] * use designated initializers [Max] 04: * don't erase user's options [Max] * use _rm_test_img [Max] * add unsupported qcow2 options [Max] v20: 04: fix a number of flaws [Vladimir] * don't use $RAND_F

[PATCH v21 1/4] qcow2: introduce compression type feature

2020-04-28 Thread Denis Plotnikov
The patch adds some preparation parts for incompatible compression type feature to qcow2 allowing the use different compression methods for image clusters (de)compressing. It is implied that the compression type is set on the image creation and can be changed only later by image conversion, thus c

Re: [PATCH 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

2020-04-28 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200428132629.796753-1-mre...@redhat.com/ Hi, This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #!/bin/bash expor

Re: [PATCH 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

2020-04-28 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200428132629.796753-1-mre...@redhat.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN ===

Re: [PATCH 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

2020-04-28 Thread Eric Blake
On 4/28/20 8:26 AM, Max Reitz wrote: Branch: https://github.com/XanClic/qemu.git fix-bdrv_make_empty-v1 Branch: https://git.xanclic.moe/XanClic/qemu.git fix-bdrv_make_empty-v1 Hi, Right now, there is no centralized bdrv_make_empty() function. Not only is it bad style to call BlockDriver method

Re: [PATCH 2/4] block: Use bdrv_make_empty() where possible

2020-04-28 Thread Eric Blake
On 4/28/20 8:26 AM, Max Reitz wrote: Signed-off-by: Max Reitz --- block/replication.c | 6 ++ block/vvfat.c | 4 +--- 2 files changed, 3 insertions(+), 7 deletions(-) Yes, definitely nicer :) May have some obvious fallout to add a 0 flag parameter, per my request on 1/4, but t

RE: fdatasync semantics and block device backup

2020-04-28 Thread Bryan S Rosenburg
Kevin Wolf wrote on 04/28/2020 07:11:24 AM: > > Am 27.04.2020 um 21:49 hat Bryan S Rosenburg geschrieben: > > Blockdev community, > > > > Our group would like to write block device backups directly to an object > > store, using an interface such as s3fs or rclone-mount. We've run into > > pr

Re: [PATCH 1/4] block: Add bdrv_make_empty()

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 15:53 hat Eric Blake geschrieben: > On 4/28/20 8:26 AM, Max Reitz wrote: > > Right now, all users of bdrv_make_empty() call the BlockDriver method > > directly. That is not only bad style, it is also wrong, unless the > > caller has a BdrvChild with a WRITE permission. > > > > In

Re: [PATCH 1/4] block: Add bdrv_make_empty()

2020-04-28 Thread Eric Blake
On 4/28/20 8:26 AM, Max Reitz wrote: Right now, all users of bdrv_make_empty() call the BlockDriver method directly. That is not only bad style, it is also wrong, unless the caller has a BdrvChild with a WRITE permission. Introduce bdrv_make_empty() that verifies that it does. Signed-off-by: M

Re: [PATCH 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

2020-04-28 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200428132629.796753-1-mre...@redhat.com/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #

Re: [PATCH 4/4] block: Use blk_make_empty() after commits

2020-04-28 Thread Eric Blake
On 4/28/20 8:26 AM, Max Reitz wrote: bdrv_commit() already has a BlockBackend pointing to the BDS that we want to empty, it just has the wrong permissions. qemu-img commit has no BlockBackend pointing to the old backing file yet, but introducing one is simple. After this commit, bdrv_make_empty

Re: fdatasync semantics and block device backup

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 15:58 hat Bryan S Rosenburg geschrieben: > Kevin Wolf wrote on 04/28/2020 07:11:24 AM: > > I think "don't do that" is a good answer actually. > > > > You may want to put an NBD indirection between QEMU and your object > > store, so that the close() syscall will just block a qemu-

Re: [PATCH 1/4] block: Add bdrv_make_empty()

2020-04-28 Thread Eric Blake
On 4/28/20 9:01 AM, Kevin Wolf wrote: Can we please fix this to take a flags parameter? I want to make it easier for callers to request BDRV_REQ_NO_FALLBACK for distinguishing between callers where the image must be made empty (read as all zeroes) regardless of time spent, vs. made empty quickl

Re: [PATCH 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

2020-04-28 Thread Eric Blake
On 4/28/20 8:49 AM, Eric Blake wrote: On 4/28/20 8:26 AM, Max Reitz wrote: Branch: https://github.com/XanClic/qemu.git fix-bdrv_make_empty-v1 Branch: https://git.xanclic.moe/XanClic/qemu.git fix-bdrv_make_empty-v1 Hi, Right now, there is no centralized bdrv_make_empty() function.  Not only is

Re: [PATCH 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

2020-04-28 Thread Eric Blake
On 4/28/20 8:43 AM, no-re...@patchew.org wrote: Patchew URL: https://patchew.org/QEMU/20200428132629.796753-1-mre...@redhat.com/ SIGNpc-bios/optionrom/pvh.bin /tmp/qemu-test/src/qemu-img.c: In function 'img_commit': /tmp/qemu-test/src/qemu-img.c:1071:9: error: implicit declaration of

Re: [PATCH 1/4] block: Add bdrv_make_empty()

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 16:07 hat Eric Blake geschrieben: > On 4/28/20 9:01 AM, Kevin Wolf wrote: > > > > Can we please fix this to take a flags parameter? I want to make it > > > easier > > > for callers to request BDRV_REQ_NO_FALLBACK for distinguishing between > > > callers where the image must be m

Re: [PATCH 1/4] block: Add bdrv_make_empty()

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 15:26 hat Max Reitz geschrieben: > Right now, all users of bdrv_make_empty() call the BlockDriver method > directly. That is not only bad style, it is also wrong, unless the > caller has a BdrvChild with a WRITE permission. > > Introduce bdrv_make_empty() that verifies that it do

Re: [PATCH 3/4] block: Add blk_make_empty()

2020-04-28 Thread Eric Blake
On 4/28/20 8:26 AM, Max Reitz wrote: Two callers of BlockDriver.bdrv_make_empty() remain that should not call this method directly. Both do not have access to a BdrvChild, but they can use a BlockBackend, so we add this function that lets them use it. Signed-off-by: Max Reitz --- include/sys

Re: [PATCH 1/4] block: Add bdrv_make_empty()

2020-04-28 Thread Eric Blake
On 4/28/20 9:16 AM, Kevin Wolf wrote: Yes. Although now I'm wondering if the two should remain separate or should just be a single driver callback where flags can include BDRV_REQ_ZERO_WRITE to distinguish whether exposing the backing file vs. reading as all zeroes is intended, or if that is m

Re: [PATCH 3/4] block: Add blk_make_empty()

2020-04-28 Thread Eric Blake
On 4/28/20 8:55 AM, Eric Blake wrote: +++ b/include/sysemu/block-backend.h @@ -266,4 +266,6 @@ int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in,   const BdrvChild *blk_root(BlockBackend *blk); +int blk_make_empty(BlockBackend *blk, Error **errp); + Again, a flag parame

Re: [PATCH v21 0/4] implement zstd cluster compression method

2020-04-28 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200428133407.10657-1-dplotni...@virtuozzo.com/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [PATCH v21 0/4] implement zstd cluster compression method Message-id: 20200428133407.10657-1-dplotni

Re: [PATCH 3/4] block: Add blk_make_empty()

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 15:26 hat Max Reitz geschrieben: > Two callers of BlockDriver.bdrv_make_empty() remain that should not call > this method directly. Both do not have access to a BdrvChild, but they > can use a BlockBackend, so we add this function that lets them use it. > > Signed-off-by: Max Rei

Re: backing chain & block status & filters

2020-04-28 Thread Vladimir Sementsov-Ogievskiy
28.04.2020 14:08, Max Reitz wrote: On 28.04.20 10:55, Vladimir Sementsov-Ogievskiy wrote: Hi! I wanted to resend my "[PATCH 0/4] fix & merge block_status_above and is_allocated_above", and returned to all the inconsistencies about block-status. I keep in mind Max's series about child-access fun

Re: [PATCH 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

2020-04-28 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200428132629.796753-1-mre...@redhat.com/ Hi, This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #!/bin/bash expor

Re: [PATCH 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

2020-04-28 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200428132629.796753-1-mre...@redhat.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN ===

Re: [PATCH 0/4] block: Do not call BlockDriver.bdrv_make_empty() directly

2020-04-28 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200428132629.796753-1-mre...@redhat.com/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #

Re: [PATCH 2/4] block: Use bdrv_make_empty() where possible

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 15:26 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf

Re: [PATCH 4/4] block: Use blk_make_empty() after commits

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 15:26 hat Max Reitz geschrieben: > bdrv_commit() already has a BlockBackend pointing to the BDS that we > want to empty, it just has the wrong permissions. > > qemu-img commit has no BlockBackend pointing to the old backing file > yet, but introducing one is simple. > > After thi

Re: [PATCH v2 04/14] block/amend: separate amend and create options for qemu-img

2020-04-28 Thread Daniel P . Berrangé
On Sun, Mar 08, 2020 at 05:18:53PM +0200, Maxim Levitsky wrote: > Some options are only useful for creation > (or hard to be amended, like cluster size for qcow2), while some other > options are only useful for amend, like upcoming keyslot management > options for luks > > Since currently only qco

Re: backing chain & block status & filters

2020-04-28 Thread Vladimir Sementsov-Ogievskiy
28.04.2020 14:28, Kevin Wolf wrote: Am 28.04.2020 um 13:08 hat Max Reitz geschrieben: On 28.04.20 10:55, Vladimir Sementsov-Ogievskiy wrote: Hi! I wanted to resend my "[PATCH 0/4] fix & merge block_status_above and is_allocated_above", and returned to all the inconsistencies about block-status

Re: [PATCH v2 04/14] block/amend: separate amend and create options for qemu-img

2020-04-28 Thread Daniel P . Berrangé
On Tue, Apr 28, 2020 at 04:03:33PM +0100, Daniel P. Berrangé wrote: > On Sun, Mar 08, 2020 at 05:18:53PM +0200, Maxim Levitsky wrote: > > Some options are only useful for creation > > (or hard to be amended, like cluster size for qcow2), while some other > > options are only useful for amend, like

Re: [PATCH v2 05/14] block/amend: refactor qcow2 amend options

2020-04-28 Thread Daniel P . Berrangé
On Sun, Mar 08, 2020 at 05:18:54PM +0200, Maxim Levitsky wrote: > Some qcow2 create options can't be used for amend. > Remove them from the qcow2 create options and add generic logic to detect > such options in qemu-img > > Signed-off-by: Maxim Levitsky > --- > block/qcow2.c | 108 +

Re: [PATCH v21 0/4] implement zstd cluster compression method

2020-04-28 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200428133407.10657-1-dplotni...@virtuozzo.com/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [PATCH v21 0/4] implement zstd cluster compression method Message-id: 20200428133407.10657-1-dplotni

Re: [PATCH v2 07/14] block/crypto: implement the encryption key management

2020-04-28 Thread Daniel P . Berrangé
On Sun, Mar 08, 2020 at 05:18:56PM +0200, Maxim Levitsky wrote: > This implements the encryption key management using the generic code in > qcrypto layer and exposes it to the user via qemu-img > > This code adds another 'write_func' because the initialization > write_func works directly on the un

Re: [PATCH v2 08/14] block/qcow2: extend qemu-img amend interface with crypto options

2020-04-28 Thread Daniel P . Berrangé
On Sun, Mar 08, 2020 at 05:18:57PM +0200, Maxim Levitsky wrote: > Now that we have all the infrastructure in place, > wire it in the qcow2 driver and expose this to the user. > > Signed-off-by: Maxim Levitsky > --- > block/qcow2.c | 80 -- > tests

Re: [PATCH v2 10/14] iotests: qemu-img tests for luks key management

2020-04-28 Thread Daniel P . Berrangé
On Sun, Mar 08, 2020 at 05:18:59PM +0200, Maxim Levitsky wrote: > This commit adds two tests, which test the new amend interface > of both luks raw images and qcow2 luks encrypted images. > > Signed-off-by: Maxim Levitsky > --- > tests/qemu-iotests/300 | 207 +

Re: [PATCH v2 09/14] iotests: filter few more luks specific create options

2020-04-28 Thread Daniel P . Berrangé
On Sun, Mar 08, 2020 at 05:18:58PM +0200, Maxim Levitsky wrote: > This allows more tests to be able to have same output on both qcow2 luks > encrypted images > and raw luks images > > Signed-off-by: Maxim Levitsky > --- > tests/qemu-iotests/087.out | 6 +++--- > tests/qemu-iotests/134.out

Re: backing chain & block status & filters

2020-04-28 Thread Eric Blake
On 4/28/20 10:13 AM, Vladimir Sementsov-Ogievskiy wrote: Hm.  I could imagine that there are formats that have non-zero holes (e.g. 0xff or just garbage).  It would be a bit wrong for them to return ZERO or DATA then. But OTOH we don’t care about such cases, do we?  We need to know whether rang

Re: [PATCH v2 14/14] iotests: add tests for blockdev-amend

2020-04-28 Thread Daniel P . Berrangé
On Sun, Mar 08, 2020 at 05:19:03PM +0200, Maxim Levitsky wrote: > This commit adds two tests that cover the > new blockdev-amend functionality of luks and qcow2 driver > > Signed-off-by: Maxim Levitsky > --- > tests/qemu-iotests/302 | 278 + > tests/qemu-i

Re: [PATCH v2 11/14] block/core: add generic infrastructure for x-blockdev-amend qmp command

2020-04-28 Thread Daniel P . Berrangé
On Sun, Mar 08, 2020 at 05:19:00PM +0200, Maxim Levitsky wrote: > blockdev-amend will be used similiar to blockdev-create > to allow on the fly changes of the structure of the format based block > devices. > > Current plan is to first support encryption keyslot management for luks > based formats

Re: [PATCH v7 04/10] qcow2: Support BDRV_REQ_ZERO_WRITE for truncate

2020-04-28 Thread Eric Blake
On 4/24/20 7:54 AM, Kevin Wolf wrote: If BDRV_REQ_ZERO_WRITE is set and we're extending the image, calling qcow2_cluster_zeroize() with flags=0 does the right thing: It doesn't undo any previous preallocation, but just adds the zero flag to all relevant L2 entries. If an external data file is in

Re: backing chain & block status & filters

2020-04-28 Thread Vladimir Sementsov-Ogievskiy
28.04.2020 19:18, Eric Blake wrote: On 4/28/20 10:13 AM, Vladimir Sementsov-Ogievskiy wrote: Hm.  I could imagine that there are formats that have non-zero holes (e.g. 0xff or just garbage).  It would be a bit wrong for them to return ZERO or DATA then. But OTOH we don’t care about such cases,

Re: [PATCH v10 00/14] iotests: use python logging

2020-04-28 Thread John Snow
On 4/28/20 8:21 AM, Kevin Wolf wrote: > Am 28.04.2020 um 13:46 hat Max Reitz geschrieben: >> On 31.03.20 02:00, John Snow wrote: >>> This series uses python logging to enable output conditionally on >>> iotests.log(). We unify an initialization call (which also enables >>> debugging output for t

Re: [PATCH v3 0/3] qcow2: Allow resize of images with internal snapshots

2020-04-28 Thread Eric Blake
On 4/28/20 7:59 AM, Max Reitz wrote: On 24.04.20 21:09, Eric Blake wrote: In v3: - patch 1: fix error returns [patchew, Max], R-b dropped - patch 2,3: unchanged, so add R-b Eric Blake (3): block: Add blk_new_with_bs() helper qcow2: Allow resize of images with internal snapshots qcow2:

Re: backing chain & block status & filters

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 18:46 hat Vladimir Sementsov-Ogievskiy geschrieben: > 28.04.2020 19:18, Eric Blake wrote: > > On 4/28/20 10:13 AM, Vladimir Sementsov-Ogievskiy wrote: > > > > > > > Hm.  I could imagine that there are formats that have non-zero holes > > > > > (e.g. 0xff or just garbage).  It woul

Re: [PATCH v7 04/10] qcow2: Support BDRV_REQ_ZERO_WRITE for truncate

2020-04-28 Thread Kevin Wolf
Am 28.04.2020 um 18:28 hat Eric Blake geschrieben: > On 4/24/20 7:54 AM, Kevin Wolf wrote: > > If BDRV_REQ_ZERO_WRITE is set and we're extending the image, calling > > qcow2_cluster_zeroize() with flags=0 does the right thing: It doesn't > > undo any previous preallocation, but just adds the zero f

Re: [PATCH v7 04/10] qcow2: Support BDRV_REQ_ZERO_WRITE for truncate

2020-04-28 Thread Eric Blake
On 4/28/20 1:45 PM, Kevin Wolf wrote: Am 28.04.2020 um 18:28 hat Eric Blake geschrieben: On 4/24/20 7:54 AM, Kevin Wolf wrote: If BDRV_REQ_ZERO_WRITE is set and we're extending the image, calling qcow2_cluster_zeroize() with flags=0 does the right thing: It doesn't undo any previous preallocati

[PATCH v4 0/3] qcow2: Allow resize of images with internal snapshots

2020-04-28 Thread Eric Blake
Re-posting this to make Max' life easier when rebasing on top of Kevin's work. Based-on: <20200424125448.63318-1-kw...@redhat.com> [PATCH v7 00/10] block: Fix resize (extending) of short overlays In v4: - patch 1: fold in Max's touchups to my v3 - patch 1: resolve merge conflict on top of Kevin's

[PATCH v4 1/3] block: Add blk_new_with_bs() helper

2020-04-28 Thread Eric Blake
There are several callers that need to create a new block backend from an existing BDS; make the task slightly easier with a common helper routine. Suggested-by: Max Reitz Signed-off-by: Eric Blake Message-Id: <20200424190903.522087-2-ebl...@redhat.com> [mreitz: Set @ret only in error paths, see

[PATCH v4 3/3] qcow2: Tweak comment about bitmaps vs. resize

2020-04-28 Thread Eric Blake
Our comment did not actually match the code. Rewrite the comment to be less sensitive to any future changes to qcow2-bitmap.c that might implement scenarios that we currently reject. Signed-off-by: Eric Blake Reviewed-by: Max Reitz --- block/qcow2.c | 2 +- 1 file changed, 1 insertion(+), 1 de

[PATCH v4 2/3] qcow2: Allow resize of images with internal snapshots

2020-04-28 Thread Eric Blake
We originally refused to allow resize of images with internal snapshots because the v2 image format did not require the tracking of snapshot size, making it impossible to safely revert to a snapshot with a different size than the current view of the image. But the snapshot size tracking was rectif

Re: backing chain & block status & filters

2020-04-28 Thread Vladimir Sementsov-Ogievskiy
28.04.2020 19:46, Vladimir Sementsov-Ogievskiy wrote: 28.04.2020 19:18, Eric Blake wrote: On 4/28/20 10:13 AM, Vladimir Sementsov-Ogievskiy wrote: Hm.  I could imagine that there are formats that have non-zero holes (e.g. 0xff or just garbage).  It would be a bit wrong for them to return ZERO

[PATCH v22 4/4] iotests: 287: add qcow2 compression type test

2020-04-28 Thread Denis Plotnikov
The test checks fulfilling qcow2 requirements for the compression type feature and zstd compression type operability. Signed-off-by: Denis Plotnikov Reviewed-by: Vladimir Sementsov-Ogievskiy Tested-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/287 | 152 ++

[PATCH v22 0/4] implement zstd cluster compression method

2020-04-28 Thread Denis Plotnikov
v22: 03: remove assignemnt in if condition v21: 03: * remove the loop on compression [Max] * use designated initializers [Max] 04: * don't erase user's options [Max] * use _rm_test_img [Max] * add unsupported qcow2 options [Max] v20: 04: fix a number

[PATCH v22 3/4] qcow2: add zstd cluster compression

2020-04-28 Thread Denis Plotnikov
zstd significantly reduces cluster compression time. It provides better compression performance maintaining the same level of the compression ratio in comparison with zlib, which, at the moment, is the only compression method available. The performance test results: Test compresses and decompresse

[PATCH v22 2/4] qcow2: rework the cluster compression routine

2020-04-28 Thread Denis Plotnikov
The patch enables processing the image compression type defined for the image and chooses an appropriate method for image clusters (de)compression. Signed-off-by: Denis Plotnikov Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia --- block/qcow2-threads.c | 71 ++

[PATCH v22 1/4] qcow2: introduce compression type feature

2020-04-28 Thread Denis Plotnikov
The patch adds some preparation parts for incompatible compression type feature to qcow2 allowing the use different compression methods for image clusters (de)compressing. It is implied that the compression type is set on the image creation and can be changed only later by image conversion, thus c

[PATCH 2/9] file-win32: Support BDRV_REQ_ZERO_WRITE for truncate

2020-04-28 Thread Eric Blake
When using bdrv_file, .bdrv_has_zero_init_truncate always returns 1; therefore, we can behave just like file-posix, and always implement BDRV_REQ_ZERO_WRITE by ignoring it since the OS gives it to us for free (note that file-posix.c had to use an 'if' because it shared code between regular files an

[PATCH 3/9] nfs: Support BDRV_REQ_ZERO_WRITE for truncate

2020-04-28 Thread Eric Blake
Our .bdrv_has_zero_init_truncate returns 1 if we detect that the OS always 0-fills; we can use that same knowledge to implement BDRV_REQ_ZERO_WRITE by ignoring it when the OS gives it to us for free. Signed-off-by: Eric Blake --- block/nfs.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a

[PATCH 0/9] More truncate improvements

2020-04-28 Thread Eric Blake
Based-on: <20200424125448.63318-1-kw...@redhat.com> [PATCH v7 00/10] block: Fix resize (extending) of short overlays After reviewing Kevin's work, I questioned if we had a redundancy with bdrv_has_zero_init_truncate. It turns out we do, and this is the result. Patch 1 has been previously posted

[PATCH 1/9] gluster: Drop useless has_zero_init callback

2020-04-28 Thread Eric Blake
block.c already defaults to 0 if we don't provide a callback; there's no need to write a callback that always fails. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia --- block/gluster.c | 14 -- 1 file changed, 14 deletions(-) diff --

[PATCH 6/9] ssh: Support BDRV_REQ_ZERO_WRITE for truncate

2020-04-28 Thread Eric Blake
Our .bdrv_has_zero_init_truncate can detect when the remote side always zero fills; we can reuse that same knowledge to implement BDRV_REQ_ZERO_WRITE by ignoring it when the server gives it to us for free. Signed-off-by: Eric Blake --- block/ssh.c | 4 1 file changed, 4 insertions(+) diff

[PATCH 4/9] rbd: Support BDRV_REQ_ZERO_WRITE for truncate

2020-04-28 Thread Eric Blake
Our .bdrv_has_zero_init_truncate always returns 1 because rbd always 0-fills; we can use that same knowledge to implement BDRV_REQ_ZERO_WRITE by ignoring it. Signed-off-by: Eric Blake --- block/rbd.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/rbd.c b/block/rbd.c index f2d52091c

[PATCH 5/9] sheepdog: Support BDRV_REQ_ZERO_WRITE for truncate

2020-04-28 Thread Eric Blake
Our .bdrv_has_zero_init_truncate always returns 1 because sheepdog always 0-fills; we can use that same knowledge to implement BDRV_REQ_ZERO_WRITE by ignoring it. Signed-off-by: Eric Blake --- block/sheepdog.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/sheepdog.c b/block/sheepdog.

[PATCH 9/9] block: Drop unused .bdrv_has_zero_init_truncate

2020-04-28 Thread Eric Blake
Now that there are no clients of bdrv_has_zero_init_truncate, none of the drivers need to worry about providing it. What's more, this eliminates a source of some confusion: a literal reading of the documentation as written in ceaca56f and implemented in commit 1dcaf527 claims that a driver which r

[PATCH 7/9] parallels: Rework truncation logic

2020-04-28 Thread Eric Blake
The parallels driver tries to use truncation for image growth, but can only do so when reads are guaranteed as zero. Now that we have a way to request zero contents from truncation, we can defer the decision to actual allocation attempts rather than up front, reducing the number of places that sti

[PATCH 8/9] vhdx: Rework truncation logic

2020-04-28 Thread Eric Blake
The vhdx driver uses truncation for image growth, with a special case for blocks that already read as zero but which are only being partially written. But with a bit of rearranging, it's just as easy to defer the decision on whether truncation resulted in zeroes to the actual allocation attempt, r

  1   2   >