Re: [Qemu-block] [PATCH v12 8/9] qcow2: Set the default cache-clean-interval to 10 minutes

2018-09-27 Thread Leonid Bloch
On September 27, 2018 5:53:34 PM CEST, Eric Blake wrote: >On 9/26/18 11:04 AM, Leonid Bloch wrote: >> The default cache-clean-interval is set to 10 minutes, in order to >lower >> the overhead of the qcow2 caches (before the default was 0, i.e. >> disabled). >> >> * For non-Linux platforms the

Re: [Qemu-block] [Qemu-devel] [PATCH] nbd: Don't take address of fields in packed structs

2018-09-27 Thread Philippe Mathieu-Daudé
On 9/27/18 7:16 PM, Eric Blake wrote: > On 9/27/18 11:42 AM, Peter Maydell wrote: >> Taking the address of a field in a packed struct is a bad idea, because >> it might not be actually aligned enough for that pointer type (and >> thus cause a crash on dereference on some host architectures). Newer

Re: [Qemu-block] [PATCH] nbd: Don't take address of fields in packed structs

2018-09-27 Thread Peter Maydell
On 27 September 2018 at 19:16, Eric Blake wrote: > Okay, I am also in favor of the complete conversion. Want me to squash in > the remaining 3 spots as part of queuing my patch, so you don't have to send > a v2? Yes, please. thanks -- PMM

Re: [Qemu-block] [PATCH 4/7] qcow2: async scheme for qcow2_co_preadv

2018-09-27 Thread Max Reitz
On 07.08.18 19:43, Vladimir Sementsov-Ogievskiy wrote: > Start several async requests instead of read chunk by chunk. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/qcow2.c | 208 > -- > 1 file changed, 204 insertions(+), 4

Re: [Qemu-block] [PATCH] nbd: Don't take address of fields in packed structs

2018-09-27 Thread Eric Blake
On 9/27/18 1:03 PM, Peter Maydell wrote: I'm a bit confused. After applying your patch (and rebasing it to my pending pull request), I still found instances of be16_to_cpus() and others. Were you only flipping instances that were members of a packed struct, while leaving other instances

Re: [Qemu-block] [PATCH] nbd: Don't take address of fields in packed structs

2018-09-27 Thread Peter Maydell
On 27 September 2018 at 18:30, Eric Blake wrote: > On 9/27/18 11:42 AM, Peter Maydell wrote: >> >> Taking the address of a field in a packed struct is a bad idea, because >> it might not be actually aligned enough for that pointer type (and >> thus cause a crash on dereference on some host

Re: [Qemu-block] [PATCH 3/7] qcow2: split out reading normal clusters from qcow2_co_preadv

2018-09-27 Thread Max Reitz
On 07.08.18 19:43, Vladimir Sementsov-Ogievskiy wrote: > Memory allocation may become less efficient for encrypted case. It's a > payment for further asynchronous scheme. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/qcow2.c | 114 >

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/5] dirty-bitmaps: fix QMP command permissions

2018-09-27 Thread John Snow
On 09/26/2018 10:23 PM, Eric Blake wrote: > On 9/25/18 6:49 PM, John Snow wrote: >> based on: jsnow/bitmaps staging branch >> >> This series builds on a previous standalone patch and adjusts >> the permission for all (or most) of the QMP bitmap commands. >> >> John Snow (5): >>   

Re: [Qemu-block] [PATCH v3 1/5] block/dirty-bitmaps: add user_modifiable status checker

2018-09-27 Thread John Snow
On 09/26/2018 10:17 PM, Eric Blake wrote: > On 9/26/18 6:53 AM, Vladimir Sementsov-Ogievskiy wrote: >> 26.09.2018 02:49, John Snow wrote: >>> Instead of both frozen and qmp_locked checks, wrap it into one check. >>> frozen implies the bitmap is split in two (for backup), and shouldn't >>> be

Re: [Qemu-block] [PATCH] nbd: Don't take address of fields in packed structs

2018-09-27 Thread Eric Blake
On 9/27/18 11:42 AM, Peter Maydell wrote: Taking the address of a field in a packed struct is a bad idea, because it might not be actually aligned enough for that pointer type (and thus cause a crash on dereference on some host architectures). Newer versions of clang warn about this. Avoid the

Re: [Qemu-block] [PATCH] util/hbitmaps: recalculate count on merge

2018-09-27 Thread John Snow
On 09/27/2018 10:09 AM, Eric Blake wrote: > On 9/27/18 2:31 AM, Vladimir Sementsov-Ogievskiy wrote: >> 27.09.2018 00:28, John Snow wrote: >>> We have been neglecting to do so, which results in wrong counts >>> after merge. In the worst case, we may think the bitmap is empty >>> when it has had

Re: [Qemu-block] [Qemu-devel] [PATCH] bitmap: Update count after a merge

2018-09-27 Thread John Snow
On 09/26/2018 11:11 PM, Eric Blake wrote: > We need an accurate count of the number of bits set in a bitmap > after a merge. In particular, since the merge operation short-circuits > a merge from an empty source, if you have bitmaps A, B, and C where > B started empty, then merge C into B, and

Re: [Qemu-block] [Qemu-devel] [PATCH] dirty-bitmaps: allow merging to disabled bitmaps

2018-09-27 Thread John Snow
On 09/26/2018 10:25 PM, Eric Blake wrote: > On 9/19/18 4:16 PM, John Snow wrote: >> >> >> On 09/19/2018 05:08 PM, Eric Blake wrote: >>> On 9/19/18 2:58 PM, John Snow wrote: We wish to prohibit merging to read-only bitmaps and frozen bitmaps, but "disabled" bitmaps only preclude their

Re: [Qemu-block] [PATCH] nbd: Don't take address of fields in packed structs

2018-09-27 Thread Eric Blake
On 9/27/18 11:42 AM, Peter Maydell wrote: Taking the address of a field in a packed struct is a bad idea, because it might not be actually aligned enough for that pointer type (and thus cause a crash on dereference on some host architectures). Newer versions of clang warn about this. Avoid the

Re: [Qemu-block] [PATCH 2/7] qcow2: bdrv_co_pwritev: move encryption code out of lock

2018-09-27 Thread Max Reitz
On 07.08.18 19:43, Vladimir Sementsov-Ogievskiy wrote: > We don't need locking for encryption code. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/qcow2.c | 20 > 1 file changed, 12 insertions(+), 8 deletions(-) > > diff --git a/block/qcow2.c b/block/qcow2.c

Re: [Qemu-block] [PATCH 1/7] qcow2: move qemu_co_mutex_lock below decryption procedure

2018-09-27 Thread Max Reitz
On 27.09.18 18:58, Max Reitz wrote: > On 07.08.18 19:43, Vladimir Sementsov-Ogievskiy wrote: >> From: "Denis V. Lunev" >> >> We are not working with a shared state data in the decruption code and (*decryption) >> thus this operation is safe. On the other hand this significantly >> reduces the

Re: [Qemu-block] [PATCH 1/7] qcow2: move qemu_co_mutex_lock below decryption procedure

2018-09-27 Thread Max Reitz
On 07.08.18 19:43, Vladimir Sementsov-Ogievskiy wrote: > From: "Denis V. Lunev" > > We are not working with a shared state data in the decruption code and > thus this operation is safe. On the other hand this significantly > reduces the scope of the lock. Sure, but does it have any effect?

[Qemu-block] [PATCH] nbd: Don't take address of fields in packed structs

2018-09-27 Thread Peter Maydell
Taking the address of a field in a packed struct is a bad idea, because it might not be actually aligned enough for that pointer type (and thus cause a crash on dereference on some host architectures). Newer versions of clang warn about this. Avoid the bug by not using the "modify in place" byte

Re: [Qemu-block] [PATCH v12 8/9] qcow2: Set the default cache-clean-interval to 10 minutes

2018-09-27 Thread Eric Blake
On 9/26/18 11:04 AM, Leonid Bloch wrote: The default cache-clean-interval is set to 10 minutes, in order to lower the overhead of the qcow2 caches (before the default was 0, i.e. disabled). * For non-Linux platforms the default is kept at 0, because cache-clean-interval is not supported

Re: [Qemu-block] [PATCH v12 0/9] Take the image size into account when allocating the L2 cache

2018-09-27 Thread Kevin Wolf
Am 26.09.2018 um 18:04 hat Leonid Bloch geschrieben: > This series makes the qcow2 L2 cache assignment aware of the image size, > with the intention for it to cover the entire image. The importance of > this change is in noticeable performance improvement, especially with > heavy random I/O. The

Re: [Qemu-block] [Qemu-devel] Can I only commit from active image to corresponding range of its backing file by qemu cmd?

2018-09-27 Thread Max Reitz
On 27.09.18 16:23, Alberto Garcia wrote: > On Thu 27 Sep 2018 04:14:15 PM CEST, Max Reitz wrote: >> On 25.09.18 16:13, Alberto Garcia wrote: >>> On Thu 13 Sep 2018 08:37:05 PM CEST, Max Reitz wrote: First, split .003 into the part we want to commit and the part we don't want to commit.

Re: [Qemu-block] [Qemu-devel] Can I only commit from active image to corresponding range of its backing file by qemu cmd?

2018-09-27 Thread Alberto Garcia
On Thu 27 Sep 2018 04:14:15 PM CEST, Max Reitz wrote: > On 25.09.18 16:13, Alberto Garcia wrote: >> On Thu 13 Sep 2018 08:37:05 PM CEST, Max Reitz wrote: >>> First, split .003 into the part we want to commit and the part we >>> don't want to commit. This is a bit tricky without qemu-img dd @seek

Re: [Qemu-block] [Qemu-devel] Can I only commit from active image to corresponding range of its backing file by qemu cmd?

2018-09-27 Thread Max Reitz
On 25.09.18 16:13, Alberto Garcia wrote: > On Thu 13 Sep 2018 08:37:05 PM CEST, Max Reitz wrote: >> First, split .003 into the part we want to commit and the part we >> don't want to commit. This is a bit tricky without qemu-img dd @seek >> (or a corresponding convert parameter), so we'll have to

Re: [Qemu-block] intermittent failure in test-replication

2018-09-27 Thread Kevin Wolf
Am 27.09.2018 um 12:08 hat Peter Maydell geschrieben: > Hi; I seem to be getting an intermittent failure in > tests/test-replication: > > MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} > gtester -k --verbose -m=quick tests/test-replication > TEST: tests/test-replication...

Re: [Qemu-block] [PATCH v3 10/10] block: Allow changing 'force-share' on reopen

2018-09-27 Thread Kevin Wolf
Am 26.09.2018 um 13:39 hat Alberto Garcia geschrieben: > On Wed 26 Sep 2018 01:34:28 PM CEST, Kevin Wolf wrote: > >> @@ -3353,6 +3370,7 @@ void bdrv_reopen_commit(BDRVReopenState > >> *reopen_state) > >> bs->open_flags = reopen_state->flags; > >> bs->read_only =

[Qemu-block] intermittent failure in test-replication

2018-09-27 Thread Peter Maydell
Hi; I seem to be getting an intermittent failure in tests/test-replication: MALLOC_PERTURB_=${MALLOC_PERTURB_:-$(( ${RANDOM:-0} % 255 + 1))} gtester -k --verbose -m=quick tests/test-replication TEST: tests/test-replication... (pid=8465) /replication/primary/read:

Re: [Qemu-block] [PATCH] bitmap: Update count after a merge

2018-09-27 Thread Vladimir Sementsov-Ogievskiy
27.09.2018 06:11, Eric Blake wrote: We need an accurate count of the number of bits set in a bitmap after a merge. In particular, since the merge operation short-circuits a merge from an empty source, if you have bitmaps A, B, and C where B started empty, then merge C into B, and B into A, an

Re: [Qemu-block] [PATCH] util/hbitmaps: recalculate count on merge

2018-09-27 Thread Vladimir Sementsov-Ogievskiy
27.09.2018 00:28, John Snow wrote: We have been neglecting to do so, which results in wrong counts after merge. In the worst case, we may think the bitmap is empty when it has had new writes merged into it. Reported-by: Eric Blake Signed-off-by: John Snow --- util/hbitmap.c | 3 +++ 1 file