Re: [PATCH v3 4/5] qemu-iotests: let "check" spawn an arbitrary test command

2021-03-30 Thread Max Reitz
On 30.03.21 12:38, Max Reitz wrote: On 26.03.21 16:05, Max Reitz wrote: On 26.03.21 15:23, Paolo Bonzini wrote: Right now there is no easy way for "check" to print a reproducer command. Because such a reproducer command line would be huge, we can instead teach check to start a comm

Re: [PATCH v3 4/5] qemu-iotests: let "check" spawn an arbitrary test command

2021-03-30 Thread Max Reitz
On 26.03.21 16:05, Max Reitz wrote: On 26.03.21 15:23, Paolo Bonzini wrote: Right now there is no easy way for "check" to print a reproducer command. Because such a reproducer command line would be huge, we can instead teach check to start a command of our choice.  This can be for

Re: [PATCH v4 for-6.0? 0/3] qcow2: fix parallel rewrite and discard (rw-lock)

2021-03-30 Thread Max Reitz
On 25.03.21 20:12, Vladimir Sementsov-Ogievskiy wrote: ping. Do we want it for 6.0? I’d rather wait. I think the conclusion was that guests shouldn’t hit this because they serialize discards? There’s also something Kevin wrote on IRC a couple of weeks ago, for which I had hoped he’d sent a

Re: [PATCH] qsd: Document FUSE exports

2021-03-29 Thread Max Reitz
On 17.02.21 12:58, Max Reitz wrote: Implementing FUSE exports required no changes to the storage daemon, so we forgot to document them there. Considering that both NBD and vhost-user-blk exports are documented in its man page (and NBD exports in its --help text), we should probably do the same

Re: [PATCH 0/2] file-posix: Cache next hole

2021-03-29 Thread Max Reitz
On 11.02.21 18:22, Max Reitz wrote: Hi, [...] (Speaking of “unless the WRITE permission is shared”: mirror_top is a bit broken in that it takes no permissions (but WRITE if necessary) and shares everything. That seems wrong. Patch 1 addresses that, so that patch 2 can actually do something

Re: [PATCH] iotests/046: Filter request length

2021-03-29 Thread Max Reitz
On 18.09.20 17:33, Max Reitz wrote: For its concurrent requests, 046 has always filtered the offset, probably because concurrent requests may settle in any order. However, it did not filter the request length, and so if requests with different lengths settle in an unexpected order (notably the

Re: [PATCH] qcow2: use external virtual timers

2021-03-29 Thread Max Reitz
On 29.03.21 10:06, Pavel Dovgalyuk wrote: Regular virtual timers are used to emulate timings related to vCPU and peripheral states. QCOW2 uses timers to clean the cache. These timers should have external flag. In the opposite case they affect the execution and it can't be recorded and replayed. T

[PATCH 0/4] iotests/297: Cover tests/

2021-03-29 Thread Max Reitz
yet. I think it would be nice if we could keep all of tests/ clean. Max Reitz (4): iotests/297: Drop 169 and 199 from the skip list migrate-bitmaps-postcopy-test: Fix pylint warnings migrate-bitmaps-test: Fix pylint warnings iotests/297: Cover tests/ tests/qemu-iotests/297

[PATCH 1/4] iotests/297: Drop 169 and 199 from the skip list

2021-03-29 Thread Max Reitz
169 and 199 have been renamed and moved to tests/ (commit a44be0334be: "iotests: rename and move 169 and 199 tests"), so we can drop them from the skip list. Signed-off-by: Max Reitz --- tests/qemu-iotests/297 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/

[PATCH 4/4] iotests/297: Cover tests/

2021-03-29 Thread Max Reitz
297 so far does not check the named tests, which reside in the tests/ directory (i.e. full path tests/qemu-iotests/tests). Fix it. Thanks to the previous two commits, all named tests pass its scrutiny, so we do not have to add anything to SKIP_FILES. Signed-off-by: Max Reitz --- tests/qemu

[PATCH 3/4] migrate-bitmaps-test: Fix pylint warnings

2021-03-29 Thread Max Reitz
me lines are too long (80 characters instead of 79) - inject_test_case()'s @name parameter shadows a top-level @name variable - "lambda self: mc(self)" is equivalent to just "mc" - Always put two empty lines after a function - f'exec: cat > /dev/null' does not need to

[PATCH 2/4] migrate-bitmaps-postcopy-test: Fix pylint warnings

2021-03-29 Thread Max Reitz
pylint complains that discards1_sha256 and all_discards_sha256 are first set in non-__init__ methods. Let's make it happy. Signed-off-by: Max Reitz --- tests/qemu-iotests/tests/migrate-bitmaps-postcopy-test | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/tests/mi

Re: [PATCH] iotests: add test for removing persistent bitmap from backing file

2021-03-29 Thread Max Reitz
On 17.03.21 17:02, Vladimir Sementsov-Ogievskiy wrote: Just demonstrate one of x-blockdev-reopen usecases. We can't simply remove persistent bitmap from RO node (for example from backing file), as we need to remove it from the image too. So, we should reopen the node first. Signed-off-by: Vladim

Re: [PATCH v2 2/2] iotests/244: Test preallocation for data-file-raw

2021-03-26 Thread Max Reitz
On 26.03.21 16:17, Eric Blake wrote: On 3/26/21 9:55 AM, Max Reitz wrote: Three test cases: (1) Adding a qcow2 (metadata) file to an existing data file, see whether we can read the existing data through the qcow2 image. (2) Append data to the data file, grow the qcow2 image accordingly

Re: [PATCH v3 4/5] qemu-iotests: let "check" spawn an arbitrary test command

2021-03-26 Thread Max Reitz
On 26.03.21 15:23, Paolo Bonzini wrote: Right now there is no easy way for "check" to print a reproducer command. Because such a reproducer command line would be huge, we can instead teach check to start a command of our choice. This can be for example a Python unit test with arguments to only r

[PATCH v2 0/2] qcow2: Force preallocation with data-file-raw

2021-03-26 Thread Max Reitz
)ontextual differences, respectively 001/2:[0012] [FC] 'qcow2: Force preallocation with data-file-raw' 002/2:[0110] [FC] 'iotests/244: Test preallocation for data-file-raw' Max Reitz (2): qcow2: Force preallocation with data-file-raw iotests/244: Test preallocation for data-file

[PATCH v2 2/2] iotests/244: Test preallocation for data-file-raw

2021-03-26 Thread Max Reitz
case, because without the L2 tables preallocated, all clusters would appear as unallocated, and so the qcow2 driver would fall through to the backing file.) Signed-off-by: Max Reitz --- tests/qemu-iotests/244 | 104 + tests/qemu-iotests/244.out | 59

[PATCH v2 1/2] qcow2: Force preallocation with data-file-raw

2021-03-26 Thread Max Reitz
data area. Signed-off-by: Max Reitz --- block/qcow2.c | 34 ++ tests/qemu-iotests/244.out | 9 - 2 files changed, 38 insertions(+), 5 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 0db1227ac9..9920c756eb 100644 --- a/block/qc

[PATCH] iotests/116: Fix reference output

2021-03-26 Thread Max Reitz
ils. Let's fix the reference output, which has the nice side effect of demonstrating 15ce94a68ca's improvements. Fixes: 15ce94a68ca6730466c565c3d29971aab3087bf1 ("block/qed: bdrv_qed_do_open: deal with errp") Signed-off-by: Max Reitz --- tests/qemu-iotests/116.out |

Re: [PATCH 1/4] qcow2: Improve refcount structure rebuilding

2021-03-26 Thread Max Reitz
On 26.03.21 12:48, Vladimir Sementsov-Ogievskiy wrote: 10.03.2021 18:59, Max Reitz wrote: When rebuilding the refcount structures (when qemu-img check -r found errors with refcount = 0, but reference count > 0), the new refcount table defaults to being put at the image file end[1].  There is

Re: [PATCH v2 0/5] qemu-iotests: quality of life improvements

2021-03-26 Thread Max Reitz
On 23.03.21 19:19, Paolo Bonzini wrote: This series adds a few usability improvements to qemu-iotests, in particular: - arguments can be passed to Python unittests scripts, for example to run only a subset of the test cases (patches 1-2) - it is possible to do "./check -- ../../../tests/qemu

Re: [PATCH] Document qemu-img options data_file and data_file_raw

2021-03-26 Thread Max Reitz
On 01.03.21 18:28, Connor Kuehl wrote: The contents of this patch were initially developed and posted by Han Han[1], however, it appears the original patch was not applied. Since then, the relevant documentation has been moved and adapted to a new format. I've taken most of the original wording

Re: [PATCH v2 0/5] qemu-iotests: quality of life improvements

2021-03-25 Thread Max Reitz
On 23.03.21 19:19, Paolo Bonzini wrote: This series adds a few usability improvements to qemu-iotests, in particular: - arguments can be passed to Python unittests scripts, for example to run only a subset of the test cases (patches 1-2) - it is possible to do "./check -- ../../../tests/qemu

Re: [PATCH 0/1] iotests: fix 051.out expected output after error

2021-03-25 Thread Max Reitz
On 18.03.21 21:09, Connor Kuehl wrote: Oops, sorry about the churn. I can see why this would have caused a failure but I'm surprised I can't reproduce this when I run the test locally. Christian, would you be willing to test this patch out as a quick sanity check too? Connor Kuehl (1): iotes

Re: [PATCH] iotests: Fix typo in iotest 051

2021-03-25 Thread Max Reitz
On 24.03.21 09:43, Tao Xu wrote: There is an typo in iotest 051, correct it. Signed-off-by: Tao Xu --- tests/qemu-iotests/051| 2 +- tests/qemu-iotests/051.pc.out | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) Thanks, applied to my block branch: https://git.xanclic.mo

Re: [PATCH V4] file-posix: allow -EBUSY error during ioctl(fd, BLKZEROOUT, range) on block

2021-03-24 Thread Max Reitz
On 22.03.21 10:25, ChangLimin wrote: For Linux 5.10/5.11, qemu write zeros to a multipath device using ioctl(fd, BLKZEROOUT, range) with cache none or directsync return -EBUSY permanently. So as far as I can track back the discussion, Kevin asked on v1 why we’d set has_write_zeroes to false, i

Re: [PATCH 2/6] block/vdi: Don't assume that blocks are larger than VdiHeader

2021-03-24 Thread Max Reitz
weird to keep that check if *block were freed by that point, I suppose this isn’t making it worse, though, so: Reviewed-by: Max Reitz +header = g_malloc(sizeof(*header)); + logout("now writing modified header\n"); assert(VDI_IS_ALLOCATED(bmap_first));

Re: [PATCH v5 0/6] coroutine rwlock downgrade fix, minor VDI changes

2021-03-24 Thread Max Reitz
On 17.03.21 19:00, Paolo Bonzini wrote: This is a resubmit of David Edmondson's series at https://patchew.org/QEMU/20210309144015.557477-1-david.edmond...@oracle.com/. After closer analysis on IRC, the CoRwlock's attempt to ensure fairness turned out to be flawed. Therefore, this series reimplem

Re: [PATCH 1/6] block/vdi: When writing new bmap entry fails, don't leak the buffer

2021-03-24 Thread Max Reitz
-Id: <20210309144015.557477-2-david.edmond...@oracle.com> Signed-off-by: Paolo Bonzini --- block/vdi.c | 1 + 1 file changed, 1 insertion(+) Reviewed-by: Max Reitz

Re: Fwd: [PATCH 0/2] block/raw: implemented persistent dirty bitmap and ability to dump bitmap content via qapi

2021-03-22 Thread Max Reitz
On 22.03.21 12:27, Patrik Janoušek wrote: On 3/22/21 11:48 AM, Max Reitz wrote: Hi, On 20.03.21 11:01, Patrik Janoušek wrote: I'm sorry, but I forgot to add you to the cc, so I'm forwarding the patch to you additionally. I don't want to spam the mailing list unnecessarily.

Re: [PATCH 2/2] hw/block/nvme: fix resource leak in nvme_format_ns

2021-03-22 Thread Max Reitz
On 22.03.21 11:48, Klaus Jensen wrote: On Mar 22 11:02, Max Reitz wrote: On 22.03.21 07:19, Klaus Jensen wrote: From: Klaus Jensen In nvme_format_ns(), if the namespace is of zero size (which might be useless, but not invalid), the `count` variable will leak. Fix this by returning early in

Re: Fwd: [PATCH 0/2] block/raw: implemented persistent dirty bitmap and ability to dump bitmap content via qapi

2021-03-22 Thread Max Reitz
Hi, On 20.03.21 11:01, Patrik Janoušek wrote: I'm sorry, but I forgot to add you to the cc, so I'm forwarding the patch to you additionally. I don't want to spam the mailing list unnecessarily. I think it’s better to still CC the list. It’s so full of mail, one more won’t hurt. :) (Re-add

Re: [PATCH 2/2] hw/block/nvme: fix resource leak in nvme_format_ns

2021-03-22 Thread Max Reitz
On 22.03.21 07:19, Klaus Jensen wrote: From: Klaus Jensen In nvme_format_ns(), if the namespace is of zero size (which might be useless, but not invalid), the `count` variable will leak. Fix this by returning early in that case. When looking at the Coverity report, something else caught my ey

Re: [PULL 5/5] m68k: add Virtual M68k Machine

2021-03-19 Thread Max Reitz
On 19.03.21 11:51, Max Reitz wrote: On 19.03.21 11:50, Laurent Vivier wrote: Le 19/03/2021 à 10:20, Max Reitz a écrit : On 19.03.21 07:32, Thomas Huth wrote: On 18/03/2021 18.28, Max Reitz wrote: [...]   From that it follows that I don’t see much use in testing specific devices either.  Say

Re: [PULL 5/5] m68k: add Virtual M68k Machine

2021-03-19 Thread Max Reitz
On 19.03.21 11:50, Laurent Vivier wrote: Le 19/03/2021 à 10:20, Max Reitz a écrit : On 19.03.21 07:32, Thomas Huth wrote: On 18/03/2021 18.28, Max Reitz wrote: [...]  From that it follows that I don’t see much use in testing specific devices either.  Say there’s a platform that provides

Re: [PULL 5/5] m68k: add Virtual M68k Machine

2021-03-19 Thread Max Reitz
On 19.03.21 07:32, Thomas Huth wrote: On 18/03/2021 18.28, Max Reitz wrote: [...]  From that it follows that I don’t see much use in testing specific devices either.  Say there’s a platform that provides both virtio-pci and virtio-mmio, the default (say virtio-pci) is fine for the iotests. I

Re: [PULL 5/5] m68k: add Virtual M68k Machine

2021-03-18 Thread Max Reitz
On 18.03.21 17:25, Philippe Mathieu-Daudé wrote: On 3/18/21 4:56 PM, Laurent Vivier wrote: Le 18/03/2021 à 16:51, Laurent Vivier a écrit : Le 18/03/2021 à 16:36, Philippe Mathieu-Daudé a écrit : On 3/18/21 11:06 AM, Laurent Vivier wrote: Le 18/03/2021 à 11:02, Philippe Mathieu-Daudé a écrit :

Re: [PATCH v3 6/6] block/qcow2: use seqcache for compressed writes

2021-03-17 Thread Max Reitz
On 16.03.21 18:48, Vladimir Sementsov-Ogievskiy wrote: 16.03.2021 15:25, Max Reitz wrote: On 15.03.21 15:40, Vladimir Sementsov-Ogievskiy wrote: 15.03.2021 12:58, Max Reitz wrote: [...] The question is whether it really makes sense to even have a seqcache_read() path when in reality it’s

Re: [PATCH v3 6/6] block/qcow2: use seqcache for compressed writes

2021-03-16 Thread Max Reitz
On 15.03.21 15:40, Vladimir Sementsov-Ogievskiy wrote: 15.03.2021 12:58, Max Reitz wrote: [...] The question is whether it really makes sense to even have a seqcache_read() path when in reality it’s probably never accessed.  I mean, besides the fact that it seems based purely on chance

Re: [PATCH v3 6/6] block/qcow2: use seqcache for compressed writes

2021-03-15 Thread Max Reitz
On 12.03.21 19:43, Vladimir Sementsov-Ogievskiy wrote: 12.03.2021 21:15, Max Reitz wrote: On 05.03.21 18:35, Vladimir Sementsov-Ogievskiy wrote: Compressed writes are unaligned to 512, which works very slow in O_DIRECT mode. Let's use the cache. Signed-off-by: Vladimir Sementsov-Ogie

Re: [PATCH v3 6/6] block/qcow2: use seqcache for compressed writes

2021-03-12 Thread Max Reitz
On 05.03.21 18:35, Vladimir Sementsov-Ogievskiy wrote: Compressed writes are unaligned to 512, which works very slow in O_DIRECT mode. Let's use the cache. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/coroutines.h | 3 + block/qcow2.h | 4 ++ block/qcow2-refcount.c

Re: [PATCH v3 5/6] block-coroutine-wrapper: allow non bdrv_ prefix

2021-03-12 Thread Max Reitz
deletions(-) Reviewed-by: Max Reitz

Re: [PATCH v3 3/6] block/qcow2: introduce inflight writes counters: fix discard

2021-03-12 Thread Max Reitz
On 12.03.21 16:24, Vladimir Sementsov-Ogievskiy wrote: 12.03.2021 18:10, Max Reitz wrote: On 12.03.21 13:46, Vladimir Sementsov-Ogievskiy wrote: 12.03.2021 15:32, Vladimir Sementsov-Ogievskiy wrote: 12.03.2021 14:17, Max Reitz wrote: On 12.03.21 10:09, Vladimir Sementsov-Ogievskiy wrote

Re: [PATCH v3 4/6] util: implement seqcache

2021-03-12 Thread Max Reitz
On 12.03.21 15:37, Vladimir Sementsov-Ogievskiy wrote: 12.03.2021 16:41, Max Reitz wrote: On 05.03.21 18:35, Vladimir Sementsov-Ogievskiy wrote: Implement cache for small sequential unaligned writes, so that they may be cached until we get a complete cluster and then write it. The cache is

Re: [PATCH v3 3/6] block/qcow2: introduce inflight writes counters: fix discard

2021-03-12 Thread Max Reitz
On 12.03.21 13:46, Vladimir Sementsov-Ogievskiy wrote: 12.03.2021 15:32, Vladimir Sementsov-Ogievskiy wrote: 12.03.2021 14:17, Max Reitz wrote: On 12.03.21 10:09, Vladimir Sementsov-Ogievskiy wrote: 11.03.2021 22:58, Max Reitz wrote: On 05.03.21 18:35, Vladimir Sementsov-Ogievskiy wrote

Re: [PATCH v3 3/6] block/qcow2: introduce inflight writes counters: fix discard

2021-03-12 Thread Max Reitz
On 12.03.21 13:42, Vladimir Sementsov-Ogievskiy wrote: 12.03.2021 15:32, Vladimir Sementsov-Ogievskiy wrote: 12.03.2021 14:17, Max Reitz wrote: On 12.03.21 10:09, Vladimir Sementsov-Ogievskiy wrote: 11.03.2021 22:58, Max Reitz wrote: On 05.03.21 18:35, Vladimir Sementsov-Ogievskiy wrote

Re: [PATCH v3 3/6] block/qcow2: introduce inflight writes counters: fix discard

2021-03-12 Thread Max Reitz
On 12.03.21 13:32, Vladimir Sementsov-Ogievskiy wrote: 12.03.2021 14:17, Max Reitz wrote: On 12.03.21 10:09, Vladimir Sementsov-Ogievskiy wrote: 11.03.2021 22:58, Max Reitz wrote: On 05.03.21 18:35, Vladimir Sementsov-Ogievskiy wrote: There is a bug in qcow2: host cluster can be discarded

Re: [PATCH v3 4/6] util: implement seqcache

2021-03-12 Thread Max Reitz
On 05.03.21 18:35, Vladimir Sementsov-Ogievskiy wrote: Implement cache for small sequential unaligned writes, so that they may be cached until we get a complete cluster and then write it. The cache is intended to be used for backup to qcow2 compressed target opened in O_DIRECT mode, but can be r

Re: [PATCH v3 3/6] block/qcow2: introduce inflight writes counters: fix discard

2021-03-12 Thread Max Reitz
On 12.03.21 10:09, Vladimir Sementsov-Ogievskiy wrote: 11.03.2021 22:58, Max Reitz wrote: On 05.03.21 18:35, Vladimir Sementsov-Ogievskiy wrote: There is a bug in qcow2: host cluster can be discarded (refcount becomes 0) and reused during data write. In this case data write may pollute another

Re: [PATCH v3 3/6] block/qcow2: introduce inflight writes counters: fix discard

2021-03-11 Thread Max Reitz
On 05.03.21 18:35, Vladimir Sementsov-Ogievskiy wrote: There is a bug in qcow2: host cluster can be discarded (refcount becomes 0) and reused during data write. In this case data write may pollute another cluster (recently allocated) or even metadata. I was about to ask whether we couldn’t some

Re: [PATCH v3 1/6] block-jobs: flush target at the end of .run()

2021-03-11 Thread Max Reitz
++ block/mirror.c | 2 ++ block/stream.c | 2 ++ blockjob.c | 16 6 files changed, 45 insertions(+), 3 deletions(-) Reviewed-by: Max Reitz Just a nit on the function’s description. diff --git a/include/block/blockjob_int.h b

Re: [PATCH] block: Introduce zero-co:// and zero-aio://

2021-03-11 Thread Max Reitz
On 10.03.21 17:35, Fam Zheng wrote: On Wed, 10 Mar 2021 at 15:02, Max Reitz <mailto:mre...@redhat.com>> wrote: On 10.03.21 15:17, f...@euphon.net <mailto:f...@euphon.net> wrote: > From: Fam Zheng mailto:famzh...@amazon.com>> > > null-co://

[PATCH 3/4] iotests/common.qemu: Allow using the QSD

2021-03-10 Thread Max Reitz
For block things, we often do not need to run all of qemu, so allow using the qemu-storage-daemon instead. Signed-off-by: Max Reitz --- tests/qemu-iotests/common.qemu | 53 +++--- 1 file changed, 43 insertions(+), 10 deletions(-) diff --git a/tests/qemu-iotests

[PATCH 4/4] iotests/108: Test new refcount rebuild algorithm

2021-03-10 Thread Max Reitz
ge file end, i.e. outside of what the block device provides, which cannot work. HEAD^ should have fixed that. ("Something like a block device" means a loop device if we can use one ("sudo -n losetup" works), or a FUSE block export with growable=false otherwise.) S

[PATCH 2/4] iotests/common.qemu: Add _cleanup_single_qemu

2021-03-10 Thread Max Reitz
_cleanup_qemu cleans up all qemu instances, which sometimes is not very useful. Pull out _cleanup_single_qemu, which does the same only for a single instance. Signed-off-by: Max Reitz --- tests/qemu-iotests/common.qemu | 55 +- 1 file changed, 34 insertions

[PATCH 1/4] qcow2: Improve refcount structure rebuilding

2021-03-10 Thread Max Reitz
the last refblock, so we have to write that refblock. In that case, we try to put the reftable in there. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1519071 Signed-off-by: Max Reitz --- block/qcow2-refcount.c | 126 ++--- 1 file changed, 67 inserti

[PATCH 0/4] qcow2: Improve refcount structure rebuilding

2021-03-10 Thread Max Reitz
like anything about 2 or 3, I also have a version of this series without patches 2 and 3, where 4 is correspondingly unnicer.) Max Reitz (4): qcow2: Improve refcount structure rebuilding iotests/common.qemu: Add _cleanup_single_qemu iotests/common.qemu: Allow using the QSD iotests/108: Test

Re: [PATCH] block: Introduce zero-co:// and zero-aio://

2021-03-10 Thread Max Reitz
On 10.03.21 15:17, f...@euphon.net wrote: From: Fam Zheng null-co:// has a read-zeroes=off default, when used to in security analysis, this can cause false positives because the driver doesn't write to the read buffer. null-co:// has the highest possible performance as a block driver, so let's

[Bug 1916501] Re: qemu-img convert segfaults with specific URL

2021-03-09 Thread Max Reitz
I think I’ve come to kind of understood what might be wrong: qemu frees CURLSocket objects when “their” transfer is done, but libcurl’s documentation actually doesn’t note any long-lasting relationship between a socket and some transfer (i.e., a CURL object), so we probably shouldn’t free CURLSocke

[PATCH 0/2] block/curl: Disconnect sockets from CURLState

2021-03-09 Thread Max Reitz
it by invoking curl_sock_cb() with CURL_POLL_REMOVE.) Max Reitz (2): curl: Store BDRVCURLState pointer in CURLSocket curl: Disconnect sockets from CURLState block/curl.c | 50 -- 1 file changed, 28 insertions(+), 22 deletions(-) -- 2.29.2

[PATCH 2/2] curl: Disconnect sockets from CURLState

2021-03-09 Thread Max Reitz
o point in freeing any socket just because a transfer is done. libcurl does invoke curl_sock_cb() with CURL_POLL_REMOVE for every socket it has. Buglink: https://bugs.launchpad.net/qemu/+bug/1916501 Signed-off-by: Max Reitz --- block/curl.c | 42 -- 1 file c

[PATCH 1/2] curl: Store BDRVCURLState pointer in CURLSocket

2021-03-09 Thread Max Reitz
A socket does not really belong to any specific state. We do not need to store a pointer to "its" state in it, a pointer to the common BDRVCURLState is sufficient. Signed-off-by: Max Reitz --- block/curl.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/bl

Re: [PATCH] docs: qsd: Explain --export nbd,name=... default

2021-03-05 Thread Max Reitz
-daemon.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) Reviewed-by: Max Reitz

[Bug 1916501] Re: qemu-img convert segfaults with specific URL

2021-03-03 Thread Max Reitz
Yes, as I wrote in comment 1, curl reports CURLMSG_DONE, the socket is freed, but then curl_multi_do() is called again for that socket (despite the CURLMSG_DONE). I suspect that qemu has interpreted the curl interface differently than curl itself (i.e., qemu has probably understood something wrong

[Bug 1916501] Re: qemu-img convert segfaults with specific URL

2021-02-23 Thread Max Reitz
I can reproduce this, and I can reproduce it back to 5.0 (haven’t tried any release before that). I couldn’t find a definite reason for why it breaks (curl_clean_state() is called because curl reports CURLMSG_DONE, freeing a socket, but then curl_multi_do() is called again for that socket, resulti

Re: [PATCH v2 0/2] block: Use 'read-zeroes=true' mode by default with 'null-co' driver

2021-02-23 Thread Max Reitz
On 23.02.21 10:21, Fam Zheng wrote: On 2021-02-22 18:55, Philippe Mathieu-Daudé wrote: On 2/22/21 6:35 PM, Fam Zheng wrote: On 2021-02-19 15:09, Philippe Mathieu-Daudé wrote: On 2/19/21 12:07 PM, Max Reitz wrote: On 13.02.21 22:54, Fam Zheng wrote: On 2021-02-11 15:26, Philippe Mathieu

Re: [PATCH v2 0/2] block: Use 'read-zeroes=true' mode by default with 'null-co' driver

2021-02-23 Thread Max Reitz
On 22.02.21 19:15, Daniel P. Berrangé wrote: On Fri, Feb 19, 2021 at 03:09:43PM +0100, Philippe Mathieu-Daudé wrote: On 2/19/21 12:07 PM, Max Reitz wrote: On 13.02.21 22:54, Fam Zheng wrote: On 2021-02-11 15:26, Philippe Mathieu-Daudé wrote: The null-co driver doesn't zeroize buffer i

Re: [PATCH 3/3] iotests/283: Check that finalize drops backup-top

2021-02-19 Thread Max Reitz
On 19.02.21 16:33, Max Reitz wrote: Without any of HEAD^ or HEAD^^ applied, qemu will most likely crash on the qemu-io invocation, for a variety of immediate reasons. The underlying problem is generally a use-after-free access into backup-top's BlockCopyState. With only HEAD^ applied, qe

[PATCH 3/3] iotests/283: Check that finalize drops backup-top

2021-02-19 Thread Max Reitz
ilter and passes the request through to its child. However, after bdrv_backup_top_drop(), that child is NULL, so the recursive call crashes. With HEAD^^ applied, this test should pass. Signed-off-by: Max Reitz --- tests/qemu-iotests/283 | 55 ++ tests

[PATCH 0/3] backup-top: Don't crash on post-finalize accesses

2021-02-19 Thread Max Reitz
of because it just made sense to me, even if it won’t do anything. Max Reitz (3): backup: Remove nodes from job in .clean() backup-top: Refuse I/O in inactive state iotests/283: Check that finalize drops backup-top block/backup-top.c | 10 +++ block/backup.c | 1

[PATCH 1/3] backup: Remove nodes from job in .clean()

2021-02-19 Thread Max Reitz
the job before bdrv_backup_top_drop(). Because there is no function to do that for a single node, just detach all of the job's nodes -- the job does not do anything past backup_clean() anyway. Signed-off-by: Max Reitz --- block/backup.c | 1 + 1 file changed, 1 insertion(+) diff --git

[PATCH 2/3] backup-top: Refuse I/O in inactive state

2021-02-19 Thread Max Reitz
e in the I/O functions.) Signed-off-by: Max Reitz --- block/backup-top.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/block/backup-top.c b/block/backup-top.c index d1253e1aa6..589e8b651d 100644 --- a/block/backup-top.c +++ b/block/backup-top.c @@ -45,6 +45,12 @@ static coro

Re: [PATCH] iotests: Drop deprecated 'props' from object-add

2021-02-19 Thread Max Reitz
/qemu-iotests/258 | 7 +++ tests/qemu-iotests/258.out | 4 ++-- tests/qemu-iotests/295 | 2 +- tests/qemu-iotests/296 | 2 +- 9 files changed, 19 insertions(+), 30 deletions(-) Reviewed-by: Max Reitz Two Python syntax nit picks below. [...] diff --git a/tests/qemu-iotests

Re: [PATCH v2 0/2] block: Use 'read-zeroes=true' mode by default with 'null-co' driver

2021-02-19 Thread Max Reitz
On 13.02.21 22:54, Fam Zheng wrote: On 2021-02-11 15:26, Philippe Mathieu-Daudé wrote: The null-co driver doesn't zeroize buffer in its default config, because it is designed for testing and tests want to run fast. However this confuses security researchers (access to uninit buffers). I'm a li

Re: [PATCH] virtiofs_submounts.py test: Note on vmlinuz param

2021-02-18 Thread Max Reitz
On 12.02.21 19:58, Cleber Rosa wrote: On Fri, Feb 12, 2021 at 04:16:49PM +0100, Max Reitz wrote: From the cancel message, it is not entirely clear why this parameter is mandatory now, or that it will be optional in the future. Add such a more detailed explanation as a comment in the test

Re: [PATCH 3/7] block/qcow2: use compressed write cache

2021-02-18 Thread Max Reitz
On 11.02.21 13:53, Vladimir Sementsov-Ogievskiy wrote: 10.02.2021 20:11, Max Reitz wrote: On 29.01.21 17:50, Vladimir Sementsov-Ogievskiy wrote: Introduce a new option: compressed-cache-size, with default to 64 clusters (to be not less than 64 default max-workers for backup job). Signed-off

Re: [PATCH 2/7] block/qcow2: introduce cache for compressed writes

2021-02-18 Thread Max Reitz
On 11.02.21 13:49, Vladimir Sementsov-Ogievskiy wrote: you may jump first to my last inline answer 10.02.2021 20:07, Max Reitz wrote: On 29.01.21 17:50, Vladimir Sementsov-Ogievskiy wrote: Compressed writes and O_DIRECT are not friends: they works too slow, because compressed writes does many

Re: [PATCH] qsd: Document FUSE exports

2021-02-18 Thread Max Reitz
On 17.02.21 17:26, Eric Blake wrote: On 2/17/21 5:58 AM, Max Reitz wrote: Implementing FUSE exports required no changes to the storage daemon, so we forgot to document them there. Considering that both NBD and vhost-user-blk exports are documented in its man page (and NBD exports in its --help

[PATCH] qsd: Document FUSE exports

2021-02-17 Thread Max Reitz
Implementing FUSE exports required no changes to the storage daemon, so we forgot to document them there. Considering that both NBD and vhost-user-blk exports are documented in its man page (and NBD exports in its --help text), we should probably do the same for FUSE. Signed-off-by: Max Reitz

[PATCH] virtiofs_submounts.py test: Note on vmlinuz param

2021-02-12 Thread Max Reitz
>From the cancel message, it is not entirely clear why this parameter is mandatory now, or that it will be optional in the future. Add such a more detailed explanation as a comment in the test source file. Suggested-by: Alex Bennée Signed-off-by: Max Reitz --- I’ve uploaded a build of Li

Re: [PATCH 2/2] file-posix: Cache next hole

2021-02-12 Thread Max Reitz
On 12.02.21 11:25, Kevin Wolf wrote: Am 12.02.2021 um 10:14 hat Max Reitz geschrieben: On 11.02.21 21:38, Vladimir Sementsov-Ogievskiy wrote: 11.02.2021 20:22, Max Reitz wrote: We have repeatedly received reports that SEEK_HOLE and SEEK_DATA are slow on certain filesystems and/or under

Re: [PATCH 2/2] file-posix: Cache next hole

2021-02-12 Thread Max Reitz
On 11.02.21 21:00, Eric Blake wrote: On 2/11/21 11:22 AM, Max Reitz wrote: We have repeatedly received reports that SEEK_HOLE and SEEK_DATA are slow on certain filesystems and/or under certain circumstances. That is why we generally try to avoid it (which is why bdrv_co_block_status() has the

Re: [PATCH 1/2] block/mirror: Fix mirror_top's permissions

2021-02-12 Thread Max Reitz
On 12.02.21 10:04, Vladimir Sementsov-Ogievskiy wrote: 11.02.2021 20:22, Max Reitz wrote: mirror_top currently shares all permissions, and takes only the WRITE permission (if some parent has taken that permission, too). That is wrong, though; mirror_top is a filter, so it should take

Re: [PATCH 2/2] file-posix: Cache next hole

2021-02-12 Thread Max Reitz
On 11.02.21 21:38, Vladimir Sementsov-Ogievskiy wrote: 11.02.2021 20:22, Max Reitz wrote: We have repeatedly received reports that SEEK_HOLE and SEEK_DATA are slow on certain filesystems and/or under certain circumstances.  That is why we generally try to avoid it (which is why

[PATCH 1/2] block/mirror: Fix mirror_top's permissions

2021-02-11 Thread Max Reitz
all images in the backing chain, so the mirror job can take it for the target BB). Signed-off-by: Max Reitz --- block/mirror.c | 32 +--- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index 8e1ad6eceb..1edfc3cc14 100644

[PATCH 2/2] file-posix: Cache next hole

2021-02-11 Thread Max Reitz
that can be saved by special-cases zeroed areas, so focussing on images that are (nearly) fully allocated is more important. Signed-off-by: Max Reitz --- block/file-posix.c | 81 +- 1 file changed, 80 insertions(+), 1 deletion(-) diff --git a/block/file

[PATCH 0/2] file-posix: Cache next hole

2021-02-11 Thread Max Reitz
no permissions (but WRITE if necessary) and shares everything. That seems wrong. Patch 1 addresses that, so that patch 2 can actually do something when mirroring an image.) Max Reitz (2): block/mirror: Fix mirror_top's permissions file-posix: Cache next hole block/file-posix.c

Re: [PULL v4 14/27] io: add qio_channel_readv_full_all_eof & qio_channel_readv_full_all helpers

2021-02-11 Thread Max Reitz
On 10.02.21 10:26, Stefan Hajnoczi wrote: From: Elena Ufimtseva Adds qio_channel_readv_full_all_eof() and qio_channel_readv_full_all() to read both data and FDs. Refactors existing code to use these helpers. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannat

Re: [PATCH 2/7] block/qcow2: introduce cache for compressed writes

2021-02-10 Thread Max Reitz
On 29.01.21 17:50, Vladimir Sementsov-Ogievskiy wrote: Compressed writes and O_DIRECT are not friends: they works too slow, because compressed writes does many small unaligned to 512 writes. Let's introduce an internal cache, so that compressed writes may work well when O_DIRECT is on. Signed-o

Re: [PATCH 1/7] qemu/queue: add some useful QLIST_ and QTAILQ_ macros

2021-02-10 Thread Max Reitz
On 29.01.21 17:50, Vladimir Sementsov-Ogievskiy wrote: Add QLIST_FOREACH_FUNC_SAFE(), QTAILQ_FOREACH_FUNC_SAFE() and QTAILQ_POP_HEAD(), to be used in following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/qemu/queue.h | 14 ++ 1 file changed, 14 insertions(+)

Re: [PATCH 3/7] block/qcow2: use compressed write cache

2021-02-10 Thread Max Reitz
On 29.01.21 17:50, Vladimir Sementsov-Ogievskiy wrote: Introduce a new option: compressed-cache-size, with default to 64 clusters (to be not less than 64 default max-workers for backup job). Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 8 +++- block/qcow2.h

Re: [PATCH 0/7] qcow2: compressed write cache

2021-02-10 Thread Max Reitz
On 09.02.21 17:52, Denis V. Lunev wrote: On 2/9/21 5:47 PM, Max Reitz wrote: On 09.02.21 15:10, Vladimir Sementsov-Ogievskiy wrote: 09.02.2021 16:25, Max Reitz wrote: On 29.01.21 17:50, Vladimir Sementsov-Ogievskiy wrote: Hi all! I know, I have several series waiting for a resend, but I had

Re: [PATCH 0/7] qcow2: compressed write cache

2021-02-10 Thread Max Reitz
On 09.02.21 19:51, Vladimir Sementsov-Ogievskiy wrote: 09.02.2021 21:41, Denis V. Lunev wrote: On 2/9/21 9:36 PM, Vladimir Sementsov-Ogievskiy wrote: 09.02.2021 19:39, Vladimir Sementsov-Ogievskiy wrote: 09.02.2021 17:47, Max Reitz wrote: On 09.02.21 15:10, Vladimir Sementsov-Ogievskiy wrote

[PATCH] iotests: Consistent $IMGOPTS boundary matching

2021-02-10 Thread Max Reitz
orted_imgopts match the regex against $IMGOPTS plus a trailing space. Suggested-by: Eric Blake Signed-off-by: Max Reitz --- Supersedes "iotests: Fix unsupported_imgopts for refcount_bits", and can be reproduced in the same way: $ ./check -qcow2 -o refcount_bits=1 7 15 29 58 62 66 68 80

Re: [PATCH] iotests: Fix unsupported_imgopts for refcount_bits

2021-02-10 Thread Max Reitz
On 09.02.21 19:49, Eric Blake wrote: On 2/9/21 12:27 PM, Max Reitz wrote: Many _unsupported_imgopts lines for refcount_bits values use something like "refcount_bits=1[^0-9]" to forbid everything but "refcount_bits=1" (e.g. "refcount_bits=16" is allowed). That do

[PATCH] iotests: Fix unsupported_imgopts for refcount_bits

2021-02-09 Thread Max Reitz
ion, which now became apparent with the "check" script rewrite. Use \b instead of [^0-9] to check for a word boundary, which is what we really want. Signed-off-by: Max Reitz --- Reproducible with: $ ./check -qcow2 -o refcount_bits=1 (The tests touched here should be skipped) I don'

[PATCH] iotests/210: Fix reference output

2021-02-09 Thread Max Reitz
Commit 69b55e03f has changed an error message, adjust the reference output to account for it. Fixes: 69b55e03f7e65a36eb954d0b7d4698b258df2708 ("block: refactor bdrv_check_request: add errp") Signed-off-by: Max Reitz --- Fun fact: The branch name "fix-210-v1" w

Re: [RFC PATCH] block/null: Use 'read-zeroes' mode by default

2021-02-09 Thread Max Reitz
x55 || buf[511] != 0xaa) { return -1; } We could audit all the uninitialized buffers and the bdrv_co_preadv() handlers, but it is simpler to change the default of this testing driver. Performance tests will have to adapt and use 'null-co,read-zeroes=on'. Suggested-by: Max

Re: [PATCH 0/7] qcow2: compressed write cache

2021-02-09 Thread Max Reitz
On 09.02.21 15:10, Vladimir Sementsov-Ogievskiy wrote: 09.02.2021 16:25, Max Reitz wrote: On 29.01.21 17:50, Vladimir Sementsov-Ogievskiy wrote: Hi all! I know, I have several series waiting for a resend, but I had to switch to another task spawned from our customer's bug. Original pr

Re: [PATCH 07/22] tests/acceptance/virtiofs_submounts.py: evaluate string not length

2021-02-09 Thread Max Reitz
On 09.02.21 13:52, Alex Bennée wrote: Max Reitz writes: On 09.02.21 12:24, Alex Bennée wrote: Max Reitz writes: On 04.02.21 14:23, Alex Bennée wrote: Cleber Rosa writes: If the vmlinuz variable is set to anything that evaluates to True, then the respective arguments should be set

<    1   2   3   4   5   6   7   8   9   10   >