Re: [PATCH v5] iotests: Test NBD client reconnection

2019-11-14 Thread Andrey Shinkevich
On 14/11/2019 18:09, Eric Blake wrote: > On 11/11/19 9:39 PM, Andrey Shinkevich wrote: >> The test for an NBD client. The NBD server is disconnected after the >> client write request. The NBD client should reconnect and complete >> the write operation. >> >> Suggested-by: Denis V. Lunev >>

[PATCH v3 4/4] tests: More iotest 223 improvements

2019-11-14 Thread Eric Blake
Run the core of the test twice, once without iothreads, and again with, for more coverage of both setups. Suggested-by: Nir Soffer Signed-off-by: Eric Blake Reviewed-by: Max Reitz --- tests/qemu-iotests/223 | 16 ++- tests/qemu-iotests/223.out | 85

[PATCH v3 1/4] iotests: Fix 173

2019-11-14 Thread Eric Blake
This test has been broken since 3.0. It used TEST_IMG to influence the name of a file created during _make_test_img, but commit 655ae6bb changed things so that the wrong file name is being created, which then caused _launch_qemu to fail. In the meantime, the set of events issued for the actions

[PATCH v3 3/4] iotests: Include QMP input in .out files

2019-11-14 Thread Eric Blake
We generally include relevant HMP input in .out files, by virtue of the fact that HMP echoes its input. But QMP does not, so we have to explicitly inject it in the output stream (appropriately filtered to keep the tests passing), in order to make it easier to read .out files to see what behavior

[PATCH v3 2/4] iotests: Switch nbd tests to use Unix rather than TCP

2019-11-14 Thread Eric Blake
Up to now, all it took to cause a lot of iotest failures was to have a background process such as 'nbdkit -p 10810 null' running, because we hard-coded the TCP port. Switching to a Unix socket eliminates this contention. We still have TCP coverage in test 233, and that test is more careful to

Re: [PATCH v2 1/3] hw/block/pflash: Remove dynamic field width from trace events

2019-11-14 Thread Philippe Mathieu-Daudé
Hi Eric, On 11/8/19 4:56 PM, Eric Blake wrote: On 11/8/19 8:40 AM, Philippe Mathieu-Daudé wrote: Since not all trace backends support dynamic field width in format (dtrace via stap does not), replace by a static field width instead. Reported-by: Eric Blake Buglink:

Re: [PATCH v2 2/3] hw/mips/gt64xxx: Remove dynamic field width from trace events

2019-11-14 Thread Philippe Mathieu-Daudé
On 11/8/19 4:58 PM, Eric Blake wrote: On 11/8/19 8:40 AM, Philippe Mathieu-Daudé wrote: Since not all trace backends support dynamic field width in format (dtrace via stap does not), replace by a static field width instead. Reported-by: Eric Blake Buglink:

Re: [PATCH v2 00/10] Further bitmaps improvements

2019-11-14 Thread Vladimir Sementsov-Ogievskiy
14.11.2019 21:47, Eric Blake wrote: > On 10/22/19 7:58 AM, Vladimir Sementsov-Ogievskiy wrote: >> Hi! >> >> The main feature here is improvement of _next_dirty_area API, which I'm >> going to use then for backup / block-copy. >> >> v2: >> 01: just use INT64_MAX instead of adding new constant >>

Re: [PATCH v2 00/10] Further bitmaps improvements

2019-11-14 Thread Eric Blake
On 10/22/19 7:58 AM, Vladimir Sementsov-Ogievskiy wrote: Hi! The main feature here is improvement of _next_dirty_area API, which I'm going to use then for backup / block-copy. v2: 01: just use INT64_MAX instead of adding new constant 08: add separate function nbd_extent_array_convert_to_be and

Re: [QEMU-SECURITY] ide: fix assertion in ide_dma_cb() to prevent qemu DoS from quest

2019-11-14 Thread Alexander Popov
On 07.11.2019 01:05, Alexander Popov wrote: > On 06.11.2019 15:05, Michael S. Tsirkin wrote: >> Do you want to cook up a patch like this then? > > Yes, I will take this task and return with a patch. > > Thanks! I've just sent the v2 of the patch. Looking forward to your feedback. Best regards,

[PATCH v2 1/1] ide: check DMA transfer size in ide_dma_cb() to prevent qemu DoS from quests

2019-11-14 Thread Alexander Popov
The commit a718978ed58a from July 2015 introduced the assertion which implies that the size of successful DMA transfers handled in ide_dma_cb() should be multiple of 512 (the size of a sector). But guest systems can initiate DMA transfers that don't fit this requirement. PoC for Linux that uses

Re: Convert VMDK to RAW

2019-11-14 Thread Max Reitz
On 14.11.19 17:12, janine.schnei...@fau.de wrote: > Ladies and Gentlemen, > >   > > I am a PhD student at the Friedrich-Alexander-University > Erlangen-Nuremberg in Bavaria, Germany and am currently working on a > forensic reconstruction tool. The tool can be used to analyze physical > and

Convert VMDK to RAW

2019-11-14 Thread janine.schneider
Ladies and Gentlemen, I am a PhD student at the Friedrich-Alexander-University Erlangen-Nuremberg in Bavaria, Germany and am currently working on a forensic reconstruction tool. The tool can be used to analyze physical and virtual hard disks and to reconstruct files. I would now like to extend

Re: [PATCH for-4.2 v2 3/3] block/file-posix: Let post-EOF fallocate serialize

2019-11-14 Thread Max Reitz
On 14.11.19 18:15, Max Reitz wrote: > On 14.11.19 17:27, Christoph Hellwig wrote: >> On Fri, Nov 01, 2019 at 04:25:10PM +0100, Max Reitz wrote: >>> The XFS kernel driver has a bug that may cause data corruption for qcow2 >>> images as of qemu commit c8bb23cbdbe32f. We can work around it by >>>

Re: [PATCH for-4.2 v2 3/3] block/file-posix: Let post-EOF fallocate serialize

2019-11-14 Thread Max Reitz
On 14.11.19 17:27, Christoph Hellwig wrote: > On Fri, Nov 01, 2019 at 04:25:10PM +0100, Max Reitz wrote: >> The XFS kernel driver has a bug that may cause data corruption for qcow2 >> images as of qemu commit c8bb23cbdbe32f. We can work around it by >> treating post-EOF fallocates as serializing

Re: [RFC PATCH v2 20/26] qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()

2019-11-14 Thread Alberto Garcia
On Tue 05 Nov 2019 12:43:16 PM CET, Max Reitz wrote: > Speaking of handle_copied(); both elements of Qcow2COWRegion are of > type unsigned. handle_copied() doesn’t look like it takes any > precautions to limit the range to even UINT_MAX (and it should > probably limit it to INT_MAX). Or rather,

Re: [PATCH for-4.2 v2 3/3] block/file-posix: Let post-EOF fallocate serialize

2019-11-14 Thread Christoph Hellwig
On Fri, Nov 01, 2019 at 04:25:10PM +0100, Max Reitz wrote: > The XFS kernel driver has a bug that may cause data corruption for qcow2 > images as of qemu commit c8bb23cbdbe32f. We can work around it by > treating post-EOF fallocates as serializing up until infinity (INT64_MAX > in practice).

Re: [RFC PATCH v2 16/26] qcow2: Add subcluster support to discard_in_l2_slice()

2019-11-14 Thread Max Reitz
On 14.11.19 16:33, Alberto Garcia wrote: > On Mon 04 Nov 2019 04:07:35 PM CET, Max Reitz wrote: >>> /* First remove L2 entries */ >>> qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); >>> -if (!full_discard && s->qcow_version >= 3) { >>> +if

Re: [Qemu-devel] [PATCH v4 03/14] qapi: Introduce default values for struct members

2019-11-14 Thread Markus Armbruster
Less than thorough review, because I expect the necessary rebase will require a bit of rewriting here and there. Max Reitz writes: > With this change, it is possible to give default values for struct > members, as follows: > > What you had to do so far: > > # @member: Some description,

Re: [RFC PATCH v2 18/26] qcow2: Add subcluster support to expand_zero_clusters_in_l1()

2019-11-14 Thread Alberto Garcia
On Tue 05 Nov 2019 12:05:02 PM CET, Max Reitz wrote: >> @@ -2102,6 +2103,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState >> *bs, uint64_t *l1_table, >> } else { >> set_l2_entry(s, l2_slice, j, offset); >> } >> +

Re: [RFC PATCH v2 16/26] qcow2: Add subcluster support to discard_in_l2_slice()

2019-11-14 Thread Alberto Garcia
On Mon 04 Nov 2019 04:07:35 PM CET, Max Reitz wrote: >> /* First remove L2 entries */ >> qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); >> -if (!full_discard && s->qcow_version >= 3) { >> +if (has_subclusters(s)) { >> +set_l2_entry(s,

Re: [RFC PATCH v2 15/26] qcow2: Add subcluster support to zero_in_l2_slice()

2019-11-14 Thread Alberto Garcia
On Mon 04 Nov 2019 04:10:58 PM CET, Max Reitz wrote: >>> qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); >>> if (cluster_type == QCOW2_CLUSTER_COMPRESSED || unmap) { >>> -set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO); >>>

Re: [PATCH] MAINTAINERS: add more bitmap-related to Dirty Bitmaps section

2019-11-14 Thread Eric Blake
On 10/26/19 11:56 AM, Vladimir Sementsov-Ogievskiy wrote: Let's add bitmaps persistence qcow2 feature and postcopy bitmaps migration to Dirty Bitmaps section. Signed-off-by: Vladimir Sementsov-Ogievskiy --- MAINTAINERS | 2 ++ 1 file changed, 2 insertions(+) I see no reason why this

Re: [PATCH v5] iotests: Test NBD client reconnection

2019-11-14 Thread Eric Blake
On 11/11/19 9:39 PM, Andrey Shinkevich wrote: The test for an NBD client. The NBD server is disconnected after the client write request. The NBD client should reconnect and complete the write operation. Suggested-by: Denis V. Lunev Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by:

Re: [RFC PATCH v2 10/26] qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()

2019-11-14 Thread Alberto Garcia
On Wed 30 Oct 2019 05:55:04 PM CET, Max Reitz wrote: >> This patch also adds the get/set_l2_bitmap() functions that are used >> to access the bitmaps. For convenience, these functions are no-ops >> when used in traditional qcow2 images. > > Granted, I haven’t seen the following patches yet, but if

Re: [PATCH v3 1/4] nbd/server: Prefer heap over stack for parsing client names

2019-11-14 Thread Eric Blake
On 11/14/19 4:04 AM, Maxim Levitsky wrote: On Wed, 2019-11-13 at 20:46 -0600, Eric Blake wrote: As long as we limit NBD names to 256 bytes (the bare minimum permitted by the standard), stack-allocation works for parsing a name received from the client. But as mentioned in a comment, we

Re: [PATCH v6 22/42] block: Fix bdrv_get_allocated_file_size's fallback

2019-11-14 Thread Max Reitz
On 11.09.19 13:00, Max Reitz wrote: > On 11.09.19 12:31, Kevin Wolf wrote: >> Am 11.09.2019 um 12:00 hat Max Reitz geschrieben: >>> On 11.09.19 10:27, Kevin Wolf wrote: Am 11.09.2019 um 09:37 hat Max Reitz geschrieben: > On 11.09.19 08:55, Kevin Wolf wrote: >> Well, by default the

Re: [RFC PATCH 02/18] qemu-storage-daemon: Add --object option

2019-11-14 Thread Kevin Wolf
Am 07.11.2019 um 21:36 hat Markus Armbruster geschrieben: > Kevin Wolf writes: > > > Add a command line option to create user-creatable QOM objects. > > > > Signed-off-by: Kevin Wolf > > --- > > qemu-storage-daemon.c | 35 +++ > > 1 file changed, 35

Re: [Qemu-devel] [PATCH v4 01/14] qapi: Parse numeric values

2019-11-14 Thread Markus Armbruster
Max Reitz writes: > On 14.11.19 10:15, Markus Armbruster wrote: >> Max Reitz writes: >> >>> Signed-off-by: Max Reitz >>> --- >>> tests/qapi-schema/bad-type-int.json | 1 - >>> tests/qapi-schema/enum-int-member.json | 1 - >>> scripts/qapi/common.py | 25

Re: [PATCH v7 1/3] block: introduce compress filter driver

2019-11-14 Thread Vladimir Sementsov-Ogievskiy
14.11.2019 14:27, Max Reitz wrote: > On 13.11.19 19:43, Andrey Shinkevich wrote: >> Allow writing all the data compressed through the filter driver. >> The written data will be aligned by the cluster size. >> Based on the QEMU current implementation, that data can be written to >> unallocated

Re: [PATCH v7 1/3] block: introduce compress filter driver

2019-11-14 Thread Max Reitz
On 13.11.19 19:43, Andrey Shinkevich wrote: > Allow writing all the data compressed through the filter driver. > The written data will be aligned by the cluster size. > Based on the QEMU current implementation, that data can be written to > unallocated clusters only. May be used for a backup job.

Re: API definition for LUKS key management

2019-11-14 Thread Maxim Levitsky
On Tue, 2019-11-12 at 10:12 +0100, Kevin Wolf wrote: > Am 11.11.2019 um 19:34 hat Daniel P. Berrangé geschrieben: > > On Mon, Nov 11, 2019 at 05:58:20PM +0200, Maxim Levitsky wrote: > > > One of the concerns that was raised during the review was that amend > > > interface for luks that I propose

Re: API definition for LUKS key management

2019-11-14 Thread Maxim Levitsky
On Tue, 2019-11-12 at 11:02 +, Daniel P. Berrangé wrote: > On Tue, Nov 12, 2019 at 10:12:45AM +0100, Kevin Wolf wrote: > > Am 11.11.2019 um 19:34 hat Daniel P. Berrangé geschrieben: > > > On Mon, Nov 11, 2019 at 05:58:20PM +0200, Maxim Levitsky wrote: > > > > One of the concerns that was

Re: [RFC PATCH 00/18] Add qemu-storage-daemon

2019-11-14 Thread Kevin Wolf
Am 24.10.2019 um 15:55 hat Vladimir Sementsov-Ogievskiy geschrieben: > This reflects our idea of using qemu binary instead of qemu-img for doing > block-layer operations offline. > > What is the practical difference between qemu-storage-daemon and starting > qemu binary in stopped state? If I'm

Re: API definition for LUKS key management

2019-11-14 Thread Maxim Levitsky
On Mon, 2019-11-11 at 18:34 +, Daniel P. Berrangé wrote: > On Mon, Nov 11, 2019 at 05:58:20PM +0200, Maxim Levitsky wrote: > > Hi! > > > > I would like to discuss the API for LUKS key management. > > > > First of all very brief overview of LUKS v1 format: > > > > Each sector of the image is

Re: [PATCH v3 1/4] nbd/server: Prefer heap over stack for parsing client names

2019-11-14 Thread Maxim Levitsky
On Wed, 2019-11-13 at 20:46 -0600, Eric Blake wrote: > As long as we limit NBD names to 256 bytes (the bare minimum permitted > by the standard), stack-allocation works for parsing a name received > from the client. But as mentioned in a comment, we eventually want to > permit up to the 4k

Re: [PATCH v3 3/4] nbd: Don't send oversize strings

2019-11-14 Thread Maxim Levitsky
On Wed, 2019-11-13 at 20:46 -0600, Eric Blake wrote: > Qemu as server currently won't accept export names larger than 256 > bytes, nor create dirty bitmap names longer than 1023 bytes, so most > uses of qemu as client or server have no reason to get anywhere near > the NBD spec maximum of a 4k

Re: [PATCH v3 2/4] bitmap: Enforce maximum bitmap name length

2019-11-14 Thread Maxim Levitsky
On Wed, 2019-11-13 at 20:46 -0600, Eric Blake wrote: > We document that for qcow2 persistent bitmaps, the name cannot exceed > 1023 bytes. It is inconsistent if transient bitmaps do not have to > abide by the same limit, and it is unlikely that any existing client > even cares about using bitmap

Re: [Qemu-devel] [PATCH v4 02/14] qapi: Move to_c_string() to common.py

2019-11-14 Thread Max Reitz
On 14.11.19 10:20, Markus Armbruster wrote: > Max Reitz writes: > >> This function will be useful for code generation once we allow default >> values, so move it to the other "C helper functions". In the process, >> rewrite it so it supports all nonprintable and non-ASCII characters. >> >>

Re: [Qemu-devel] [PATCH v4 01/14] qapi: Parse numeric values

2019-11-14 Thread Max Reitz
On 14.11.19 10:15, Markus Armbruster wrote: > Max Reitz writes: > >> Signed-off-by: Max Reitz >> --- >> tests/qapi-schema/bad-type-int.json | 1 - >> tests/qapi-schema/enum-int-member.json | 1 - >> scripts/qapi/common.py | 25 >>

Re: [Qemu-devel] [PATCH v4 02/14] qapi: Move to_c_string() to common.py

2019-11-14 Thread Markus Armbruster
Max Reitz writes: > This function will be useful for code generation once we allow default > values, so move it to the other "C helper functions". In the process, > rewrite it so it supports all nonprintable and non-ASCII characters. > > Signed-off-by: Max Reitz Please have a close look at

Re: [Qemu-devel] [PATCH v4 01/14] qapi: Parse numeric values

2019-11-14 Thread Markus Armbruster
Max Reitz writes: > Signed-off-by: Max Reitz > --- > tests/qapi-schema/bad-type-int.json | 1 - > tests/qapi-schema/enum-int-member.json | 1 - > scripts/qapi/common.py | 25 > scripts/qapi/introspect.py | 2 ++ >

Re: [PATCH v4 00/14] block: Try to create well-typed json:{} filenames

2019-11-14 Thread Markus Armbruster
Max Reitz writes: > On 13.09.19 13:49, Max Reitz wrote: >> Another gentle ping. > > And another. Conflicts with the refactoring merged in commit 69717d0f890. Please accept my apologies for the inconvenience caused by the excessive delay. I'll try to review anyway.