[Qemu-block] [PATCH v11 06/10] file-posix: reset fallocate-related flags without CONFIG_FALLOCATE*

2018-12-17 Thread Anton Nefedov
these flags currently affect nothing without CONFIG_FALLOCATE*, so it's not a bug. Fixing it makes possible to adjust supported zero flag BDRV_REQ_ALLOCATE regardless of configuration (in the following patch). Signed-off-by: Anton Nefedov --- block/file-posix.c | 8 ++-- 1 file changed, 6 in

[Qemu-block] [PATCH v11 04/10] block: introduce BDRV_REQ_ALLOCATE flag

2018-12-17 Thread Anton Nefedov
The flag is supposed to indicate that the region of the disk image has to be sufficiently allocated so it reads as zeroes. The call with the flag set must return -ENOTSUP if allocation cannot be done efficiently. This has to be made sure of by both - the drivers that support the flag - and the

[Qemu-block] [PATCH v11 01/10] mirror: inherit supported write/zero flags

2018-12-17 Thread Anton Nefedov
Signed-off-by: Anton Nefedov Reviewed-by: Alberto Garcia Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/mirror.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index ab59ad77e8..be52c9be9c 100644 --- a/block/mirror.c +++ b/block/

[Qemu-block] sheepdog build warning

2018-12-17 Thread Michael S. Tsirkin
mingw32 build on fedora fails with this warning: /scm/qemu/block/sheepdog.c: In function 'find_vdi_name': /scm/qemu/block/sheepdog.c:1239:5: error: 'strncpy' specified bound 256 equals destination size [-Werror=stringop-truncation] strncpy(buf + SD_MAX_VDI_LEN, tag, SD_MAX_VDI_TAG_LEN);

Re: [Qemu-block] sheepdog build warning

2018-12-17 Thread Michael S. Tsirkin
On Mon, Dec 17, 2018 at 11:03:11PM -0500, Michael S. Tsirkin wrote: > mingw32 build on fedora fails with this warning: > > /scm/qemu/block/sheepdog.c: In function 'find_vdi_name': > /scm/qemu/block/sheepdog.c:1239:5: error: 'strncpy' specified bound 256 > equals destination size [-Werror=stringop

Re: [Qemu-block] Virtio-BLK/SCSI write requests and data payload checksums

2018-12-17 Thread Michael S. Tsirkin
On Mon, Dec 17, 2018 at 04:19:53PM +0100, Peter Lieven wrote: > Am 17.12.18 um 15:48 schrieb Stefan Hajnoczi: > > On Sun, Dec 16, 2018 at 06:53:44PM +0100, Peter Lieven wrote: > > > It turned out that for writes a bounce buffer is indeed always necessary. > > > But what I found out is that > > > i

[Qemu-block] [PATCH v12 30/31] block: BDS options may lack the "driver" option

2018-12-17 Thread Max Reitz
When BDSs are created by qemu itself (e.g. as filters in block jobs), they may not have a "driver" option in their options QDict. When generating a json:{} filename, however, it must always be present. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- block.c | 6 ++ 1 file changed,

[Qemu-block] [PATCH v12 29/31] block/null: Generate filename even with latency-ns

2018-12-17 Thread Max Reitz
While we cannot represent the latency-ns option in a filename, it is not a strong option so not being able to should not stop us from generating a filename nonetheless. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- block/null.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)

[Qemu-block] [PATCH v12 28/31] block/curl: Implement bdrv_refresh_filename()

2018-12-17 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- block/curl.c | 21 + 1 file changed, 21 insertions(+) diff --git a/block/curl.c b/block/curl.c index 02f9d9d692..1ccc2e92e1 100644 --- a/block/curl.c +++ b/block/curl.c @@ -963,6 +963,23 @@ static int64_t curl_getlengt

[Qemu-block] [PATCH v12 31/31] iotests: Test json:{} filenames of internal BDSs

2018-12-17 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- tests/qemu-iotests/224 | 139 + tests/qemu-iotests/224.out | 18 + tests/qemu-iotests/group | 1 + 3 files changed, 158 insertions(+) create mode 100755 tests/qemu-iotests/224 create mode

[Qemu-block] [PATCH v12 27/31] block/curl: Harmonize option defaults

2018-12-17 Thread Max Reitz
Both of the defaults we currently have in the curl driver are named based on a slightly different schema, let's unify that and call both CURL_BLOCK_OPT_${NAME}_DEFAULT. While at it, we can add a macro for the third option for which a default exists, namely "sslverify". Signed-off-by: Max Reitz R

[Qemu-block] [PATCH v12 26/31] block/nvme: Fix bdrv_refresh_filename()

2018-12-17 Thread Max Reitz
Currently, nvme's bdrv_refresh_filename() is an exact copy of null's implementation. However, for null, "null-co://" and "null-aio://" are indeed valid filenames -- for nvme, they are not, as a device address is still required. The correct implementation should generate a filename of the form "nv

[Qemu-block] [PATCH v12 25/31] block: Do not copy exact_filename from format file

2018-12-17 Thread Max Reitz
If a format BDS's file BDS is in turn a format BDS, we cannot simply use the same filename, because when opening a BDS tree based on a filename alone, qemu will create only one format node on top of one protocol node (disregarding a potential backing file). Signed-off-by: Max Reitz Reviewed-by: A

[Qemu-block] [PATCH v12 22/31] block: Add BlockDriver.bdrv_gather_child_options

2018-12-17 Thread Max Reitz
Some follow-up patches will rework the way bs->full_open_options is refreshed in bdrv_refresh_filename(). The new implementation will remove the need for the block drivers' bdrv_refresh_filename() implementations to set bs->full_open_options; instead, it will be generic and use static information f

[Qemu-block] [PATCH v12 20/31] iotests: Add quorum case to test 110

2018-12-17 Thread Max Reitz
Test 110 tests relative backing filenames for complex BDS trees. Now that the originally supposedly failing test passes, let us add a new failing test: Quorum can never work automatically (without detecting whether all child nodes have the same base directory, but that would be rather inconsistent

[Qemu-block] [PATCH v12 23/31] block: Generically refresh runtime options

2018-12-17 Thread Max Reitz
Instead of having every block driver which implements bdrv_refresh_filename() copy all of the strong runtime options over to bs->full_open_options, implement this process generically in bdrv_refresh_filename(). This patch only adds this new generic implementation, it does not remove the old functi

[Qemu-block] [PATCH v12 24/31] block: Purify .bdrv_refresh_filename()

2018-12-17 Thread Max Reitz
Currently, BlockDriver.bdrv_refresh_filename() is supposed to both refresh the filename (BDS.exact_filename) and set BDS.full_open_options. Now that we have generic code in the central bdrv_refresh_filename() for creating BDS.full_open_options, we can drop the latter part from all BlockDriver.bdrv_

[Qemu-block] [PATCH v12 19/31] block: Use bdrv_dirname() for relative filenames

2018-12-17 Thread Max Reitz
bdrv_get_full_backing_filename_from_filename() breaks down when it comes to JSON filenames. Using bdrv_dirname() as the basis is better because since we have BDS, we can descend through the BDS tree to the protocol layer, which gives us a greater probability of finding a non-JSON name; also, bdrv_d

[Qemu-block] [PATCH v12 21/31] block: Add strong_runtime_opts to BlockDriver

2018-12-17 Thread Max Reitz
This new field can be set by block drivers to list the runtime options they accept that may influence the contents of the respective BDS. As of a follow-up patch, this list will be used by the common bdrv_refresh_filename() implementation to decide which options to put into BDS.full_open_options (a

[Qemu-block] [PATCH v12 16/31] quorum: Make bdrv_dirname() return NULL

2018-12-17 Thread Max Reitz
While the common implementation for bdrv_dirname() should return NULL for quorum BDSs already (because they do not have a file node and their exact_filename field should be empty), there is no reason not to make that explicit. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto

[Qemu-block] [PATCH v12 17/31] block/nbd: Make bdrv_dirname() return NULL

2018-12-17 Thread Max Reitz
The generic bdrv_dirname() implementation would be able to generate some form of directory name for many NBD nodes, but it would be always wrong. Therefore, we have to explicitly make it an error (until NBD has some form of specification for export paths, if it ever will). Signed-off-by: Max Reitz

[Qemu-block] [PATCH v12 18/31] block/nfs: Implement bdrv_dirname()

2018-12-17 Thread Max Reitz
While the basic idea is obvious and could be handled by the default bdrv_dirname() implementation, we cannot generate a directory name if the gid or uid are set, so we have to explicitly return NULL in those cases. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- block/nfs.c | 15 ++

[Qemu-block] [PATCH v12 13/31] block: Fix bdrv_find_backing_image()

2018-12-17 Thread Max Reitz
bdrv_find_backing_image() should use bdrv_get_full_backing_filename() or bdrv_make_absolute_filename() instead of trying to do what those functions do by itself. path_combine_deprecated() can now be dropped, so let's do that. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- block.c | 3

[Qemu-block] [PATCH v12 14/31] block: Add bdrv_dirname()

2018-12-17 Thread Max Reitz
This function may be implemented by block drivers to derive a directory name from a BDS. Concatenating this g_free()-able string with a relative filename must result in a valid (not necessarily existing) filename, so this is a function that should generally be not implemented by format drivers, bec

[Qemu-block] [PATCH v12 15/31] blkverify: Make bdrv_dirname() return NULL

2018-12-17 Thread Max Reitz
blkverify's BDSs have a file BDS, but we do not want this to be preferred over the raw node. There is no way to decide between the two (and not really a reason to, either), so just return NULL in blkverify's implementation of bdrv_dirname(). Signed-off-by: Max Reitz Reviewed-by: Eric Blake Revie

[Qemu-block] [PATCH v12 12/31] block: Add bdrv_make_absolute_filename()

2018-12-17 Thread Max Reitz
This is a general function for making a filename that is relative to a certain BDS absolute. It calls bdrv_get_full_backing_filename_from_filename() for now, but that will be changed in a follow-up patch. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- block.c | 27 +++

[Qemu-block] [PATCH v12 09/31] block: Make path_combine() return the path

2018-12-17 Thread Max Reitz
Besides being safe for arbitrary path lengths, after some follow-up patches all callers will want a freshly allocated buffer anyway. In the meantime, path_combine_deprecated() is added which has the same interface as path_combine() had before this patch. All callers to that function will be conver

[Qemu-block] [PATCH v12 11/31] block: bdrv_get_full_backing_filename's ret. val.

2018-12-17 Thread Max Reitz
Make bdrv_get_full_backing_filename() return an allocated string instead of placing the result in a caller-provided buffer. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- include/block/block.h | 3 +-- block.c | 48 +++ block/qapi

[Qemu-block] [PATCH v12 10/31] block: bdrv_get_full_backing_filename_from_...'s ret. val.

2018-12-17 Thread Max Reitz
Make bdrv_get_full_backing_filename_from_filename() return an allocated string instead of placing the result in a caller-provided buffer. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- include/block/block.h | 7 +++--- block.c | 53 ++

[Qemu-block] [PATCH v12 08/31] iotests: Add test for backing file overrides

2018-12-17 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- tests/qemu-iotests/228 | 235 + tests/qemu-iotests/228.out | 84 + tests/qemu-iotests/group | 1 + 3 files changed, 320 insertions(+) create mode 100755 tests/qemu-iotests/228 crea

[Qemu-block] [PATCH v12 07/31] iotests.py: Add node_info()

2018-12-17 Thread Max Reitz
This function queries a node; since we cannot do that right now, it executes query-named-block-nodes and returns the matching node's object. Signed-off-by: Max Reitz Reviewed-by: John Snow Reviewed-by: Alberto Garcia Reviewed-by: Eric Blake --- tests/qemu-iotests/iotests.py | 7 +++ 1 fil

[Qemu-block] [PATCH v12 06/31] iotests.py: Add filter_imgfmt()

2018-12-17 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia --- tests/qemu-iotests/iotests.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index d537538ba0..0c1b8ad272 100644 --- a/tests/qemu-iotests/iotest

[Qemu-block] [PATCH v12 05/31] block: Respect backing bs in bdrv_refresh_filename

2018-12-17 Thread Max Reitz
Basically, bdrv_refresh_filename() should respect all children of a BlockDriverState. However, generally those children are driver-specific, so this function cannot handle the general case. On the other hand, there are only few drivers which use other children than @file and @backing (that being vm

[Qemu-block] [PATCH v12 02/31] block: Use children list in bdrv_refresh_filename

2018-12-17 Thread Max Reitz
bdrv_refresh_filename() should invoke itself recursively on all children, not just on file. With that change, we can remove the manual invocations in blkverify, quorum, commit, mirror, and blklogwrites. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia --- block.c

[Qemu-block] [PATCH v12 01/31] block: Use bdrv_refresh_filename() to pull

2018-12-17 Thread Max Reitz
Before this patch, bdrv_refresh_filename() is used in a pushing manner: Whenever the BDS graph is modified, the parents of the modified edges are supposed to be updated (recursively upwards). However, that is nonviable, considering that we want child changes not to concern parents. Also, in the l

[Qemu-block] [PATCH v12 04/31] block: Add BDS.auto_backing_file

2018-12-17 Thread Max Reitz
If the backing file is overridden, this most probably does change the guest-visible data of a BDS. Therefore, we will need to consider this in bdrv_refresh_filename(). To see whether it has been overridden, we might want to compare bs->backing_file and bs->backing->bs->filename. However, bs->bac

[Qemu-block] [PATCH v12 03/31] block: Skip implicit nodes for filename info

2018-12-17 Thread Max Reitz
bdrv_refresh_filename() should simply skip all implicit nodes. They are supposed to be invisible to the user, so they should not appear in filename information. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia --- block.c | 14 ++ 1 file changed, 14 ins

[Qemu-block] [PATCH v12 00/31] block: Fix some filename generation issues

2018-12-17 Thread Max Reitz
Once more, I’ll spare both me and you another iteration of the cover letter, so I direct you to the previous version’s cover letter (which will direct you further): http://lists.nongnu.org/archive/html/qemu-block/2018-10/msg00229.html There are only minor changes in this version. Provided no maj

Re: [Qemu-block] [PATCH v3 7/7] iotests: add iotest 236 for testing bitmap merge

2018-12-17 Thread John Snow
On 12/17/18 10:48 AM, Vladimir Sementsov-Ogievskiy wrote: > 15.12.2018 2:15, John Snow wrote: >> New interface, new smoke test. >> --- >> tests/qemu-iotests/236 | 124 +++ >> tests/qemu-iotests/236.out | 200 + >> tests/qemu-iotests

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/7] bitmaps: remove x- prefix from QMP api

2018-12-17 Thread John Snow
On 12/14/18 6:15 PM, John Snow wrote: > Touch up a few last things and remove the x- prefix. > > V3: > - Reworked qmp_log to pretty-print the outgoing command, too [Vladimir] > - Modified test to log only bitmap information [Vladimir] > - Test disable/enable transaction toggle [Eric] > > No

Re: [Qemu-block] [PATCH v3 6/7] iotests: allow pretty-print for qmp_log

2018-12-17 Thread John Snow
On 12/17/18 11:37 AM, Vladimir Sementsov-Ogievskiy wrote: > 17.12.2018 17:26, Vladimir Sementsov-Ogievskiy wrote: >> 15.12.2018 2:15, John Snow wrote: >>> If iotests have lines exceeding >998 characters long, git doesn't >>> want to send it plaintext to the list. We can solve this by allowing >>

[Qemu-block] [PATCH] iotests.py: rework log and qmp_log

2018-12-17 Thread Vladimir Sementsov-Ogievskiy
There are several problems to be solved: 1. log() may try to apply filters (which should be str -> str functions) to objects, which is not correct. Let's instead do json.dumps first and then apply filters. 2. Let's drop trailing whitespaces, if there are any. Hopefully, now only two test

[Qemu-block] [QEMU-devel][PATCH v3] aio-posix: Fix concurrent aio_poll/set_fd_handler.

2018-12-17 Thread remy . noel
From: Remy Noel It is possible for an io_poll callback to be concurrently executed along with an aio_set_fd_handlers. This can cause all sorts of problems, like a NULL callback or a bad opaque pointer. This changes set_fd_handlers so that it no longer modify existing handlers entries and instead

Re: [Qemu-block] [PATCH v3 6/7] iotests: allow pretty-print for qmp_log

2018-12-17 Thread Vladimir Sementsov-Ogievskiy
17.12.2018 17:26, Vladimir Sementsov-Ogievskiy wrote: > 15.12.2018 2:15, John Snow wrote: >> If iotests have lines exceeding >998 characters long, git doesn't >> want to send it plaintext to the list. We can solve this by allowing >> the iotests to use pretty printed QMP output that we can match ag

Re: [Qemu-block] [PATCH v3 7/7] iotests: add iotest 236 for testing bitmap merge

2018-12-17 Thread Vladimir Sementsov-Ogievskiy
15.12.2018 2:15, John Snow wrote: > New interface, new smoke test. > --- > tests/qemu-iotests/236 | 124 +++ > tests/qemu-iotests/236.out | 200 + > tests/qemu-iotests/group | 1 + > 3 files changed, 325 insertions(+) > create m

Re: [Qemu-block] [PATCH v2 18/22] nbd/client: Add nbd_receive_export_list()

2018-12-17 Thread Eric Blake
On 12/15/18 9:42 AM, Richard W.M. Jones wrote: On Sat, Dec 15, 2018 at 07:53:20AM -0600, Eric Blake wrote: We want to be able to detect whether a given qemu NBD server is exposing the right export(s) and dirty bitmaps, at least for regression testing. We could use 'nbd-client -l' from the upstr

Re: [Qemu-block] [PATCH v2 13/22] nbd/client: Split out nbd_send_one_meta_context()

2018-12-17 Thread Eric Blake
On 12/15/18 9:22 AM, Richard W.M. Jones wrote: On Sat, Dec 15, 2018 at 07:53:15AM -0600, Eric Blake wrote: Refactor nbd_negotiate_simple_meta_context() to pull out the code that can be reused to send a LIST request for 0 or 1 query. No semantic change. Signed-off-by: Eric Blake --- v2: split

Re: [Qemu-block] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context()

2018-12-17 Thread Eric Blake
On 12/15/18 9:19 AM, Richard W.M. Jones wrote: On Sat, Dec 15, 2018 at 07:53:14AM -0600, Eric Blake wrote: Always allocate space for the reply returned by the server and hoist the trace earlier, as it is more interesting to trace the server's reply (even if it is unexpected) than parroting our r

Re: [Qemu-block] [PATCH v2 12/22] nbd/client: Improve error handling in nbd_negotiate_simple_meta_context()

2018-12-17 Thread Eric Blake
On 12/15/18 9:19 AM, Richard W.M. Jones wrote: On Sat, Dec 15, 2018 at 07:53:14AM -0600, Eric Blake wrote: Always allocate space for the reply returned by the server and hoist the trace earlier, as it is more interesting to trace the server's reply (even if it is unexpected) than parroting our r

Re: [Qemu-block] Virtio-BLK/SCSI write requests and data payload checksums

2018-12-17 Thread Peter Lieven
Am 17.12.18 um 15:48 schrieb Stefan Hajnoczi: On Sun, Dec 16, 2018 at 06:53:44PM +0100, Peter Lieven wrote: It turned out that for writes a bounce buffer is indeed always necessary. But what I found out is that it seems that even for reads it happens that the OS (Windows in this case) issues 2

Re: [Qemu-block] [PATCH v2 04/22] qemu-nbd: Enhance man page

2018-12-17 Thread Eric Blake
On 12/15/18 8:13 AM, Richard W.M. Jones wrote: On Sat, Dec 15, 2018 at 07:53:06AM -0600, Eric Blake wrote: Document some useful qemu-nbd command lines. Mention some restrictions on particular options, like -p being only for MBR images, or -c/-d being Linux-only. Update some text given the recen

Re: [Qemu-block] Virtio-BLK/SCSI write requests and data payload checksums

2018-12-17 Thread Stefan Hajnoczi
On Sun, Dec 16, 2018 at 06:53:44PM +0100, Peter Lieven wrote: > It turned out that for writes a bounce buffer is indeed always necessary. But > what I found out is that > it seems that even for reads it happens that the OS (Windows in this case) > issues 2 read requests with > the same buffer in

Re: [Qemu-block] [PATCH v3 5/7] iotests: add filter_generated_node_ids

2018-12-17 Thread Vladimir Sementsov-Ogievskiy
15.12.2018 2:15, John Snow wrote: > To mimic the common filter of the same name, but for the python tests. > > Signed-off-by: John Snow '>' and also, strange CC line in header, including "John Snow , John" > Reviewed-by: Eric Blake > Reviewed-by: Vladimir Sementsov-Ogievskiy > --- > tests/

Re: [Qemu-block] [PATCH v3 6/7] iotests: allow pretty-print for qmp_log

2018-12-17 Thread Vladimir Sementsov-Ogievskiy
15.12.2018 2:15, John Snow wrote: > If iotests have lines exceeding >998 characters long, git doesn't > want to send it plaintext to the list. We can solve this by allowing > the iotests to use pretty printed QMP output that we can match against > instead. > > As a bonus, it's much nicer for human

Re: [Qemu-block] [PATCH 2/2] avoid TABs in files that only contain a few

2018-12-17 Thread Stefan Markovic
On 13.12.18. 23:37, Paolo Bonzini wrote: > Most files that have TABs only contain a handful of them. Change > them to spaces so that we don't confuse people. > > disas, standard-headers, linux-headers and libdecnumber are imported > from other projects and probably should be exempted from the che

[Qemu-block] [PATCH v6 15/18] xen: add a mechanism to automatically create XenDevice-s...

2018-12-17 Thread Paul Durrant
...that maintains compatibility with existing Xen toolstacks. Xen toolstacks instantiate PV backends by simply writing information into xenstore and expecting a backend implementation to be watching for this. This patch adds a new 'xen-backend' module to allow individual XenDevice implementations

[Qemu-block] [PATCH v6 11/18] xen: remove 'XenBlkDev' and 'blkdev' names from dataplane/xen-block

2018-12-17 Thread Paul Durrant
This is a purely cosmetic patch that substitutes the old 'struct XenBlkDev' name with 'XenBlockDataPlane' and 'blkdev' field/variable names with 'dataplane', and then does necessary fix-up to adhere to coding style. No functional change. Signed-off-by: Paul Durrant Acked-by: Anthony Perard ---

[Qemu-block] [PATCH v6 13/18] xen: purge 'blk' and 'ioreq' from function names in dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
This is a purely cosmetic patch that purges remaining use of 'blk' and 'ioreq' in local function names, and then makes sure all functions are prefixed with 'xen_block_'. No functional change. Signed-off-by: Paul Durrant Acked-by: Anthony Perard --- Cc: Stefano Stabellini Cc: Stefan Hajnoczi C

[Qemu-block] [PATCH v6 10/18] xen: add header and build dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
This patch adds the transformations necessary to get dataplane/xen-block.c to build against the new XenBus/XenDevice framework. MAINTAINERS is also updated due to the introduction of dataplane/xen-block.h. NOTE: Existing data structure names are retained for the moment. These will be modifie

[Qemu-block] [PATCH v6 14/18] xen: add implementations of xen-block connect and disconnect functions...

2018-12-17 Thread Paul Durrant
...and wire in the dataplane. This patch adds the remaining code to make the xen-block XenDevice functional. The parameters that a block frontend expects to find are populated in the backend xenstore area, and the 'ring-ref' and 'event-channel' values specified in the frontend xenstore area are ma

[Qemu-block] [PATCH v6 17/18] MAINTAINERS: add myself as a Xen maintainer

2018-12-17 Thread Paul Durrant
I have made many significant contributions to the Xen code in QEMU, particularly the recent patches introducing a new PV device framework. I intend to make further significant contributions, porting other PV back- ends to the new framework with the intent of eventually removing the legacy code. It

[Qemu-block] [PATCH v6 16/18] xen: automatically create XenBlockDevice-s

2018-12-17 Thread Paul Durrant
This patch adds create and destroy function for XenBlockDevice-s so that they can be created automatically when the Xen toolstack instantiates a new PV backend via xenstore. When the XenBlockDevice is created this way it is also necessary to create a 'drive' which matches the configuration that the

[Qemu-block] [PATCH v6 12/18] xen: remove 'ioreq' struct/varable/field names from dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
This is a purely cosmetic patch that purges the name 'ioreq' from struct, variable and field names. (This name has been problematic for a long time as 'ioreq' is the name used for generic I/O requests coming from Xen). The patch replaces 'struct ioreq' with a new 'XenBlockRequest' type and 'ioreq'

[Qemu-block] [PATCH v6 18/18] xen: remove the legacy 'xen_disk' backend

2018-12-17 Thread Paul Durrant
This backend has now been replaced by the 'xen-qdisk' XenDevice. Signed-off-by: Paul Durrant Acked-by: Anthony Perard --- Cc: Kevin Wolf Cc: Max Reitz Cc: Stefano Stabellini --- hw/block/Makefile.objs |1 - hw/block/xen_disk.c| 1011 2 files c

Re: [Qemu-block] [PATCH v2 0/8] qcow2: encryption threads

2018-12-17 Thread Vladimir Sementsov-Ogievskiy
11.12.2018 19:43, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > The series brings threads to qcow2 encryption/decryption path, > like it is already done for compression. > > v2: - multiple cipher inside QCryptoBlock instead of multiple >blocks inside qcow2, as suggested by Daniel, and

[Qemu-block] [PATCH v6 09/18] xen: remove unnecessary code from dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
Not all of the code duplicated from xen_disk.c is required as the basis for the new dataplane implementation so this patch removes extraneous code, along with the legacy #includes and calls to the legacy xen_pv_printf() function. Error messages are changed to be reported using error_report(). NOTE

[Qemu-block] [PATCH v6 01/18] xen: re-name XenDevice to XenLegacyDevice...

2018-12-17 Thread Paul Durrant
...and xen_backend.h to xen-legacy-backend.h Rather than attempting to convert the existing backend infrastructure to be QOM compliant (which would be hard to do in an incremental fashion), subsequent patches will introduce a completely new framework for Xen PV backends. Hence it is necessary to r

[Qemu-block] [PATCH v6 05/18] xen: add xenstore watcher infrastructure

2018-12-17 Thread Paul Durrant
A Xen PV frontend communicates its state to the PV backend by writing to the 'state' key in the frontend area in xenstore. It is therefore necessary for a XenDevice implementation to be notified whenever the value of this key changes. This patch adds code to do this as follows: - an 'fd handler'

[Qemu-block] [PATCH v6 06/18] xen: add grant table interface for XenDevice-s

2018-12-17 Thread Paul Durrant
The legacy PV backend infrastructure provides functions to map, unmap and copy pages granted by frontends. Similar functionality will be required by XenDevice implementations so this patch adds the necessary support. Signed-off-by: Paul Durrant Reviewed-by: Anthony Perard --- Cc: Stefano Stabell

[Qemu-block] [PATCH v6 03/18] xen: introduce 'xen-block', 'xen-disk' and 'xen-cdrom'

2018-12-17 Thread Paul Durrant
This patch adds new XenDevice-s: 'xen-disk' and 'xen-cdrom', both derived from a common 'xen-block' parent type. These will eventually replace the 'xen_disk' (note the underscore rather than hyphen) legacy PV backend but it is illustrative to build up the implementation incrementally, along with th

[Qemu-block] [PATCH v6 08/18] xen: duplicate xen_disk.c as basis of dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
The new xen-block XenDevice implementation requires the same core dataplane as the legacy xen_disk implementation it will eventually replace. This patch therefore copies the legacy xen_disk.c source module into a new dataplane/xen-block.c source module as the basis for the new dataplane and adjusts

[Qemu-block] [PATCH v6 07/18] xen: add event channel interface for XenDevice-s

2018-12-17 Thread Paul Durrant
The legacy PV backend infrastructure provides functions to bind, unbind and send notifications to event channnels. Similar functionality will be required by XenDevice implementations so this patch adds the necessary support. Signed-off-by: Paul Durrant Reviewed-by: Anthony Perard --- Cc: Stefano

[Qemu-block] [PATCH v6 00/18] Xen PV backend 'qdevification'

2018-12-17 Thread Paul Durrant
This series introduces a new QOM compliant framework for Xen PV backends. This is achieved by first moving the current non-compliant framework aside, before building up a new framework incrementally. This series was prompted by a thread [1] started by Kevin Wolf in response to patches against xen_

[Qemu-block] [PATCH v6 02/18] xen: introduce new 'XenBus' and 'XenDevice' object hierarchy

2018-12-17 Thread Paul Durrant
This patch adds the basic boilerplate for a 'XenBus' object that will act as a parent to 'XenDevice' PV backends. A new 'XenBridge' object is also added to connect XenBus to the system bus. The XenBus object is instantiated by a new xen_bus_init() function called from the same sites as the legacy

[Qemu-block] [PATCH v6 04/18] xen: create xenstore areas for XenDevice-s

2018-12-17 Thread Paul Durrant
This patch adds a new source module, xen-bus-helper.c, which builds on basic libxenstore primitives to provide functions to create (setting permissions appropriately) and destroy xenstore areas, and functions to 'printf' and 'scanf' nodes therein. The main xen-bus code then uses these primitives [1

Re: [Qemu-block] [PATCH] qemu: avoid memory leak while remove disk

2018-12-17 Thread wangjian
ok, I will send the patch by git send-email. On 12/15/2018 4:22 AM, Michael S. Tsirkin wrote: On Fri, Dec 07, 2018 at 09:53:06AM +0800, wangjian wrote: Memset vhost_dev to zero in the vhost_dev_cleanup function. This causes dev.vqs to be NULL, so that vqs does not free up space when calling the

Re: [Qemu-block] [Xen-devel] [PATCH v5 09/18] xen: remove unnecessary code from dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
> -Original Message- > From: Anthony PERARD [mailto:anthony.per...@citrix.com] > Sent: 17 December 2018 12:28 > To: Paul Durrant > Cc: qemu-de...@nongnu.org; qemu-block@nongnu.org; xen- > de...@lists.xenproject.org; Kevin Wolf ; Stefano > Stabellini ; Stefan Hajnoczi > ; Max Reitz > Subje

Re: [Qemu-block] [Xen-devel] [PATCH v5 09/18] xen: remove unnecessary code from dataplane/xen-block.c

2018-12-17 Thread Anthony PERARD
On Mon, Dec 17, 2018 at 11:40:39AM +, Paul Durrant wrote: > Not all of the code duplicated from xen_disk.c is required as the basis for > the new dataplane implementation so this patch removes extraneous code, > along with the legacy #includes and calls to the legacy xen_pv_printf() > function.

[Qemu-block] [PATCH v5 14/18] xen: add implementations of xen-block connect and disconnect functions...

2018-12-17 Thread Paul Durrant
...and wire in the dataplane. This patch adds the remaining code to make the xen-block XenDevice functional. The parameters that a block frontend expects to find are populated in the backend xenstore area, and the 'ring-ref' and 'event-channel' values specified in the frontend xenstore area are ma

[Qemu-block] [PATCH v5 13/18] xen: purge 'blk' and 'ioreq' from function names in dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
This is a purely cosmetic patch that purges remaining use of 'blk' and 'ioreq' in local function names, and then makes sure all functions are prefixed with 'xen_block_'. No functional change. Signed-off-by: Paul Durrant Acked-by: Anthony Perard --- Cc: Stefano Stabellini Cc: Stefan Hajnoczi C

[Qemu-block] [PATCH v5 18/18] xen: remove the legacy 'xen_disk' backend

2018-12-17 Thread Paul Durrant
This backend has now been replaced by the 'xen-qdisk' XenDevice. Signed-off-by: Paul Durrant Acked-by: Anthony Perard --- Cc: Kevin Wolf Cc: Max Reitz Cc: Stefano Stabellini --- hw/block/Makefile.objs |1 - hw/block/xen_disk.c| 1011 2

[Qemu-block] [PATCH v5 10/18] xen: add header and build dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
This patch adds the transformations necessary to get dataplane/xen-block.c to build against the new XenBus/XenDevice framework. MAINTAINERS is also updated due to the introduction of dataplane/xen-block.h. NOTE: Existing data structure names are retained for the moment. These will be modifie

[Qemu-block] [PATCH v5 17/18] MAINTAINERS: add myself as a Xen maintainer

2018-12-17 Thread Paul Durrant
I have made many significant contributions to the Xen code in QEMU, particularly the recent patches introducing a new PV device framework. I intend to make further significant contributions, porting other PV back- ends to the new framework with the intent of eventually removing the legacy code. It

[Qemu-block] [PATCH v5 15/18] xen: add a mechanism to automatically create XenDevice-s...

2018-12-17 Thread Paul Durrant
...that maintains compatibility with existing Xen toolstacks. Xen toolstacks instantiate PV backends by simply writing information into xenstore and expecting a backend implementation to be watching for this. This patch adds a new 'xen-backend' module to allow individual XenDevice implementations

[Qemu-block] [PATCH v5 11/18] xen: remove 'XenBlkDev' and 'blkdev' names from dataplane/xen-block

2018-12-17 Thread Paul Durrant
This is a purely cosmetic patch that substitutes the old 'struct XenBlkDev' name with 'XenBlockDataPlane' and 'blkdev' field/variable names with 'dataplane', and then does necessary fix-up to adhere to coding style. No functional change. Signed-off-by: Paul Durrant Acked-by: Anthony Perard ---

[Qemu-block] [PATCH v5 12/18] xen: remove 'ioreq' struct/varable/field names from dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
This is a purely cosmetic patch that purges the name 'ioreq' from struct, variable and field names. (This name has been problematic for a long time as 'ioreq' is the name used for generic I/O requests coming from Xen). The patch replaces 'struct ioreq' with a new 'XenBlockRequest' type and 'ioreq'

[Qemu-block] [PATCH v5 16/18] xen: automatically create XenBlockDevice-s

2018-12-17 Thread Paul Durrant
This patch adds create and destroy function for XenBlockDevice-s so that they can be created automatically when the Xen toolstack instantiates a new PV backend via xenstore. When the XenBlockDevice is created this way it is also necessary to create a 'drive' which matches the configuration that the

[Qemu-block] [PATCH v5 05/18] xen: add xenstore watcher infrastructure

2018-12-17 Thread Paul Durrant
A Xen PV frontend communicates its state to the PV backend by writing to the 'state' key in the frontend area in xenstore. It is therefore necessary for a XenDevice implementation to be notified whenever the value of this key changes. This patch adds code to do this as follows: - an 'fd handler'

[Qemu-block] [PATCH v5 08/18] xen: duplicate xen_disk.c as basis of dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
The new xen-block XenDevice implementation requires the same core dataplane as the legacy xen_disk implementation it will eventually replace. This patch therefore copies the legacy xen_disk.c source module into a new dataplane/xen-block.c source module as the basis for the new dataplane and adjusts

[Qemu-block] [PATCH v5 04/18] xen: create xenstore areas for XenDevice-s

2018-12-17 Thread Paul Durrant
This patch adds a new source module, xen-bus-helper.c, which builds on basic libxenstore primitives to provide functions to create (setting permissions appropriately) and destroy xenstore areas, and functions to 'printf' and 'scanf' nodes therein. The main xen-bus code then uses these primitives [1

[Qemu-block] [PATCH v5 09/18] xen: remove unnecessary code from dataplane/xen-block.c

2018-12-17 Thread Paul Durrant
Not all of the code duplicated from xen_disk.c is required as the basis for the new dataplane implementation so this patch removes extraneous code, along with the legacy #includes and calls to the legacy xen_pv_printf() function. Error messages are changed to be reported using error_report(). NOTE

[Qemu-block] [PATCH v5 03/18] xen: introduce 'xen-block', 'xen-disk' and 'xen-cdrom'

2018-12-17 Thread Paul Durrant
This patch adds new XenDevice-s: 'xen-disk' and 'xen-cdrom', both derived from a common 'xen-block' parent type. These will eventually replace the 'xen_disk' (note the underscore rather than hyphen) legacy PV backend but it is illustrative to build up the implementation incrementally, along with th

[Qemu-block] [PATCH v5 01/18] xen: re-name XenDevice to XenLegacyDevice...

2018-12-17 Thread Paul Durrant
...and xen_backend.h to xen-legacy-backend.h Rather than attempting to convert the existing backend infrastructure to be QOM compliant (which would be hard to do in an incremental fashion), subsequent patches will introduce a completely new framework for Xen PV backends. Hence it is necessary to r

[Qemu-block] [PATCH v5 00/18] Xen PV backend 'qdevification'

2018-12-17 Thread Paul Durrant
This series introduces a new QOM compliant framework for Xen PV backends. This is achieved by first moving the current non-compliant framework aside, before building up a new framework incrementally. This series was prompted by a thread [1] started by Kevin Wolf in response to patches against xen_

[Qemu-block] [PATCH v5 06/18] xen: add grant table interface for XenDevice-s

2018-12-17 Thread Paul Durrant
The legacy PV backend infrastructure provides functions to map, unmap and copy pages granted by frontends. Similar functionality will be required by XenDevice implementations so this patch adds the necessary support. Signed-off-by: Paul Durrant Reviewed-by: Anthony Perard --- Cc: Stefano Stabell

[Qemu-block] [PATCH v5 07/18] xen: add event channel interface for XenDevice-s

2018-12-17 Thread Paul Durrant
The legacy PV backend infrastructure provides functions to bind, unbind and send notifications to event channnels. Similar functionality will be required by XenDevice implementations so this patch adds the necessary support. Signed-off-by: Paul Durrant Reviewed-by: Anthony Perard --- Cc: Stefano

[Qemu-block] [PATCH v5 02/18] xen: introduce new 'XenBus' and 'XenDevice' object hierarchy

2018-12-17 Thread Paul Durrant
This patch adds the basic boilerplate for a 'XenBus' object that will act as a parent to 'XenDevice' PV backends. A new 'XenBridge' object is also added to connect XenBus to the system bus. The XenBus object is instantiated by a new xen_bus_init() function called from the same sites as the legacy

Re: [Qemu-block] [PATCH v10 8/9] qcow2: skip writing zero buffers to empty COW areas

2018-12-17 Thread Anton Nefedov
On 14/12/2018 7:20 PM, Vladimir Sementsov-Ogievskiy wrote: > 03.12.2018 13:14, Anton Nefedov wrote: >> If COW areas of the newly allocated clusters are zeroes on the backing image, >> efficient bdrv_write_zeroes(flags=BDRV_REQ_ALLOCATE) can be used on the whole >> cluster instead of writing expli

Re: [Qemu-block] [Qemu-devel] [PATCH v2 6/7] iotests: allow pretty-print for qmp_log

2018-12-17 Thread Vladimir Sementsov-Ogievskiy
14.12.2018 23:51, John Snow wrote: > > > On 12/13/18 8:09 AM, Vladimir Sementsov-Ogievskiy wrote: >> 13.12.2018 4:50, John Snow wrote: >>> If iotests have lines exceeding >998 characters long, git doesn't >>> want to send it plaintext to the list. We can solve this by allowing >>> the iotests to