Re: [RFC PATCH 01/22] nbd: Remove unused nbd_export_get_blockdev()

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > include/block/nbd.h | 2 -- > nbd/server.c| 5 - > 2 files changed, 7 deletions(-) Reviewed-by: Max Reitz signature.asc Description: OpenPGP digital signature

Re: [RFC PATCH 02/22] qapi: Create block-export module

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > Move all block export related types and commands from block-core to the > new QAPI module block-export. > > Signed-off-by: Kevin Wolf > --- > qapi/block-core.json | 166 -- > qapi/block-export.json | 172

Re: [PATCH 10/12] block/file-posix: fix a possible undefined behavior

2020-08-17 Thread Stefano Garzarella
On Fri, Aug 14, 2020 at 12:02:39PM -0400, Pan Nengyuan wrote: > local_err is not initialized to NULL, it will cause a assert error as below: > qemu/util/error.c:59: error_setv: Assertion `*errp == NULL' failed. > > Fixes: c6447510690 > Reported-by: Euler Robot > Signed-off-by: Pan Nengyuan > ---

[PATCH v3 06/12] block: introduce BDRV_REQ_NO_WAIT flag

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
Add flag to make serialising request no wait: if there are conflicting requests, just return error immediately. It's will be used in upcoming preallocate filter. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 9 - block/io.c| 13 - 2 file

[PATCH v3 04/12] block/io: bdrv_wait_serialising_requests_locked: drop extra bs arg

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
bs is linked in req, so no needs to pass it separately. Most of tracked-requests API doesn't have bs argument. Actually, after this patch only tracked_request_begin has it, but it's for purpose. While being here, also add a comment about what "_locked" is. Signed-off-by: Vladimir Sementsov-Ogievs

[PATCH v3 01/12] block: simplify comment to BDRV_REQ_SERIALISING

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
1. BDRV_REQ_NO_SERIALISING doesn't exist already, don't mention it. 2. We are going to add one more user of BDRV_REQ_SERIALISING, so comment about backup becomes a bit confusing here. The use case in backup is documented in block/backup.c, so let's just drop duplication here. 3. The fact

Re: [RFC PATCH 03/22] qapi: Rename BlockExport to BlockExportOptions

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > The name BlockExport will be used for the struct containing the runtime > state of block exports, so change the name of export creation options. > > Signed-off-by: Kevin Wolf > --- > qapi/block-export.json | 12 ++-- > block/monitor/block-hm

Re: [PATCH 0/7] block: Use definitions instead of magic values

2020-08-17 Thread Stefano Garzarella
On Fri, Aug 14, 2020 at 10:28:34AM +0200, Philippe Mathieu-Daudé wrote: > Trivial block patches: > - Fix a typo > - Replace '1 << 30' by '1 * GiB' in null-co > - Replace 512 by BDRV_SECTOR_SIZE when appropriate. > > Philippe Mathieu-Daudé (7): > block/null: Make more explicit the driver defaul

[PATCH v3 00/12] preallocate filter

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
Hi all! Here is a filter, which does preallocation on write. In Virtuozzo we have to deal with some custom distributed storage solution, where allocation is relatively expensive operation. We have to workaround it in Qemu, so here is a new filter. Patches 1-10 introduces the new filter and sugge

[PATCH v3 03/12] block/io: split out bdrv_find_conflicting_request

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
To be reused in separate. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/io.c | 71 +++--- 1 file changed, 41 insertions(+), 30 deletions(-) diff --git a/block/io.c b/block/io.c index b18680a842..5b96715058 100644 --- a/block/io.c +++ b/blo

[PATCH v3 07/12] block: introduce preallocate filter

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
It's intended to be inserted between format and protocol nodes to preallocate additional space (expanding protocol file) on writes crossing EOF. It improves performance for file-systems with slow allocation. Signed-off-by: Vladimir Sementsov-Ogievskiy --- docs/system/qemu-block-drivers.rst.inc |

[PATCH v3 05/12] block: bdrv_mark_request_serialising: split non-waiting function

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
We'll need a separate function, which will only "mark" request serialising with specified align but not wait for conflicting requests. So, it will be like old bdrv_mark_request_serialising(), before merging bdrv_wait_serialising_requests_locked() into it. To reduce the possible mess, let's do the

[PATCH v3 10/12] iotests: add 298 to test new preallocate filter driver

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/298 | 50 ++ tests/qemu-iotests/298.out | 6 + tests/qemu-iotests/group | 1 + 3 files changed, 57 insertions(+) create mode 100644 tests/qemu-iotests/298 create mode 100644 tests/

[PATCH v3 02/12] block/io.c: drop assertion on double waiting for request serialisation

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
The comments states, that on misaligned request we should have already been waiting. But for bdrv_padding_rmw_read, we called bdrv_mark_request_serialising with align = request_alignment, and now we serialise with align = cluster_size. So we may have to wait again with larger alignment. Note, that

[PATCH v3 08/12] iotests.py: add verify_o_direct helper

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
Add python notrun-helper similar to _check_o_direct for bash tests. To be used in the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 6 ++ 1 file changed, 6 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotes

[PATCH v3 09/12] iotests.py: add filter_img_check

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
Add analog of bash _filter_qemu_img_check to python framework. Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 4 1 file changed, 4 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 369e9918b4..ef3da4ee61 100644 -

[PATCH v3 11/12] block: add bdrv_is_file_on_fuse helper

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
Add a function to check, is it a file-posix node on top of file in FUSE file system. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 2 ++ block/file-posix.c| 21 + 2 files changed, 23 insertions(+) diff --git a/include/block/block.h b/include/bl

[PATCH v3 12/12] block/qcow2: automatically insert preallocate filter when on FUSE

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
vstorage has slow allocation, so this patch detect vstorage (I hope, we don't use other FUSE filesystems) and inserts preallocate filter between qcow2 node and its file child. The following test executes more than 10 times faster (43.2s -> 3.9s for me) with this patch. (/newssd3 is mount point of

Re: [PATCH 10/12] block/file-posix: fix a possible undefined behavior

2020-08-17 Thread Kevin Wolf
Am 14.08.2020 um 18:02 hat Pan Nengyuan geschrieben: > local_err is not initialized to NULL, it will cause a assert error as below: > qemu/util/error.c:59: error_setv: Assertion `*errp == NULL' failed. > > Fixes: c6447510690 > Reported-by: Euler Robot > Signed-off-by: Pan Nengyuan Reviewed-by:

Re: [PATCH 09/12] blockdev: Fix a memleak in drive_backup_prepare()

2020-08-17 Thread Kevin Wolf
Am 14.08.2020 um 18:02 hat Pan Nengyuan geschrieben: > 'local_err' seems forgot to propagate in error path, it'll cause > a memleak. Fix it. > > Reported-by: Euler Robot > Signed-off-by: Pan Nengyuan I wonder if using ERRP_GUARD() wouldn't simplify this. Anyway, the fix looks correct: Reviewed

Re: [PATCH v3 00/12] preallocate filter

2020-08-17 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200817091553.283155-1-vsement...@virtuozzo.com/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEG

Re: [PATCH v3 00/12] preallocate filter

2020-08-17 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20200817091553.283155-1-vsement...@virtuozzo.com/ Hi, This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BE

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > We want to have a common set of commands for all types of block exports. > Currently, this is only NBD, but we're going to add more types. > > This patch adds the basic BlockExport and BlockExportDriver structs and > a QMP command block-export-add that create

[PATCH v2 1/3] hw/sd: sd: Fix incorrect populated function switch status data structure

2020-08-17 Thread Bin Meng
At present the function switch status data structure bit [399:376] are wrongly pupulated. These 3 bytes encode function switch status for the 6 function groups, with 4 bits per group, starting from function group 6 at bit 399, then followed by function group 5 at bit 395, and so on. However the co

[PATCH v2 0/3] hw/sd: Add Cadence SDHCI emulation

2020-08-17 Thread Bin Meng
This series is spun off from the following series as it is hw/sd centric, so that it can be picked up separately by Philippe. http://patchwork.ozlabs.org/project/qemu-devel/list/?series=195648 This series fixed 2 SD card issues, and added a new model for Cadence SDHCI controller. Patch "[09/18]

[PATCH v2 1/3] hw/sd: sd: Fix incorrect populated function switch status data structure

2020-08-17 Thread Bin Meng
At present the function switch status data structure bit [399:376] are wrongly pupulated. These 3 bytes encode function switch status for the 6 function groups, with 4 bits per group, starting from function group 6 at bit 399, then followed by function group 5 at bit 395, and so on. However the co

[PATCH v2 3/3] hw/sd: Add Cadence SDHCI emulation

2020-08-17 Thread Bin Meng
Cadence SD/SDIO/eMMC Host Controller (SD4HC) is an SDHCI compatible controller. The SDHCI compatible registers start from offset 0x200, which are called Slot Register Set (SRS) in its datasheet. This creates a Cadence SDHCI model built on top of the existing generic SDHCI model. Cadence specific H

[PATCH v2 2/3] hw/sd: sd: Correct the maximum size of a Standard Capacity SD Memory Card

2020-08-17 Thread Bin Meng
Per the SD spec, Standard Capacity SD Memory Card (SDSC) supports capacity up to and including 2 GiB. Fixes: 2d7adea4fe ("hw/sd: Support SDHC size cards") Signed-off-by: Bin Meng --- Changes in v2: - fix SDSC size check in sd_set_csd() too hw/sd/sd.c | 6 -- 1 file changed, 4 insertions(+

[PATCH v2 0/3] hw/sd: Add Cadence SDHCI emulation

2020-08-17 Thread Bin Meng
This series is spun off from the following series as it is hw/sd centric, so that it can be picked up separately by Philippe. http://patchwork.ozlabs.org/project/qemu-devel/list/?series=195648 This series fixed 2 SD card issues, and added a new model for Cadence SDHCI controller. Patch "[09/18]

[PATCH v2 2/3] hw/sd: sd: Correct the maximum size of a Standard Capacity SD Memory Card

2020-08-17 Thread Bin Meng
Per the SD spec, Standard Capacity SD Memory Card (SDSC) supports capacity up to and including 2 GiB. Fixes: 2d7adea4fe ("hw/sd: Support SDHC size cards") Signed-off-by: Bin Meng --- Changes in v2: - fix SDSC size check in sd_set_csd() too hw/sd/sd.c | 6 -- 1 file changed, 4 insertions(+

Re: [PATCH v2 2/3] hw/sd: sd: Correct the maximum size of a Standard Capacity SD Memory Card

2020-08-17 Thread Philippe Mathieu-Daudé
On 8/17/20 12:03 PM, Bin Meng wrote: > Per the SD spec, Standard Capacity SD Memory Card (SDSC) supports > capacity up to and including 2 GiB. > > Fixes: 2d7adea4fe ("hw/sd: Support SDHC size cards") > Signed-off-by: Bin Meng Reviewed-by: Philippe Mathieu-Daudé > --- > > Changes in v2: > - fi

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-17 Thread Kevin Wolf
Am 14.08.2020 um 16:57 hat Alberto Garcia geschrieben: > Hi, > > the patch is self-explanatory, but I'm using the cover letter to raise > a couple of related questions. > > Since commit c8bb23cbdbe / QEMU 4.1.0 (and if the storage backend > allows it) writing to an image created with preallocatio

Re: [RFC PATCH 05/22] qemu-storage-daemon: Use qmp_block_export_add()

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > No reason to duplicate the functionality locally, we can now just reuse > the QMP command block-export-add for --export. > > Signed-off-by: Kevin Wolf > --- > qemu-storage-daemon.c | 13 + > 1 file changed, 1 insertion(+), 12 deletions(-) Revie

Re: [PATCH 00/18] hw/riscv: Add Microchip PolarFire SoC Icicle Kit board support

2020-08-17 Thread Bin Meng
Hi Anup, On Sat, Aug 15, 2020 at 1:44 AM Anup Patel wrote: > > On Fri, Aug 14, 2020 at 10:12 PM Bin Meng wrote: > > > > From: Bin Meng > > > > This adds support for Microchip PolarFire SoC Icicle Kit board. > > The Icicle Kit board integrates a PolarFire SoC, with one SiFive's > > E51 plus four

Re: [PATCH v3 00/12] preallocate filter

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
Aha, if we want commit 11, we'll need also a stub function for win32. 17.08.2020 12:45, no-re...@patchew.org wrote: Patchew URL: https://patchew.org/QEMU/20200817091553.283155-1-vsement...@virtuozzo.com/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing com

Re: [PATCH v3 00/12] preallocate filter

2020-08-17 Thread Vladimir Sementsov-Ogievskiy
Hmm strange. Probably need to check #ifdef FUSE_SUPER_MAGIC But again, it's a problem of patch 12, which is just an rfc not to be merged. 17.08.2020 12:48, no-re...@patchew.org wrote: Patchew URL: https://patchew.org/QEMU/20200817091553.283155-1-vsement...@virtuozzo.com/ Hi, This series fa

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > Instead of implementing qemu-nbd --offset in the NBD code, just put a > raw block node with the requested offset on top of the user image and > rely on that doing the job. > > This does not only simplify the nbd_export_new() interface and bring it > closer to

[PATCH] qemu-img: Explicit number replaced by a constant

2020-08-17 Thread Yi Li
Signed-off-by: Yi Li --- qemu-img.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index 5308773811..a0fbc2757c 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -1181,7 +1181,7 @@ static int64_t find_nonzero(const uint8_t *buf, int64_t n) }

Re: [PATCH 1/7] block/null: Make more explicit the driver default size is 1GiB

2020-08-17 Thread Kevin Wolf
Am 14.08.2020 um 10:28 hat Philippe Mathieu-Daudé geschrieben: > As it is not obvious the default size for the null block driver > is 1 GiB, replace the obfuscated '1 << 30' magic value by a > definition using IEC binary prefixes. > > Signed-off-by: Philippe Mathieu-Daudé > --- > block/null.c

Re: [PATCH 2/7] hw/ide/core: Trivial typo fix

2020-08-17 Thread Kevin Wolf
Am 14.08.2020 um 10:28 hat Philippe Mathieu-Daudé geschrieben: > Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Kevin Wolf

Re: [PATCH 3/7] hw/ide/core: Replace magic '512' value by BDRV_SECTOR_SIZE

2020-08-17 Thread Kevin Wolf
Am 14.08.2020 um 10:28 hat Philippe Mathieu-Daudé geschrieben: > Use self-explicit definitions instead of magic '512' value. > > Signed-off-by: Philippe Mathieu-Daudé BDRV_SECTOR_SIZE is the arbitrary unit in which some block layer functions and variables work (such as bs->total_sectors). It h

Re: [PATCH 7/7] hw/scsi/scsi-disk: Replace magic '512' value by BDRV_SECTOR_SIZE

2020-08-17 Thread Kevin Wolf
Am 14.08.2020 um 10:28 hat Philippe Mathieu-Daudé geschrieben: > Use self-explicit definitions instead of magic '512' value. > > Signed-off-by: Philippe Mathieu-Daudé In this patch, BDRV_SECTOR_SIZE actually looks correct to me. The values have already been converted from s->qdev.blocksize in

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > nbd-server-add tries to be convenient and adds two questionable > features that we don't want to share in block-export-add, even for NBD > exports: > > 1. When requesting a writable export of a read-only device, the export >is silently downgraded to read-

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > Instead of implementing qemu-nbd --offset in the NBD code, just put a > raw block node with the requested offset on top of the user image and > rely on that doing the job. > > This does not only simplify the nbd_export_new() interface and bring it > closer to

Re: [PATCH v9 4/5] vhost-user block device backend server

2020-08-17 Thread Coiby Xu
On Thu, Jun 18, 2020 at 05:57:40PM +0200, Kevin Wolf wrote: Am 14.06.2020 um 20:39 hat Coiby Xu geschrieben: By making use of libvhost-user, block device drive can be shared to the connected vhost-user client. Only one client can connect to the server one time. Since vhost-user-server needs a b

Choice of BDRV_REQUEST_MAX_SECTORS

2020-08-17 Thread Peter Lieven
Hi, I am currently debugging a performance issue in qemu-img convert. I think I have found the cause and will send a patch later. But is there any reason why BDRV_REQUEST_MAX_SECTORS is not at least aligned down to 8 (4k sectors)? Any operation that is not able to determinate an optimal or m

Re: [RFC PATCH 08/22] nbd: Add max-connections to nbd-server-start

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > This is a QMP equivalent of qemu-nbd's --share option, limiting the *--shared > maximum number of clients that can attach at the same time. > > Signed-off-by: Kevin Wolf > --- > qapi/block-export.json | 10 -- > include/block/nbd.h

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

2020-08-17 Thread Kevin Wolf
Am 17.08.2020 um 12:03 hat Max Reitz geschrieben: > On 13.08.20 18:29, Kevin Wolf wrote: > > We want to have a common set of commands for all types of block exports. > > Currently, this is only NBD, but we're going to add more types. > > > > This patch adds the basic BlockExport and BlockExportDri

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

2020-08-17 Thread Kevin Wolf
Am 17.08.2020 um 13:41 hat Max Reitz geschrieben: > On 13.08.20 18:29, Kevin Wolf wrote: > > nbd-server-add tries to be convenient and adds two questionable > > features that we don't want to share in block-export-add, even for NBD > > exports: > > > > 1. When requesting a writable export of a rea

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > qemu-nbd allows use of writethrough cache modes, which mean that write > requests made through NBD will cause a flush before they complete. > Expose the same functionality in block-export-add. > > Signed-off-by: Kevin Wolf > --- > qapi/block-export.json | 7

Re: [RFC PATCH 08/22] nbd: Add max-connections to nbd-server-start

2020-08-17 Thread Kevin Wolf
Am 17.08.2020 um 14:37 hat Max Reitz geschrieben: > On 13.08.20 18:29, Kevin Wolf wrote: > > This is a QMP equivalent of qemu-nbd's --share option, limiting the > > *--shared > > > maximum number of clients that can attach at the same time. > > > > Signed-off-by: Kevin Wolf > > --- > > qapi/bl

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

2020-08-17 Thread Kevin Wolf
Am 17.08.2020 um 14:56 hat Max Reitz geschrieben: > On 13.08.20 18:29, Kevin Wolf wrote: > > qemu-nbd allows use of writethrough cache modes, which mean that write > > requests made through NBD will cause a flush before they complete. > > Expose the same functionality in block-export-add. > > > >

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

2020-08-17 Thread Max Reitz
On 17.08.20 14:45, Kevin Wolf wrote: > Am 17.08.2020 um 12:03 hat Max Reitz geschrieben: >> On 13.08.20 18:29, Kevin Wolf wrote: >>> We want to have a common set of commands for all types of block exports. >>> Currently, this is only NBD, but we're going to add more types. >>> >>> This patch adds t

Re: [RFC PATCH 07/22] block/export: Remove magic from block-export-add

2020-08-17 Thread Max Reitz
On 17.08.20 14:49, Kevin Wolf wrote: > Am 17.08.2020 um 13:41 hat Max Reitz geschrieben: >> On 13.08.20 18:29, Kevin Wolf wrote: >>> nbd-server-add tries to be convenient and adds two questionable >>> features that we don't want to share in block-export-add, even for NBD >>> exports: >>> >>> 1. Whe

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

2020-08-17 Thread Kevin Wolf
Am 17.08.2020 um 15:19 hat Max Reitz geschrieben: > On 17.08.20 14:45, Kevin Wolf wrote: > > Am 17.08.2020 um 12:03 hat Max Reitz geschrieben: > >> On 13.08.20 18:29, Kevin Wolf wrote: > >>> We want to have a common set of commands for all types of block exports. > >>> Currently, this is only NBD,

Re: Choice of BDRV_REQUEST_MAX_SECTORS

2020-08-17 Thread Eric Blake
On 8/17/20 7:32 AM, Peter Lieven wrote: Hi, I am currently debugging a performance issue in qemu-img convert. I think I have found the cause and will send a patch later. But is there any reason why BDRV_REQUEST_MAX_SECTORS is not at least aligned down to 8 (4k sectors)? Any operation that

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

2020-08-17 Thread Max Reitz
On 17.08.20 15:13, Kevin Wolf wrote: > Am 17.08.2020 um 14:56 hat Max Reitz geschrieben: >> On 13.08.20 18:29, Kevin Wolf wrote: >>> qemu-nbd allows use of writethrough cache modes, which mean that write >>> requests made through NBD will cause a flush before they complete. >>> Expose the same func

Re: [RFC PATCH 04/22] block/export: Add BlockExport infrastructure and block-export-add

2020-08-17 Thread Max Reitz
On 17.08.20 15:29, Kevin Wolf wrote: > Am 17.08.2020 um 15:19 hat Max Reitz geschrieben: >> On 17.08.20 14:45, Kevin Wolf wrote: >>> Am 17.08.2020 um 12:03 hat Max Reitz geschrieben: On 13.08.20 18:29, Kevin Wolf wrote: > We want to have a common set of commands for all types of block expo

Re: [RFC PATCH 10/22] nbd: Remove NBDExport.close callback

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > The export close callback is unused by the built-in NBD server. qemu-nbd > uses it only during shutdown to wait for the unrefed export to actually > go away. It can just use nbd_export_close_all() instead and do without > the callback. > > This removes the cl

Re: Choice of BDRV_REQUEST_MAX_SECTORS

2020-08-17 Thread Peter Lieven
Am 17.08.20 um 15:44 schrieb Eric Blake: On 8/17/20 7:32 AM, Peter Lieven wrote: Hi, I am currently debugging a performance issue in qemu-img convert. I think I have found the cause and will send a patch later. But is there any reason why BDRV_REQUEST_MAX_SECTORS is not at least aligned dow

Re: [RFC PATCH 11/22] qemu-nbd: Use blk_exp_add() to create the export

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > With this change, NBD exports are only created through the BlockExport > interface any more. This allows us finally to move things from the NBD > layer to the BlockExport layer if they make sense for other export > types, too. I see. > blk_exp_add() returns

Re: [RFC PATCH 12/22] nbd/server: Simplify export shutdown

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > Closing export is somewhat convoluted because nbd_export_close() and > nbd_export_put() call each other and the ways they actually end up being > nested is not necessarily obvious. > > However, it is not really necessary to call nbd_export_close() from > nbd_

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

2020-08-17 Thread Kevin Wolf
Am 17.08.2020 um 15:51 hat Max Reitz geschrieben: > On 17.08.20 15:13, Kevin Wolf wrote: > > Am 17.08.2020 um 14:56 hat Max Reitz geschrieben: > >> On 13.08.20 18:29, Kevin Wolf wrote: > >>> qemu-nbd allows use of writethrough cache modes, which mean that write > >>> requests made through NBD will

Re: [RFC PATCH 11/22] qemu-nbd: Use blk_exp_add() to create the export

2020-08-17 Thread Max Reitz
On 17.08.20 16:27, Max Reitz wrote: > On 13.08.20 18:29, Kevin Wolf wrote: >> With this change, NBD exports are only created through the BlockExport >> interface any more. This allows us finally to move things from the NBD >> layer to the BlockExport layer if they make sense for other export >> typ

Re: [RFC PATCH 14/22] block/export: Move AioContext from NBDExport to BlockExport

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > Signed-off-by: Kevin Wolf > --- > include/block/export.h | 6 ++ > nbd/server.c | 26 +- > 2 files changed, 19 insertions(+), 13 deletions(-) Reviewed-by: Max Reitz > diff --git a/include/block/export.h b/include/blo

[PULL 2/3] async: always set ctx->notified in aio_notify()

2020-08-17 Thread Stefan Hajnoczi
aio_notify() does not set ctx->notified when called with ctx->aio_notify_me disabled. Therefore aio_notify_me needs to be enabled during polling. This is suboptimal since expensive event_notifier_set(&ctx->notifier) and event_notifier_test_and_clear(&ctx->notifier) calls are required when ctx->aio

Re: [RFC PATCH 14/22] block/export: Move AioContext from NBDExport to BlockExport

2020-08-17 Thread Kevin Wolf
Am 17.08.2020 um 16:56 hat Max Reitz geschrieben: > On 13.08.20 18:29, Kevin Wolf wrote: > > Signed-off-by: Kevin Wolf > > --- > > include/block/export.h | 6 ++ > > nbd/server.c | 26 +- > > 2 files changed, 19 insertions(+), 13 deletions(-) > > Reviewed-b

Re: [RFC PATCH 13/22] block/export: Move refcount from NBDExport to BlockExport

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > Having a refcount makes sense for all types of block exports. It is also > a prerequisite for keeping a list of all exports at the BlockExport > level. > > Signed-off-by: Kevin Wolf > --- > include/block/export.h | 10 ++ > include/block/nbd.h| 2 -

Re: [RFC PATCH 15/22] block/export: Move device to BlockExportOptions

2020-08-17 Thread Max Reitz
On 17.08.20 17:27, Kevin Wolf wrote: > Am 17.08.2020 um 17:13 hat Max Reitz geschrieben: >> On 13.08.20 18:29, Kevin Wolf wrote: >>> Every block export needs a block node to export, so move the 'device' >>> option from BlockExportOptionsNbd to BlockExportOptions. >>> >>> To maintain compatibility i

Re: [RFC PATCH 15/22] block/export: Move device to BlockExportOptions

2020-08-17 Thread Max Reitz
On 13.08.20 18:29, Kevin Wolf wrote: > Every block export needs a block node to export, so move the 'device' > option from BlockExportOptionsNbd to BlockExportOptions. > > To maintain compatibility in nbd-server-add, BlockExportOptionsNbd needs > to be wrapped by a new type NbdServerAddOptions tha

Re: [RFC PATCH 11/22] qemu-nbd: Use blk_exp_add() to create the export

2020-08-17 Thread Kevin Wolf
Am 17.08.2020 um 16:27 hat Max Reitz geschrieben: > On 13.08.20 18:29, Kevin Wolf wrote: > > With this change, NBD exports are only created through the BlockExport > > interface any more. This allows us finally to move things from the NBD > > layer to the BlockExport layer if they make sense for ot

[PULL 0/3] Block patches

2020-08-17 Thread Stefan Hajnoczi
The following changes since commit d0ed6a69d399ae193959225cdeaa9382746c91cc: Update version for v5.1.0 release (2020-08-11 17:07:03 +0100) are available in the Git repository at: https://github.com/stefanha/qemu.git tags/block-pull-request for you to fetch changes up to 44277bf914471962c9e8

[PULL 3/3] aio-posix: keep aio_notify_me disabled during polling

2020-08-17 Thread Stefan Hajnoczi
Polling only monitors the ctx->notified field and does not need the ctx->notifier EventNotifier to be signalled. Keep ctx->aio_notify_me disabled while polling to avoid unnecessary EventNotifier syscalls. This optimization improves virtio-blk 4KB random read performance by 18%. The following resul

[PULL 1/3] async: rename event_notifier_dummy_cb/poll()

2020-08-17 Thread Stefan Hajnoczi
The event_notifier_*() prefix can be confused with the EventNotifier APIs that are also called event_notifier_*(). Rename the functions to aio_context_notifier_*() to make it clear that they relate to the AioContext::notifier field. Signed-off-by: Stefan Hajnoczi Reviewed-by: Philippe Mathieu-Da

Re: [RFC PATCH 15/22] block/export: Move device to BlockExportOptions

2020-08-17 Thread Kevin Wolf
Am 17.08.2020 um 17:13 hat Max Reitz geschrieben: > On 13.08.20 18:29, Kevin Wolf wrote: > > Every block export needs a block node to export, so move the 'device' > > option from BlockExportOptionsNbd to BlockExportOptions. > > > > To maintain compatibility in nbd-server-add, BlockExportOptionsNbd

Re: [RFC PATCH 09/22] nbd: Add writethrough to block-export-add

2020-08-17 Thread Max Reitz
On 17.08.20 16:32, Kevin Wolf wrote: > Am 17.08.2020 um 15:51 hat Max Reitz geschrieben: >> On 17.08.20 15:13, Kevin Wolf wrote: >>> Am 17.08.2020 um 14:56 hat Max Reitz geschrieben: On 13.08.20 18:29, Kevin Wolf wrote: > qemu-nbd allows use of writethrough cache modes, which mean that wri

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-17 Thread Alberto Garcia
On Mon 17 Aug 2020 05:53:07 PM CEST, Kevin Wolf wrote: >> > Or are you saying that ZERO_RANGE + pwrite on a sparse file (= >> > cluster allocation) is faster for you than just the pwrite alone (= >> > writing to already allocated cluster)? >> >> Yes, 20% faster in my tests (4KB random writes), but

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-17 Thread Alberto Garcia
On Mon 17 Aug 2020 12:10:19 PM CEST, Kevin Wolf wrote: >> Since commit c8bb23cbdbe / QEMU 4.1.0 (and if the storage backend >> allows it) writing to an image created with preallocation=metadata >> can be slower (20% in my tests) than writing to an image with no >> preallocation at all. > > A while

Re: [RFC PATCH 14/22] block/export: Move AioContext from NBDExport to BlockExport

2020-08-17 Thread Max Reitz
On 17.08.20 17:22, Kevin Wolf wrote: > Am 17.08.2020 um 16:56 hat Max Reitz geschrieben: >> On 13.08.20 18:29, Kevin Wolf wrote: >>> Signed-off-by: Kevin Wolf >>> --- >>> include/block/export.h | 6 ++ >>> nbd/server.c | 26 +- >>> 2 files changed, 19 insert

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-17 Thread Kevin Wolf
Am 17.08.2020 um 17:31 hat Alberto Garcia geschrieben: > On Mon 17 Aug 2020 12:10:19 PM CEST, Kevin Wolf wrote: > >> Since commit c8bb23cbdbe / QEMU 4.1.0 (and if the storage backend > >> allows it) writing to an image created with preallocation=metadata > >> can be slower (20% in my tests) than wr

Re: [PATCH 2/2] scsi-generic: Fix HM-zoned device scan

2020-08-17 Thread Alistair Francis
On Tue, Aug 11, 2020 at 3:52 PM Dmitry Fomichev wrote: > > Several important steps during device scan depend on SCSI type of the > device. For example, max_transfer property is only determined and > assigned if the device has the type of TYPE_DISK. > > Host-managed ZBC disks retain most of the pro

Re: [PATCH 0/2] block; scsi-generic: Fix max transfer size calculation

2020-08-17 Thread Paolo Bonzini
On 12/08/20 00:51, Dmitry Fomichev wrote: > When a host-managed zoned device is passed through to the > guest system using scsi-generic driver, the maximum i/o size for the > drive at the guest may end up being larger than at the host, causing > i/o errors while accessing the backing zoned drive at

Re: [PATCH 0/2] Fix the assert failure in scsi_dma_complete

2020-08-17 Thread Paolo Bonzini
On 15/08/20 16:19, Li Qiang wrote: > Currently in 'megasas_map_sgl' when 'iov_count=0' will just return > success however the 'cmd' doens't contain any iov. This will cause > the assert in 'scsi_dma_complete' failed. This is because in > 'dma_blk_cb' the 'dbs->sg_cur_index == dbs->sg->nsg' will be

Re: [PATCH] hw: add a number of SPI-flash's of m25p80 family

2020-08-17 Thread i . kononenko
No, the ext ID wasn't be checked at a real HW. Just copied it from the U-boot official repository https://github.com/u-boot/u-boot/blob/789bfb52668ee609b2043de645e2f94bbd24fd1f/drivers/mtd/spi/spi-nor-ids.c#L183 Do i need to take it from a real HW and compare? On 12.08.2020 10:27, Cédric Le Goate

Re: [RFC PATCH 06/22] qemu-nbd: Use raw block driver for --offset

2020-08-17 Thread Nir Soffer
On Thu, Aug 13, 2020 at 7:36 PM Kevin Wolf wrote: > Instead of implementing qemu-nbd --offset in the NBD code, just put a > raw block node with the requested offset on top of the user image and > rely on that doing the job. > > This does not only simplify the nbd_export_new() interface and bring

Re: [PATCH 00/18] hw/riscv: Add Microchip PolarFire SoC Icicle Kit board support

2020-08-17 Thread via
Hi Anup, On 8/17/20 11:30 AM, Bin Meng wrote: > EXTERNAL EMAIL: Do not click links or open attachments unless you know the > content is safe > > Hi Anup, > > On Sat, Aug 15, 2020 at 1:44 AM Anup Patel wrote: >> On Fri, Aug 14, 2020 at 10:12 PM Bin Meng wrote: >>> From: Bin Meng >>> >>> This ad

Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster

2020-08-17 Thread Alberto Garcia
On Mon 17 Aug 2020 05:53:07 PM CEST, Kevin Wolf wrote: > Maybe the difference is in allocating 64k at once instead of doing a > separate allocation for every 4k block? But with the extent size hint > patches to file-posix, we should allocate 1 MB at once by default now > (if your test image was new

Re: [PATCH] hw/sd/allwinner-sdhost: Use AddressSpace for DMA transfers

2020-08-17 Thread Niek Linnenbank
Hi Philippe, Nice improvement, I didnt know about this API. Makes sense to use it indeed. The patch works fine. I tested your patches by applying the previous two sets first, and then this one. It ran well with the avocado tests and also with the official image OrangePi_pc_debian_stretch_server_li

Re: [PATCH 00/18] hw/riscv: Add Microchip PolarFire SoC Icicle Kit board support

2020-08-17 Thread Alistair Francis
On Mon, Aug 17, 2020 at 11:12 AM via wrote: > > Hi Anup, > > On 8/17/20 11:30 AM, Bin Meng wrote: > > EXTERNAL EMAIL: Do not click links or open attachments unless you know the > > content is safe > > > > Hi Anup, > > > > On Sat, Aug 15, 2020 at 1:44 AM Anup Patel wrote: > >> On Fri, Aug 14, 202

Re: [PATCH 00/18] hw/riscv: Add Microchip PolarFire SoC Icicle Kit board support

2020-08-17 Thread via
On 8/17/20 8:28 PM, Alistair Francis wrote: > EXTERNAL EMAIL: Do not click links or open attachments unless you know the > content is safe > > On Mon, Aug 17, 2020 at 11:12 AM via wrote: >> Hi Anup, >> >> On 8/17/20 11:30 AM, Bin Meng wrote: >>> EXTERNAL EMAIL: Do not click links or open attachme

Re: [PATCH 0/2] Fix the assert failure in scsi_dma_complete

2020-08-17 Thread Li Qiang
Paolo Bonzini 于2020年8月18日周二 上午1:05写道: > On 15/08/20 16:19, Li Qiang wrote: > > Currently in 'megasas_map_sgl' when 'iov_count=0' will just return > > success however the 'cmd' doens't contain any iov. This will cause > > the assert in 'scsi_dma_complete' failed. This is because in > > 'dma_blk_cb

Re: [PATCH] hw: add a number of SPI-flash's of m25p80 family

2020-08-17 Thread Cédric Le Goater
On 8/17/20 7:16 PM, i.kononenko wrote: > No, the ext ID wasn't be checked at a real HW. > Just copied it from the U-boot official repository > https://github.com/u-boot/u-boot/blob/789bfb52668ee609b2043de645e2f94bbd24fd1f/drivers/mtd/spi/spi-nor-ids.c#L183 OK. Reviewed-by: Cédric Le Goater > D

Re: What is bs->reqs_lock for?

2020-08-17 Thread Paolo Bonzini
On 13/08/20 18:34, Vladimir Sementsov-Ogievskiy wrote: > I thought bs is attached to one aio context and aio context attached to > one iothread. For now yes, but with multiqueue there would be many iothreads sending requests to the AioContext. The BDS would still have a "home" aiocontext to reque

Re: [PATCH 00/18] hw/riscv: Add Microchip PolarFire SoC Icicle Kit board support

2020-08-17 Thread Anup Patel
On Tue, Aug 18, 2020 at 1:23 AM wrote: > > On 8/17/20 8:28 PM, Alistair Francis wrote: > > EXTERNAL EMAIL: Do not click links or open attachments unless you know the > > content is safe > > > > On Mon, Aug 17, 2020 at 11:12 AM via wrote: > >> Hi Anup, > >> > >> On 8/17/20 11:30 AM, Bin Meng wrot