Re: [PATCH v2 2/4] block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()

2020-12-17 Thread Sergio Lopez
On Wed, Dec 16, 2020 at 07:31:02PM +0100, Kevin Wolf wrote: > Am 16.12.2020 um 15:55 hat Sergio Lopez geschrieben: > > On Wed, Dec 16, 2020 at 01:35:14PM +0100, Kevin Wolf wrote: > > > Am 15.12.2020 um 18:23 hat Sergio Lopez geschrieben: > > > > On Tue, Dec 15, 2020 at 04:01:19PM +0100, Kevin Wolf

Re: [PATCH v2 2/4] block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()

2020-12-17 Thread Kevin Wolf
Am 17.12.2020 um 10:37 hat Sergio Lopez geschrieben: > On Wed, Dec 16, 2020 at 07:31:02PM +0100, Kevin Wolf wrote: > > Am 16.12.2020 um 15:55 hat Sergio Lopez geschrieben: > > > On Wed, Dec 16, 2020 at 01:35:14PM +0100, Kevin Wolf wrote: > > > > Am 15.12.2020 um 18:23 hat Sergio Lopez geschrieben:

aio-poll dead-lock

2020-12-17 Thread Vladimir Sementsov-Ogievskiy
I don't think that it's something new, but just to keep it in mind: blk_prw do polling with increased in-flight counter. So, if some bh wants to do drain, we definitely dead-lock in a nested aio_poll loop. Here is a backtrace (comes from Virtuozzo branch, so I don't have reproducer for master

Re: [PATCH v2 2/4] block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()

2020-12-17 Thread Vladimir Sementsov-Ogievskiy
17.12.2020 13:58, Kevin Wolf wrote: Am 17.12.2020 um 10:37 hat Sergio Lopez geschrieben: On Wed, Dec 16, 2020 at 07:31:02PM +0100, Kevin Wolf wrote: Am 16.12.2020 um 15:55 hat Sergio Lopez geschrieben: On Wed, Dec 16, 2020 at 01:35:14PM +0100, Kevin Wolf wrote: Am 15.12.2020 um 18:23 hat Serg

Re: [PATCH v2 2/4] block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()

2020-12-17 Thread Sergio Lopez
On Thu, Dec 17, 2020 at 11:58:30AM +0100, Kevin Wolf wrote: > Am 17.12.2020 um 10:37 hat Sergio Lopez geschrieben: > > Do you think it's safe to re-enter backup-top, or should we look for a > > way to avoid this? > > I think it should be avoided, but I don't understand why putting all > children o

Re: [PATCH v2 2/4] block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()

2020-12-17 Thread Kevin Wolf
Am 17.12.2020 um 13:50 hat Vladimir Sementsov-Ogievskiy geschrieben: > 17.12.2020 13:58, Kevin Wolf wrote: > > Am 17.12.2020 um 10:37 hat Sergio Lopez geschrieben: > > > On Wed, Dec 16, 2020 at 07:31:02PM +0100, Kevin Wolf wrote: > > > > Am 16.12.2020 um 15:55 hat Sergio Lopez geschrieben: > > > >

Re: [PATCH v2 2/4] block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()

2020-12-17 Thread Sergio Lopez
On Thu, Dec 17, 2020 at 02:06:02PM +0100, Kevin Wolf wrote: > Am 17.12.2020 um 13:50 hat Vladimir Sementsov-Ogievskiy geschrieben: > > 17.12.2020 13:58, Kevin Wolf wrote: > > > Am 17.12.2020 um 10:37 hat Sergio Lopez geschrieben: > > > > On Wed, Dec 16, 2020 at 07:31:02PM +0100, Kevin Wolf wrote: >

Re: [PATCH v2 2/4] block: Avoid processing BDS twice in bdrv_set_aio_context_ignore()

2020-12-17 Thread Vladimir Sementsov-Ogievskiy
17.12.2020 16:06, Kevin Wolf wrote: Am 17.12.2020 um 13:50 hat Vladimir Sementsov-Ogievskiy geschrieben: 17.12.2020 13:58, Kevin Wolf wrote: Am 17.12.2020 um 10:37 hat Sergio Lopez geschrieben: On Wed, Dec 16, 2020 at 07:31:02PM +0100, Kevin Wolf wrote: Am 16.12.2020 um 15:55 hat Sergio Lopez

Re: [PATCH v3 1/3] docs: generate qemu-storage-daemon-qmp-ref(7) man page

2020-12-17 Thread Stefan Hajnoczi
On Wed, Dec 16, 2020 at 06:59:03PM +0100, Kevin Wolf wrote: > Am 16.12.2020 um 17:21 hat Stefan Hajnoczi geschrieben: > > On Tue, Dec 15, 2020 at 05:11:06PM +0100, Kevin Wolf wrote: > > > > diff --git a/docs/interop/qemu-storage-daemon-qmp-ref.rst > > > > b/docs/interop/qemu-storage-daemon-qmp-ref

[PATCH v2 0/2] Fix _send_qemu_cmd with bash 5.1

2020-12-17 Thread Max Reitz
Hi, See patch 2 for what’s changed in bash 5.1 that makes this series necessary, and what we need to do. (tl;dr: Bash presumably fixed a bug for array slices, and that now breaks nearly all iotests that use _send_qemu_cmd – but we don’t even need an array slice there.) Patch 1 fixes a bug in 102

[PATCH v2 1/2] iotests/102: Pass $QEMU_HANDLE to _send_qemu_cmd

2020-12-17 Thread Max Reitz
The first parameter passed to _send_qemu_cmd is supposed to be the $QEMU_HANDLE. 102 does not do so here, fix it. As a result, the output changes: Now we see the prompt this command is supposedly waiting for before the resize message - as it should be. Signed-off-by: Max Reitz --- tests/qemu-i

[PATCH v1 0/3] Fix some style problems in block

2020-12-17 Thread chaihaoyu
This patch fixed some code style problems while using checkpatch.pl tool, please review. Date: Wen, 16 Dec 2020 17:19:37 +0800 Subject: [PATCH] Fix some style problems in block signed-off-by: Haoyu Chai --- Haoyu Chai (3): block: some space-style errors while coding block: braces {} are nece

[PATCH v1 1/3] some space-style errors while coding

2020-12-17 Thread chaihaoyu
This patch fixes error style problems found by checkpatch.pl, please review: some space-style errors while coding Signed-off-by: Haoyu Chai --- block/bochs.c| 10 block/cloop.c| 2 +- block/curl.c | 7 +++ block/dmg.c | 4 +- block/file-posix.c

[PATCH v1 2/3]block: braces {} are necessary for all arms of this statement

2020-12-17 Thread chaihaoyu
This patch fixes error style problems found by checkpatch.pl, please review: braces {} are necessary for all arms of this statement Signed-off-by: Haoyu Chai --- block/bochs.c| 6 ++- block/commit.c | 3 +- block/curl.c | 15 +++-- block/file-posix.c | 27 ++--- blo

[PATCH v2 2/2] iotests: Fix _send_qemu_cmd with bash 5.1

2020-12-17 Thread Max Reitz
With bash 5.1, the output of the following script changes: a=("double space") a=${a[@]:0:1} echo "$a" from "double space" to "double space", i.e. all white space is preserved as-is. This is probably what we actually want here (judging from the "...to accommodate pathnames with spaces" co

[PATCH v1 3/3] block: "(foo*)" should be "(foo *)"

2020-12-17 Thread chaihaoyu
This patch fixes error style problems found by checkpatch.pl, please review: "(foo*)" should be "(foo *)" Signed-off-by: Haoyu Chai --- block/curl.c | 2 +- block/file-win32.c | 2 +- block/qcow2.c| 10 +++--- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/blo

Re: [PATCH] block/nvme: Do not allow image creation with NVMe block driver

2020-12-17 Thread Stefan Hajnoczi
On Mon, Dec 07, 2020 at 06:16:04PM +0100, Philippe Mathieu-Daudé wrote: > On 12/4/20 11:28 PM, Philippe Mathieu-Daudé wrote: > > On 12/4/20 5:57 PM, Philippe Mathieu-Daudé wrote: > >> The NVMe driver does not support image creation. > >> The full drive has to be passed to the guest. > >> > >> Befor

[PATCH 2/9] block/vpc: Don't abuse the footer buffer as BAT sector buffer

2020-12-17 Thread Markus Armbruster
create_dynamic_disk() takes a buffer holding the footer as first argument. It writes out the footer (512 bytes), then reuses the buffer to initialize and write out the dynamic header (1024 bytes), then reuses it again to initialize and write out BAT sectors (512). Works, because the caller passes

[PATCH 4/9] block/vpc: Make vpc_checksum() take void *

2020-12-17 Thread Markus Armbruster
Some of the next commits will checksum structs. Change vpc_checksum() to take void * instead of uint8_t, to save us pointless casts to uint8_t *. Signed-off-by: Markus Armbruster --- block/vpc.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/block/vpc.c b/block/vpc.c inde

[PATCH 9/9] block/vpc: Use sizeof() instead of HEADER_SIZE for footer size

2020-12-17 Thread Markus Armbruster
Signed-off-by: Markus Armbruster --- block/vpc.c | 29 ++--- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/block/vpc.c b/block/vpc.c index aac13788df..17a705b482 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -39,8 +39,6 @@ /

[PATCH 1/9] block/vpc: Make vpc_open() read the full dynamic header

2020-12-17 Thread Markus Armbruster
The dynamic header's size is 1024 bytes. vpc_open() reads only the 512 bytes of the dynamic header into buf[]. Works, because it doesn't actually access the second half. However, a colleague told me that GCC 11 warns: ../block/vpc.c:358:51: error: array subscript 'struct VHDDynDiskHeader[0]'

[PATCH 3/9] block/vpc: Don't abuse the footer buffer for dynamic header

2020-12-17 Thread Markus Armbruster
create_dynamic_disk() takes a buffer holding the footer as first argument. It writes out the footer (512 bytes), then reuses the buffer to initialize and write out the dynamic header (1024 bytes). Works, because the caller passes a buffer that is large enough for both purposes. I hate that. Use

[PATCH 0/9] block/vpc: Clean up some buffer abuse

2020-12-17 Thread Markus Armbruster
Markus Armbruster (9): block/vpc: Make vpc_open() read the full dynamic header block/vpc: Don't abuse the footer buffer as BAT sector buffer block/vpc: Don't abuse the footer buffer for dynamic header block/vpc: Make vpc_checksum() take void * block/vpc: Pad VHDDynDiskHeader, replace uint

[PATCH 6/9] block/vpc: Use sizeof() instead of 1024 for dynamic header size

2020-12-17 Thread Markus Armbruster
Signed-off-by: Markus Armbruster --- block/vpc.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/vpc.c b/block/vpc.c index 08a0f710ad..6cb656ac82 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -344,7 +344,7 @@ static int vpc_open(BlockDriverState *bs, QDict *opti

[PATCH 8/9] block/vpc: Pass footer buffers as VHDFooter * instead of uint8_t *

2020-12-17 Thread Markus Armbruster
Signed-off-by: Markus Armbruster --- block/vpc.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/block/vpc.c b/block/vpc.c index f3ea92dcb0..aac13788df 100644 --- a/block/vpc.c +++ b/block/vpc.c @@ -819,7 +819,7 @@ static int calculate_geometry(int64_t total_sec

[PATCH 5/9] block/vpc: Pad VHDDynDiskHeader, replace uint8_t[] buffers

2020-12-17 Thread Markus Armbruster
Pad VHDDynDiskHeader as specified in the "Virtual Hard Disk Image Format Specification" version 1.0[*]. Change dynamic disk header buffers from uint8_t[1024] to VHDDynDiskHeader. Their size remains the same. The VHDDynDiskHeader * variables pointing to a VHDDynDiskHeader variable right next to i

[PATCH 7/9] block/vpc: Pad VHDFooter, replace uint8_t[] buffers

2020-12-17 Thread Markus Armbruster
Pad VHDFooter as specified in the "Virtual Hard Disk Image Format Specification" version 1.0[*]. Change footer buffers from uint8_t[HEADER_SIZE] to VHDFooter. Their size remains the same. The VHDFooter * variables pointing to a VHDFooter variable right next to it are now silly. Eliminate them,

Re: [PATCH v2 1/5] file-posix: split hdev_refresh_limits from raw_refresh_limits

2020-12-17 Thread Maxim Levitsky
On Thu, 2020-12-10 at 18:36 +0800, Tom Yan wrote: > On Wed, 9 Dec 2020 at 21:54, Maxim Levitsky wrote: > > From: Tom Yan > > > > We can and should get max transfer length and max segments for all host > > devices / cdroms (on Linux). > > > > Also use MIN_NON_ZERO instead when we clamp max trans

[PATCH v3 0/5] SCSI: fix transfer limits for SCSI passthrough

2020-12-17 Thread Maxim Levitsky
This patch series attempts to provide a solution to the problem of the transfer limits of the raw file driver (host_device/file-posix), some of which I already tried to fix in the past. I included 2 patches from Tom Yan which fix two issues with reading the limits correctly from the */dev/sg* char

[PATCH v3 4/5] block: use blk_get_max_ioctl_transfer for SCSI passthrough

2020-12-17 Thread Maxim Levitsky
Switch file-posix to expose only the max_ioctl_transfer limit. Let the iscsi driver work as it did before since it is bound by the transfer limit in both regular read/write and in SCSI passthrough case. Switch the scsi-disk and scsi-block drivers to read the SG max transfer limits using the new b

[PATCH v3 2/5] file-posix: add sg_get_max_segments that actually works with sg

2020-12-17 Thread Maxim Levitsky
From: Tom Yan sg devices have different major/minor than their corresponding block devices. Using sysfs to get max segments never really worked for them. Fortunately the sg driver provides an ioctl to get sg_tablesize, which is apparently equivalent to max segments. Signed-off-by: Tom Yan Sign

[PATCH v3 5/5] block/scsi: correctly emulate the VPD block limits page

2020-12-17 Thread Maxim Levitsky
When the device doesn't support the VPD block limits page, we emulate it even for SCSI passthrough. As a part of the emulation we need to add it to the 'Supported VPD Pages' The code that does this adds it to the page, but it doesn't increase the length of the data to be copied to the guest, thus

[PATCH v3 1/5] file-posix: split hdev_refresh_limits from raw_refresh_limits

2020-12-17 Thread Maxim Levitsky
From: Tom Yan We can and should get max transfer length and max segments for all host devices / cdroms (on Linux). Also use MIN_NON_ZERO instead when we clamp max transfer length against max segments. Signed-off-by: Tom Yan Signed-off-by: Maxim Levitsky --- block/file-posix.c | 57 ++

[PATCH v3 3/5] block: add max_ioctl_transfer to BlockLimits

2020-12-17 Thread Maxim Levitsky
Maximum transfer size when accessing a kernel block device is only relevant when using SCSI passthrough (SG_IO ioctl) since only in this case the requests are passed directly to underlying hardware with no pre-processing. Same is true when using /dev/sg* character devices (which only support SG_IO)

[PATCH v6 1/3] crypto: luks: Fix tiny memory leak

2020-12-17 Thread Maxim Levitsky
When the underlying block device doesn't support the bdrv_co_delete_file interface, an 'Error' object was leaked. Signed-off-by: Maxim Levitsky Reviewed-by: Alberto Garcia Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/crypto.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/c

[PATCH v6 2/3] block: add bdrv_co_delete_file_noerr

2020-12-17 Thread Maxim Levitsky
This function wraps bdrv_co_delete_file for the common case of removing a file, which was just created by format driver, on an error condition. It hides the -ENOTSUPP error, and reports all other errors otherwise. Use it in luks driver Signed-off-by: Maxim Levitsky Reviewed-by: Alberto Garcia

[PATCH v6 0/3] qcow2: don't leave partially initialized file on image creation

2020-12-17 Thread Maxim Levitsky
Use the bdrv_co_delete_file interface to delete the underlying file if qcow2 initialization fails (e.g due to bad encryption secret) This makes the qcow2 driver behave the same way as the luks driver behaves. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1845353 V3: addressed review feedb

[PATCH v6 3/3] block: qcow2: remove the created file on initialization error

2020-12-17 Thread Maxim Levitsky
If the qcow initialization fails, we should remove the file if it was already created, to avoid leaving stale files around. We already do this for luks raw images. Signed-off-by: Maxim Levitsky Reviewed-by: Alberto Garcia --- block/qcow2.c | 8 +--- 1 file changed, 5 insertions(+), 3 delet

[PATCH RFC 0/3] hw/block/nvme: dif-based end-to-end data protection support

2020-12-17 Thread Klaus Jensen
From: Klaus Jensen This series adds support for extended LBAs and end-to-end data protection. Marked RFC, since there are a bunch of issues that could use some discussion. Storing metadata bytes contiguously with the logical block data and creating a physically extended logical block basically b

[PATCH RFC 1/3] nvme: add support for extended LBAs

2020-12-17 Thread Klaus Jensen
From: Gollu Appalanaidu This allows logical blocks to be extended with a number of metadata bytes specified by the new namespace parameter 'ms'. The additional bytes are stored immediately after each logical block. The Deallocated or Unwritten Logical Block Error recovery feature is not supporte

[PATCH RFC 3/3] hw/block/nvme: end-to-end data protection

2020-12-17 Thread Klaus Jensen
From: Gollu Appalanaidu Add support for namespaces formatted with protection information in the form of the Data Integrity Field (DIF) where the protection information is contiguous with the logical block data (extended logical blocks). The type of end-to-end data protection (i.e. Type 1, Type 2

[PATCH RFC 2/3] hw/block/nvme: refactor nvme_dma

2020-12-17 Thread Klaus Jensen
From: Klaus Jensen The nvme_dma function doesn't just do DMA (QEMUSGList-based) memory transfers; it also handles QEMUIOVector copies. Introduce the NvmeTxDirection enum and rename to nvme_tx. Remove mapping of PRPs/SGLs from nvme_tx and assert that they have been mapped previously. This allows

Re: [PATCH RFC 0/3] hw/block/nvme: dif-based end-to-end data protection support

2020-12-17 Thread Keith Busch
On Thu, Dec 17, 2020 at 10:02:19PM +0100, Klaus Jensen wrote: > From: Klaus Jensen > > This series adds support for extended LBAs and end-to-end data > protection. Marked RFC, since there are a bunch of issues that could use > some discussion. > > Storing metadata bytes contiguously with the log

Re: [PATCH v6 2/3] block: add bdrv_co_delete_file_noerr

2020-12-17 Thread Vladimir Sementsov-Ogievskiy
17.12.2020 20:09, Maxim Levitsky wrote: This function wraps bdrv_co_delete_file for the common case of removing a file, which was just created by format driver, on an error condition. It hides the -ENOTSUPP error, and reports all other errors otherwise. Use it in luks driver Signed-off-by: Max

Re: [PATCH v6 3/3] block: qcow2: remove the created file on initialization error

2020-12-17 Thread Vladimir Sementsov-Ogievskiy
17.12.2020 20:09, Maxim Levitsky wrote: If the qcow initialization fails, we should remove the file if it was already created, to avoid leaving stale files around. We already do this for luks raw images. Signed-off-by: Maxim Levitsky Reviewed-by: Alberto Garcia Reviewed-by: Vladimir Sementsov