Re: [Libguestfs] [libnbd PATCH v3 07/22] generator: Add struct nbd_extent in prep for 64-bit extents

2023-07-24 Thread Eric Blake
On Mon, Jul 24, 2023 at 07:04:15PM -0500, Eric Blake wrote: > On Wed, Jun 07, 2023 at 04:23:27PM +0200, Laszlo Ersek wrote: > > > +++ b/generator/GoLang.ml > > > @@ -524,6 +528,18 @@ let > > > copy(ret, s) > > > return ret > > > } > > > + > > > +func copy_extent_array (entries *C.nbd_ext

Re: [Libguestfs] [libnbd PATCH v3 07/22] generator: Add struct nbd_extent in prep for 64-bit extents

2023-07-24 Thread Eric Blake
On Wed, Jun 07, 2023 at 04:23:27PM +0200, Laszlo Ersek wrote: > > +++ b/generator/GoLang.ml > > @@ -524,6 +528,18 @@ let > > copy(ret, s) > > return ret > > } > > + > > +func copy_extent_array (entries *C.nbd_extent, count C.size_t) > > []LibnbdExtent { > > +unsafePtr := unsafe.Poin

Re: [PATCH] vhost-user-scsi: support reconnect to backend

2023-07-24 Thread Michael S. Tsirkin
On Mon, Jul 24, 2023 at 05:21:37PM +, Raphael Norwitz wrote: > Very excited to see this. High level looks good modulo a few small things. > > My major concern is around existing vhost-user-scsi backends which don’t > support VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD. IMO we should hide the > reco

Re: [Libguestfs] [libnbd PATCH v4 6/6] states: Break deadlock if server goofs on extended replies

2023-07-24 Thread Eric Blake
On Mon, Jul 24, 2023 at 11:58:32AM +0200, Laszlo Ersek wrote: > On 7/21/23 18:08, Eric Blake wrote: > > One of the benefits of extended replies is that we can do a > > fixed-length read for the entire header of every server reply, which > > is fewer syscalls than the split-read approach required by

Re: [PATCH] vhost-user-scsi: support reconnect to backend

2023-07-24 Thread Raphael Norwitz
Very excited to see this. High level looks good modulo a few small things. My major concern is around existing vhost-user-scsi backends which don’t support VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD. IMO we should hide the reconnect behavior behind a VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD check. We may

[PATCH 2/2] block/blkio: use blkio_set_int("fd") to check fd support

2023-07-24 Thread Stefano Garzarella
The way the virtio-blk driver is implemented in libblkio, it's much easier to use blkio_set_int() instead of blkio_get_int() and have it fail right away to see if `fd` is supported by the transport. See https://gitlab.com/libblkio/libblkio/-/merge_requests/208 Signed-off-by: Stefano Garzarella --

[PATCH 0/2] block/blkio: fix opening virtio-blk drivers

2023-07-24 Thread Stefano Garzarella
There is a problem with virtio-blk-vhost-vdpa. The first patch fixes the issue, the second patch tries to prepare QEMU for a future version of libblkio where we can use blkio_set_fd() to check whether the property is supported or not. Stefano Garzarella (2): block/blkio: fix opening virtio-blk d

[PATCH 1/2] block/blkio: fix opening virtio-blk drivers

2023-07-24 Thread Stefano Garzarella
libblkio 1.3.0 added support of "fd" property for virtio-blk-vhost-vdpa driver. In QEMU, starting from commit cad2ccc395 ("block/blkio: use qemu_open() to support fd passing for virtio-blk") we are using `blkio_get_int(..., "fd")` to check if the "fd" property is supported for all the virtio-blk-*

[PATCH v1] block/stream:add flush l2_table_cache, ensure data integrity

2023-07-24 Thread Evanzhang
Verification steps: 1.qemu-img create -f qcow2 data1.img 10M qemu-img create -f qcow2 -b data1.img data1.qcow2 2.write 1M data to data1.img 3.create vm use data1.qcow2 as the data disk /usr/libexec/qemu-kvm -M pc,accel=kvm -smp 4 -cpu host -m 4g -drive file=./centos_7.5_64_20200603.qco

[PATCH v1] block/stream:add flush l2_table_cache, ensure data integrity

2023-07-24 Thread Evanzhang
block_stream will not actively flush l2_table_cache,when qemu process exception exit,causing disk data loss Signed-off-by: Evanzhang --- block/stream.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/block/stream.c b/block/stream.c index e522bbd..a5e08da 100644 --- a/block/stream.c +++

Re: [PATCH] hw/sd/sdhci: Do not force sdhci_mmio_*_ops onto all SD controllers

2023-07-24 Thread Philippe Mathieu-Daudé
On 24/7/23 15:44, Guenter Roeck wrote: On 7/24/23 00:18, Bernhard Beschow wrote: Ping^2 I would like to have the bug fixed in 8.1. +1 Not that I care too much - I build qemu myself anyway and carry the patch locally - but this really should get fixed. Guenter Patch queued, thanks!

Re: [PATCH] hw/sd/sdhci: Do not force sdhci_mmio_*_ops onto all SD controllers

2023-07-24 Thread Guenter Roeck
On 7/24/23 00:18, Bernhard Beschow wrote: Am 16. Juli 2023 19:53:37 UTC schrieb Bernhard Beschow : Am 10. Juli 2023 16:01:46 UTC schrieb Bernhard Beschow : Am 10. Juli 2023 10:16:35 UTC schrieb "Philippe Mathieu-Daudé" : On 9/7/23 10:09, Bernhard Beschow wrote: Since commit c0a55a0c9da

Re: [PATCH v2 0/3] qemu-img: map: implement support for compressed clusters

2023-07-24 Thread Andrey Drobyshev
On 7/6/23 19:30, Andrey Drobyshev wrote: > v1 --> v2: > * Add vmdk format to the 1st commit. Tweak commit message accordingly; > * Make "compressed" field in MapEntry optional. > > v1: https://lists.nongnu.org/archive/html/qemu-block/2023-06/msg00184.html > > Andrey Drobyshev (3): > block:

Re: [PATCH v2 0/3] block: align CoR requests to subclusters

2023-07-24 Thread Andrey Drobyshev
On 7/11/23 20:25, Andrey Drobyshev wrote: > v1 --> v2: > * Fixed line indentation; > * Fixed wording in a comment; > * Added R-b. > > v1: https://lists.nongnu.org/archive/html/qemu-block/2023-06/msg00606.html > > Andrey Drobyshev (3): > block: add subcluster_size field to BlockDriverInfo >

Re: [PATCH 0/6] qemu-img: rebase: add compression support

2023-07-24 Thread Andrey Drobyshev
On 6/30/23 13:54, Denis V. Lunev wrote: > On 6/1/23 21:28, Andrey Drobyshev wrote: >> This series is adding [-c | --compress] option to "qemu-img rebase" >> command, which might prove useful for saving some disk space when, for >> instance, manipulating chains of backup images.  Along the way I had

[PATCH v4 9/9] block/throttle-groups: Use ThrottleDirection instread of bool is_write

2023-07-24 Thread zhenwei pi
'bool is_write' style is obsolete from throttle framework, adapt block throttle groups to the new style. Use a simple python script to test the new style: #!/usr/bin/python3 import subprocess import random import time commands = ['virsh blkdeviotune jammy vda --write-bytes-sec ', \ '

[PATCH v4 8/9] fsdev: Use ThrottleDirection instread of bool is_write

2023-07-24 Thread zhenwei pi
'bool is_write' style is obsolete from throttle framework, adapt fsdev to the new style. Cc: Greg Kurz Signed-off-by: zhenwei pi --- fsdev/qemu-fsdev-throttle.c | 14 +++--- fsdev/qemu-fsdev-throttle.h | 4 ++-- hw/9pfs/cofile.c| 4 ++-- 3 files changed, 11 insertions(+),

[PATCH v4 5/9] cryptodev: use NULL throttle timer cb for read direction

2023-07-24 Thread zhenwei pi
Operations on a cryptodev are considered as *write* only, the callback of read direction is never invoked. Use NULL instead of an unreachable path(cryptodev_backend_throttle_timer_cb on read direction). The dummy read timer(never invoked) is already removed here, it means that the 'FIXME' tag is n

[PATCH v4 1/9] throttle: introduce enum ThrottleDirection

2023-07-24 Thread zhenwei pi
Use enum ThrottleDirection instead of number index. Reviewed-by: Alberto Garcia Reviewed-by: Hanna Czenczek Signed-off-by: zhenwei pi --- include/qemu/throttle.h | 11 --- util/throttle.c | 16 +--- 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/in

[PATCH v4 7/9] throttle: use THROTTLE_MAX/ARRAY_SIZE for hard code

2023-07-24 Thread zhenwei pi
The first dimension of both to_check and bucket_types_size/bucket_types_units is used as throttle direction, use THROTTLE_MAX instead of hard coded number. Also use ARRAY_SIZE() to avoid hard coded number for the second dimension. Hanna noticed that the two array should be static. Yes, turn them i

[PATCH v4 2/9] test-throttle: use enum ThrottleDirection

2023-07-24 Thread zhenwei pi
Use enum ThrottleDirection instead in the throttle test codes. Reviewed-by: Alberto Garcia Reviewed-by: Hanna Czenczek Signed-off-by: zhenwei pi --- tests/unit/test-throttle.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/unit/test-throttle.c b/tests/unit/test

[PATCH v4 6/9] throttle: use enum ThrottleDirection instead of bool is_write

2023-07-24 Thread zhenwei pi
enum ThrottleDirection is already there, use ThrottleDirection instead of 'bool is_write' for throttle API, also modify related codes from block, fsdev, cryptodev and tests. Signed-off-by: zhenwei pi --- backends/cryptodev.c| 9 + block/throttle-groups.c | 6 -- fsdev/q

[PATCH v4 3/9] throttle: support read-only and write-only

2023-07-24 Thread zhenwei pi
Only one direction is necessary in several scenarios: - a read-only disk - operations on a device are considered as *write* only. For example, encrypt/decrypt/sign/verify operations on a cryptodev use a single *write* timer(read timer callback is defined, but never invoked). Allow a single dir

[PATCH v4 4/9] test-throttle: test read only and write only

2023-07-24 Thread zhenwei pi
Reviewed-by: Alberto Garcia Reviewed-by: Hanna Czenczek Signed-off-by: zhenwei pi --- tests/unit/test-throttle.c | 66 ++ 1 file changed, 66 insertions(+) diff --git a/tests/unit/test-throttle.c b/tests/unit/test-throttle.c index a60b5fe22e..5547837a58 10064

[PATCH v4 0/9] Misc fixes for throttle

2023-07-24 Thread zhenwei pi
v3 -> v4: - Hanna pointed out that 'throttle type' is not clear enough, 'throttle direction' would be better in the v3. Use 'ThrottleDirection' instead, also rename 'ThrottleType throttle' to 'ThrottleDirection direction'. - For patch 'throttle: support read-only and write-only', reduce codes

Re: [Libguestfs] [libnbd PATCH v4 6/6] states: Break deadlock if server goofs on extended replies

2023-07-24 Thread Laszlo Ersek
On 7/21/23 18:08, Eric Blake wrote: > One of the benefits of extended replies is that we can do a > fixed-length read for the entire header of every server reply, which > is fewer syscalls than the split-read approach required by structured > replies. But one of the drawbacks of doing a large read

Re: [Libguestfs] [libnbd PATCH v4 5/6] states: Prepare to receive 64-bit replies

2023-07-24 Thread Laszlo Ersek
On 7/21/23 18:08, Eric Blake wrote: > Support receiving headers for 64-bit replies if extended headers were > negotiated. We already insist that the server not send us too much > payload in one reply, so we can exploit that and continue to use > h->payload_left as a 32-bit length for the rest of t

Re: [Libguestfs] [libnbd PATCH v4 4/6] states: Prepare to send 64-bit requests

2023-07-24 Thread Laszlo Ersek
On 7/21/23 18:08, Eric Blake wrote: > Support sending 64-bit requests if extended headers were negotiated. > This includes setting NBD_CMD_FLAG_PAYLOAD_LEN any time we send an > extended NBD_CMD_WRITE; this is such a fundamental part of the > protocol that for now it is easier to silently ignore wh

Re: [Libguestfs] [libnbd PATCH v4 1/6] internal: Track chunk payload length left

2023-07-24 Thread Laszlo Ersek
On 7/21/23 18:08, Eric Blake wrote: > While working on later patches, I noticed that I was frequently > referring to the wire contents of h->sbuf.reply.hdr.structured.length, > byte-swapping it, then repeating open-coded math for how many bytes > were consumed in earlier states. It's easier to nor

Re: [Libguestfs] [libnbd PATCH v4 2/6] block_status: Refactor array storage

2023-07-24 Thread Laszlo Ersek
On 7/21/23 18:08, Eric Blake wrote: > For 32-bit block status, we were able to cheat and use an array with > an odd number of elements, with array[0] holding the context id, and > passing &array[1] to the user's callback. But once we have 64-bit > extents, we can no longer abuse array element 0 li

Re: [libnbd PATCH v4 3/6] protocol: Add definitions for extended headers

2023-07-24 Thread Laszlo Ersek
On 7/21/23 18:08, Eric Blake wrote: > Add the magic numbers and new structs necessary to implement the NBD > protocol extension of extended headers providing 64-bit lengths. This > corresponds to upstream nbd commits 36abf47d and a9384e2f on the > extension-ext-header branch[1] (commit e6f3b94a fo

Re: [PATCH] hw/sd/sdhci: Do not force sdhci_mmio_*_ops onto all SD controllers

2023-07-24 Thread Bernhard Beschow
Am 16. Juli 2023 19:53:37 UTC schrieb Bernhard Beschow : > > >Am 10. Juli 2023 16:01:46 UTC schrieb Bernhard Beschow : >> >> >>Am 10. Juli 2023 10:16:35 UTC schrieb "Philippe Mathieu-Daudé" >>: >>>On 9/7/23 10:09, Bernhard Beschow wrote: Since commit c0a55a0c9da2 "hw/sd/sdhci: Support big