Re: [PATCH 3/4] libqtest: fix memory leak in the qtest_qmp_event_ref
On 19/10/2020 18.37, Maxim Levitsky wrote:
> The g_list_remove_link doesn't free the link element,
> opposed to what I thought.
> Switch to g_list_delete_link that does free it.
>
> Also refactor the code a bit.
> Thanks for Max Reitz for helping me with this.
>
> Signed-off-by: Maxim Levitsky
> ---
> tests/qtest/libqtest.c | 11 ---
> 1 file changed, 4 insertions(+), 7 deletions(-)
>
> diff --git a/tests/qtest/libqtest.c b/tests/qtest/libqtest.c
> index bd96cb6fdd..9ae052d566 100644
> --- a/tests/qtest/libqtest.c
> +++ b/tests/qtest/libqtest.c
> @@ -795,15 +795,12 @@ void qtest_qmp_send_raw(QTestState *s, const char *fmt,
> ...)
>
> QDict *qtest_qmp_event_ref(QTestState *s, const char *event)
> {
> -GList *next = NULL;
> -QDict *response;
> -
> -for (GList *it = s->pending_events; it != NULL; it = next) {
> +while (s->pending_events) {
>
> -next = it->next;
> -response = (QDict *)it->data;
> +GList *first = s->pending_events;
> +QDict *response = (QDict *)first->data;
>
> -s->pending_events = g_list_remove_link(s->pending_events, it);
> +s->pending_events = g_list_delete_link(s->pending_events, first);
>
> if (!strcmp(qdict_get_str(response, "event"), event)) {
> return response;
>
Thanks, queued (together with patch 2) to qtest-next:
https://gitlab.com/huth/qemu/-/commits/qtest-next/
Thomas
Re: [PATCH 1/4] qdev: Fix two typos
On 19/10/2020 18.36, Maxim Levitsky wrote:
> Signed-off-by: Maxim Levitsky
> ---
> include/hw/qdev-core.h | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/include/hw/qdev-core.h b/include/hw/qdev-core.h
> index 868973319e..3761186804 100644
> --- a/include/hw/qdev-core.h
> +++ b/include/hw/qdev-core.h
> @@ -163,8 +163,8 @@ struct NamedClockList {
> /**
> * DeviceState:
> * @realized: Indicates whether the device has been fully constructed.
> - *When accessed outsize big qemu lock, must be accessed with
> - *atomic_load_acquire()
> + *When accessed outside big qemu lock, must be accessed with
> + *qatomic_load_acquire()
> * @reset: ResettableState for the device; handled by Resettable interface.
> *
> * This structure should not be accessed directly. We declare it here
>
Reviewed-by: Thomas Huth
[PATCH] hw/sd: Zero out function selection fields before being populated
From: Bin Meng
The function selection fields (399:376) should be zeroed out to
prevent leftover from being or'ed into the switch function status
data structure.
This fixes the boot failure as seen in the acceptance testing on
the orangepi target.
Fixes: b638627c723a ("hw/sd: Fix incorrect populated function switch status
data structure")
Reported-by: Michael Roth
Signed-off-by: Bin Meng
---
hw/sd/sd.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/hw/sd/sd.c b/hw/sd/sd.c
index c3febed243..bd10ec8fc4 100644
--- a/hw/sd/sd.c
+++ b/hw/sd/sd.c
@@ -824,6 +824,7 @@ static void sd_function_switch(SDState *sd, uint32_t arg)
sd->data[12] = 0x80; /* Supported group 1 functions */
sd->data[13] = 0x03;
+memset(&sd->data[14], 0, 3);
for (i = 0; i < 6; i ++) {
new_func = (arg >> (i * 4)) & 0x0f;
if (mode && new_func != 0x0f)
--
2.25.1
Re: [RFC PATCH 0/4] tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC
Hi Philippe, On Sat, Oct 24, 2020 at 1:56 AM Philippe Mathieu-Daudé wrote: > > On 10/23/20 7:42 PM, Bin Meng wrote: > > Hi Philippe, > > > > On Fri, Oct 23, 2020 at 9:18 PM Philippe Mathieu-Daudé > > wrote: > >> > >> Series meant to help Bin Meng to debug the SD card issue > >> reported by Michael Roth. > > > > Thank you for the patches. > > > >> > >> Philippe Mathieu-Daudé (4): > >>Revert "hw/sd: Fix incorrect populated function switch status data > >> structure" > >>tests/acceptance: Allow running Orange Pi test using cached artifacts > >>tests/acceptance: Extract do_test_arm_orangepi_armbian_uboot() method > >>tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC > >> > >> hw/sd/sd.c | 3 +- > >> tests/acceptance/boot_linux_console.py | 68 +++--- > >> 2 files changed, 50 insertions(+), 21 deletions(-) > > > > With this series, I used: > > > > $ ARMBIAN_ARTIFACTS_CACHED=1 AVOCADO_ALLOW_LARGE_STORAGE=1 make > > check-acceptance > > > > It looks that the failure still exists? Log below: > > > > 13-tests_acceptance_boot_linux_console.py_BootLinuxConsole.test_arm_orangepi_bionic_20_08/debug.log: > > > > 01:11:27 DEBUG| => boot > > 01:11:27 DEBUG| unable to select a mode > > 01:11:27 DEBUG| Device 0: unknown device > > 01:11:27 DEBUG| BOOTP broadcast 1 > > 01:11:27 DEBUG| DHCP client bound to address 10.0.2.15 (1 ms) > > 01:11:27 DEBUG| *** Warning: no boot file name; using '0A00020F.img' > > 01:11:27 DEBUG| Using ethernet@1c3 device > > 01:11:27 DEBUG| TFTP from server 10.0.2.2; our IP address is 10.0.2.15 > > 01:11:27 DEBUG| Filename '0A00020F.img'. > > 01:11:27 DEBUG| Load address: 0x4200 > > 01:11:27 DEBUG| Loading: *^H > > 01:11:27 DEBUG| TFTP error: 'Access violation' (2) > > 01:11:27 DEBUG| Not retrying... > > Have you rebuilt qemu-system-arm with the reverted patch included? Oops, I took it for granted that the `make check-acceptance` will automatically rebuild the QEMU binary, which is not the case. Should we enforce the rebuild before testing in Makefiles? Regards, Bin
Re: [PATCH v5 12/12] qapi: Use QAPI_LIST_ADD() where possible
On 10/23/20 1:36 PM, Eric Blake wrote:
> Anywhere we create a list of just one item or by prepending items
> (typically because order doesn't matter), we can use the now-public
> macro. But places where we must keep the list in order by appending
> remain open-coded.
>
> Signed-off-by: Eric Blake
> ---
> docs/devel/writing-qmp-commands.txt | 13 +++--
> +++ b/docs/devel/writing-qmp-commands.txt
> @@ -531,15 +531,10 @@ TimerAlarmMethodList *qmp_query_alarm_methods(Error
> **errp)
> bool current = true;
>
> for (p = alarm_timers; p->name; p++) {
> -TimerAlarmMethodList *info = g_malloc0(sizeof(*info));
> -info->value = g_malloc0(sizeof(*info->value));
> -info->value->method_name = g_strdup(p->name);
> -info->value->current = current;
> -
> -current = false;
> -
> -info->next = method_list;
> -method_list = info;
> + TimerAlarmMethod *value = g_new0(TimerAlarmMethod, 1);
> +value->method_name = g_strdup(p->name);
Oops, tab damage.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[PATCH v5 12/12] qapi: Use QAPI_LIST_ADD() where possible
Anywhere we create a list of just one item or by prepending items
(typically because order doesn't matter), we can use the now-public
macro. But places where we must keep the list in order by appending
remain open-coded.
Signed-off-by: Eric Blake
---
docs/devel/writing-qmp-commands.txt | 13 +++--
hw/net/rocker/rocker_fp.h | 2 +-
block/gluster.c | 19 +
chardev/char.c | 21 +++
hw/core/machine.c | 6 +
hw/net/rocker/rocker.c | 8 +++---
hw/net/rocker/rocker_fp.c | 14 +-
hw/net/virtio-net.c | 21 +--
migration/migration.c | 7 ++---
migration/postcopy-ram.c| 7 ++---
monitor/hmp-cmds.c | 11
qemu-img.c | 5 ++--
qga/commands-posix.c| 13 +++--
qga/commands-win32.c| 17 +++-
qga/commands.c | 6 +
qom/qom-qmp-cmds.c | 29 ++--
target/arm/helper.c | 6 +
target/arm/monitor.c| 13 ++---
target/i386/cpu.c | 6 +
target/mips/helper.c| 6 +
target/s390x/cpu_models.c | 12 ++---
tests/test-clone-visitor.c | 7 ++---
tests/test-qobject-output-visitor.c | 42 ++---
tests/test-visitor-serialization.c | 5 +---
trace/qmp.c | 22 +++
ui/vnc.c| 21 +--
util/qemu-config.c | 14 +++---
target/ppc/translate_init.c.inc | 12 ++---
28 files changed, 119 insertions(+), 246 deletions(-)
diff --git a/docs/devel/writing-qmp-commands.txt
b/docs/devel/writing-qmp-commands.txt
index 46a6c48683f5..3e11eeaa1893 100644
--- a/docs/devel/writing-qmp-commands.txt
+++ b/docs/devel/writing-qmp-commands.txt
@@ -531,15 +531,10 @@ TimerAlarmMethodList *qmp_query_alarm_methods(Error
**errp)
bool current = true;
for (p = alarm_timers; p->name; p++) {
-TimerAlarmMethodList *info = g_malloc0(sizeof(*info));
-info->value = g_malloc0(sizeof(*info->value));
-info->value->method_name = g_strdup(p->name);
-info->value->current = current;
-
-current = false;
-
-info->next = method_list;
-method_list = info;
+ TimerAlarmMethod *value = g_new0(TimerAlarmMethod, 1);
+value->method_name = g_strdup(p->name);
+value->current = current;
+QAPI_LIST_ADD(method_list, value);
}
return method_list;
diff --git a/hw/net/rocker/rocker_fp.h b/hw/net/rocker/rocker_fp.h
index dbe1dd329a4b..4cb0bb9ccf81 100644
--- a/hw/net/rocker/rocker_fp.h
+++ b/hw/net/rocker/rocker_fp.h
@@ -28,7 +28,7 @@ int fp_port_eg(FpPort *port, const struct iovec *iov, int
iovcnt);
char *fp_port_get_name(FpPort *port);
bool fp_port_get_link_up(FpPort *port);
-void fp_port_get_info(FpPort *port, RockerPortList *info);
+void fp_port_get_info(FpPort *port, RockerPort *info);
void fp_port_get_macaddr(FpPort *port, MACAddr *macaddr);
void fp_port_set_macaddr(FpPort *port, MACAddr *macaddr);
uint8_t fp_port_get_learning(FpPort *port);
diff --git a/block/gluster.c b/block/gluster.c
index 4f1448e2bc88..cf446c23f85d 100644
--- a/block/gluster.c
+++ b/block/gluster.c
@@ -359,8 +359,8 @@ static int qemu_gluster_parse_uri(BlockdevOptionsGluster
*gconf,
return -EINVAL;
}
-gconf->server = g_new0(SocketAddressList, 1);
-gconf->server->value = gsconf = g_new0(SocketAddress, 1);
+gsconf = g_new0(SocketAddress, 1);
+QAPI_LIST_ADD(gconf->server, gsconf);
/* transport */
if (!uri->scheme || !strcmp(uri->scheme, "gluster")) {
@@ -514,7 +514,7 @@ static int qemu_gluster_parse_json(BlockdevOptionsGluster
*gconf,
{
QemuOpts *opts;
SocketAddress *gsconf = NULL;
-SocketAddressList *curr = NULL;
+SocketAddressList **curr;
QDict *backing_options = NULL;
Error *local_err = NULL;
char *str = NULL;
@@ -547,6 +547,7 @@ static int qemu_gluster_parse_json(BlockdevOptionsGluster
*gconf,
}
gconf->path = g_strdup(ptr);
qemu_opts_del(opts);
+curr = &gconf->server;
for (i = 0; i < num_servers; i++) {
str = g_strdup_printf(GLUSTER_OPT_SERVER_PATTERN"%d.", i);
@@ -655,15 +656,9 @@ static int qemu_gluster_parse_json(BlockdevOptionsGluster
*gconf,
qemu_opts_del(opts);
}
-if (gconf->server == NULL) {
-gconf->server = g_new0(SocketAddressList, 1);
-gconf->server->value = gsconf;
-curr = gconf->server;
-} else {
-curr->next = g_new0(SocketAddressList, 1);
-curr->next->value = gsconf;
-curr = curr->next;
-}
+*curr = g_new0(SocketAddressList, 1);
+(*curr)->value = gsconf;
+curr = &(*curr)->next;
[PATCH v5 11/12] nbd: Expose actual depth in qemu:allocation-depth
Preserve the tri-state encoding in the low bits, as that still remains
a valuable way to utilize qemu-img map with x-dirty-bitmap for
accessing quick information without needing a third-party NBD client.
But now that the block layer gives us an actual depth, we can easily
expose it in the remaining bits of our metadata context (leaving two
bits reserved, to make it easier to read depth out of a raw hex
number). This assumes no one runs a backing chain larger than 256M
elements.
iotest 309 remains unchanged (an example of the above-mentioned
x-dirty-bitmap hack); actually testing the new bits requires libnbd or
a similar client, and I didn't want to make iotests depend on libnbd
at this point in time; rather, see the libnbd project for interop
tests that exercise this new feature.
Signed-off-by: Eric Blake
---
docs/interop/nbd.txt | 6 +-
include/block/nbd.h | 2 ++
nbd/server.c | 27 +++
3 files changed, 22 insertions(+), 13 deletions(-)
diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt
index 7e948bd42218..d90723ffe991 100644
--- a/docs/interop/nbd.txt
+++ b/docs/interop/nbd.txt
@@ -34,7 +34,8 @@ the image, with a single metadata context named:
qemu:allocation-depth
-In the allocation depth context, bits 0 and 1 form a tri-state value:
+In the allocation depth context, bits 0 and 1 form a tri-state value,
+along with 28 bits giving an actual depth:
bits 0-1: 00: NBD_STATE_DEPTH_UNALLOC, the extent is unallocated
01: NBD_STATE_DEPTH_LOCAL, the extent is allocated in the
@@ -42,6 +43,9 @@ In the allocation depth context, bits 0 and 1 form a
tri-state value:
10: NBD_STATE_DEPTH_BACKING, the extent is inherited from a
backing layer
11: invalid, never returned
+bits 2-3: reserved, always 0
+bits 4-31: NBD_STATE_DEPTH_RAW, the backing layer depth (0 if
+ UNALLOC, 1 for LOCAL, 2 or more for BACKING)
For NBD_OPT_LIST_META_CONTEXT the following queries are supported
in addition to the specific "qemu:allocation-depth" and
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 956687f5c368..3c0692aec642 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -264,6 +264,8 @@ enum {
#define NBD_STATE_DEPTH_UNALLOC 0x0
#define NBD_STATE_DEPTH_LOCAL0x1
#define NBD_STATE_DEPTH_BACKING 0x2
+#define NBD_STATE_DEPTH_RAW_MASK 0xfff0
+#define NBD_STATE_DEPTH_RAW_SHIFT4
static inline bool nbd_reply_type_is_error(int type)
{
diff --git a/nbd/server.c b/nbd/server.c
index 53526090b0a2..afa79e63a7a6 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -2037,22 +2037,25 @@ static int blockalloc_to_extents(BlockDriverState *bs,
uint64_t offset,
while (bytes) {
uint32_t flags;
int64_t num;
-int ret = bdrv_is_allocated(bs, offset, bytes, &num);
+int depth = bdrv_is_allocated_above(bs, NULL, false, offset, bytes,
+&num);
-if (ret < 0) {
-return ret;
-}
-
-if (ret == 1) {
+switch (depth) {
+case 0:
+flags = NBD_STATE_DEPTH_UNALLOC;
+break;
+case 1:
flags = NBD_STATE_DEPTH_LOCAL;
-} else {
-ret = bdrv_is_allocated_above(bs, NULL, false, offset, num,
- &num);
-if (ret < 0) {
-return ret;
+break;
+default:
+if (depth < 0) {
+return depth;
}
-flags = ret ? NBD_STATE_DEPTH_BACKING : NBD_STATE_DEPTH_UNALLOC;
+flags = NBD_STATE_DEPTH_BACKING;
+break;
}
+assert(depth <= UINT32_MAX >> NBD_STATE_DEPTH_RAW_SHIFT);
+flags |= depth << NBD_STATE_DEPTH_RAW_SHIFT;
if (nbd_extent_array_add(ea, num, flags) < 0) {
return 0;
--
2.29.0
Re: [PATCH v5 00/12] Exposing backing-chain allocation over NBD
On 10/23/20 1:36 PM, Eric Blake wrote: > v4 was here: > https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg02708.html > > Since then: > - rebase to master > - patches 1, 2, and 12 are new based on Vladimir's observation of > QAPI_LIST_ADD > - patches 10-11 are new based on prior discussion on exposing actual > depth in addition to a tri-state encoding > - patch 3 has a nasty bug fixed that was causing iotest failures > - patch 6 updated to take advantage of patch 2 > - other minor tweaks based on review comments > > 001/12:[down] 'qapi: Move GenericList to qapi/util.h' > 002/12:[down] 'qapi: Make QAPI_LIST_ADD() public' > 003/12:[0002] [FC] 'nbd: Utilize QAPI_CLONE for type conversion' > 004/12:[0010] [FC] 'nbd: Add new qemu:allocation-depth metadata context' > 005/12:[] [--] 'nbd: Add 'qemu-nbd -A' to expose allocation depth' > 006/12:[0014] [FC] 'nbd: Update qapi to support exporting multiple bitmaps' > 007/12:[] [--] 'nbd: Simplify qemu bitmap context name' > 008/12:[] [--] 'nbd: Refactor counting of metadata contexts' > 009/12:[0017] [FC] 'nbd: Allow export of multiple bitmaps for one device' > 010/12:[down] 'block: Return depth level during bdrv_is_allocated_above' > 011/12:[down] 'nbd: Expose actual depth in qemu:allocation-depth' > 012/12:[down] 'qapi: Use QAPI_LIST_ADD() where possible' and I meant to add: Also available at: https://repo.or.cz/qemu/ericb.git/shortlog/refs/tags/nbd-alloc-depth-v5 patch 12 is the largest; it may be worth splitting that by maintainer, or even pushing it off post-5.2. Logically, it can go in anywhere after patch 2, but by putting it last, I'm hoping to send a pull request for soft freeze next week for patches 1-11 for sure, and only include 12 if we get enough positive review in time. I did not try to see if Coccinelle could make the work done in patch 12 more automatable. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
[PATCH v5 09/12] nbd: Allow export of multiple bitmaps for one device
With this, 'qemu-nbd -B b0 -B b1 -f qcow2 img.qcow2' can let you sniff
out multiple bitmaps from one server. qemu-img as client can still
only read one bitmap per client connection, but other NBD clients
(hello libnbd) can now read multiple bitmaps in a single pass.
Signed-off-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
---
nbd/server.c | 100 -
tests/qemu-iotests/291 | 6 +--
2 files changed, 72 insertions(+), 34 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index 27d943529409..53526090b0a2 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -28,6 +28,7 @@
#define NBD_META_ID_BASE_ALLOCATION 0
#define NBD_META_ID_ALLOCATION_DEPTH 1
+/* Dirty bitmaps use 'NBD_META_ID_DIRTY_BITMAP + i', so keep this id last. */
#define NBD_META_ID_DIRTY_BITMAP 2
/*
@@ -96,7 +97,8 @@ struct NBDExport {
Notifier eject_notifier;
bool allocation_depth;
-BdrvDirtyBitmap *export_bitmap;
+BdrvDirtyBitmap **export_bitmaps;
+size_t nr_export_bitmaps;
};
static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports);
@@ -109,7 +111,10 @@ typedef struct NBDExportMetaContexts {
size_t count; /* number of negotiated contexts */
bool base_allocation; /* export base:allocation context (block status) */
bool allocation_depth; /* export qemu:allocation-depth */
-bool bitmap; /* export qemu:dirty-bitmap: */
+bool *bitmaps; /*
+* export qemu:dirty-bitmap:,
+* sized by exp->nr_export_bitmaps
+*/
} NBDExportMetaContexts;
struct NBDClient {
@@ -861,6 +866,8 @@ static bool nbd_meta_base_query(NBDClient *client,
NBDExportMetaContexts *meta,
static bool nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta,
const char *query)
{
+size_t i;
+
if (!nbd_strshift(&query, "qemu:")) {
return false;
}
@@ -869,7 +876,7 @@ static bool nbd_meta_qemu_query(NBDClient *client,
NBDExportMetaContexts *meta,
if (!*query) {
if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
meta->allocation_depth = meta->exp->allocation_depth;
-meta->bitmap = !!meta->exp->export_bitmap;
+memset(meta->bitmaps, 1, meta->exp->nr_export_bitmaps);
}
trace_nbd_negotiate_meta_query_parse("empty");
return true;
@@ -882,17 +889,26 @@ static bool nbd_meta_qemu_query(NBDClient *client,
NBDExportMetaContexts *meta,
}
if (nbd_strshift(&query, "dirty-bitmap:")) {
-const char *bm_name;
-
trace_nbd_negotiate_meta_query_parse("dirty-bitmap:");
-if (!meta->exp->export_bitmap) {
-trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported");
+if (!*query) {
+if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
+memset(meta->bitmaps, 1, meta->exp->nr_export_bitmaps);
+}
+trace_nbd_negotiate_meta_query_parse("empty");
return true;
}
-bm_name = bdrv_dirty_bitmap_name(meta->exp->export_bitmap);
-if (nbd_meta_empty_or_pattern(client, bm_name, query)) {
-meta->bitmap = true;
+
+for (i = 0; i < meta->exp->nr_export_bitmaps; i++) {
+const char *bm_name;
+
+bm_name = bdrv_dirty_bitmap_name(meta->exp->export_bitmaps[i]);
+if (strcmp(bm_name, query) == 0) {
+meta->bitmaps[i] = true;
+trace_nbd_negotiate_meta_query_parse(query);
+return true;
+}
}
+trace_nbd_negotiate_meta_query_skip("no dirty-bitmap match");
return true;
}
@@ -954,9 +970,10 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
{
int ret;
g_autofree char *export_name = NULL;
-NBDExportMetaContexts local_meta;
+g_autofree bool *bitmaps = NULL;
+NBDExportMetaContexts local_meta = {0};
uint32_t nb_queries;
-int i;
+size_t i;
size_t count = 0;
if (!client->structured_reply) {
@@ -971,6 +988,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
meta = &local_meta;
}
+g_free(meta->bitmaps);
memset(meta, 0, sizeof(*meta));
ret = nbd_opt_read_name(client, &export_name, NULL, errp);
@@ -985,6 +1003,10 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
return nbd_opt_drop(client, NBD_REP_ERR_UNKNOWN, errp,
"export '%s' not present", sane_name);
}
+meta->bitmaps = g_new0(bool, meta->exp->nr_export_bitmaps);
+if (client->opt == NBD_OPT_LIST_META_CONTEXT) {
+bitmaps = meta->bitmaps;
+}
ret = nbd_opt_read(client, &nb_queries, sizeof(nb_queries), false, errp);
if (ret <= 0) {
@@ -998,7 +1020,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
/* enable all known contexts */
meta->base_allocation = true;
meta->allocation_d
[PATCH v5 08/12] nbd: Refactor counting of metadata contexts
Rather than open-code the count of negotiated contexts at several
sites, embed it directly into the struct.
Signed-off-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
---
nbd/server.c | 23 +++
1 file changed, 11 insertions(+), 12 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index 05a8154975a1..27d943529409 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -106,8 +106,7 @@ static QTAILQ_HEAD(, NBDExport) exports =
QTAILQ_HEAD_INITIALIZER(exports);
* NBD_OPT_LIST_META_CONTEXT. */
typedef struct NBDExportMetaContexts {
NBDExport *exp;
-bool valid; /* means that negotiation of the option finished without
- errors */
+size_t count; /* number of negotiated contexts */
bool base_allocation; /* export base:allocation context (block status) */
bool allocation_depth; /* export qemu:allocation-depth */
bool bitmap; /* export qemu:dirty-bitmap: */
@@ -448,7 +447,9 @@ static int nbd_negotiate_handle_list(NBDClient *client,
Error **errp)
static void nbd_check_meta_export(NBDClient *client)
{
-client->export_meta.valid &= client->exp == client->export_meta.exp;
+if (client->exp != client->export_meta.exp) {
+client->export_meta.count = 0;
+}
}
/* Send a reply to NBD_OPT_EXPORT_NAME.
@@ -956,6 +957,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
NBDExportMetaContexts local_meta;
uint32_t nb_queries;
int i;
+size_t count = 0;
if (!client->structured_reply) {
return nbd_opt_invalid(client, errp,
@@ -1013,6 +1015,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
if (ret < 0) {
return ret;
}
+count++;
}
if (meta->allocation_depth) {
@@ -1022,6 +1025,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
if (ret < 0) {
return ret;
}
+count++;
}
if (meta->bitmap) {
@@ -1035,11 +1039,12 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
if (ret < 0) {
return ret;
}
+count++;
}
ret = nbd_negotiate_send_rep(client, NBD_REP_ACK, errp);
if (ret == 0) {
-meta->valid = true;
+meta->count = count;
}
return ret;
@@ -2400,15 +2405,9 @@ static coroutine_fn int nbd_handle_request(NBDClient
*client,
return nbd_send_generic_reply(client, request->handle, -EINVAL,
"need non-zero length", errp);
}
-if (client->export_meta.valid &&
-(client->export_meta.base_allocation ||
- client->export_meta.allocation_depth ||
- client->export_meta.bitmap))
-{
+if (client->export_meta.count) {
bool dont_fragment = request->flags & NBD_CMD_FLAG_REQ_ONE;
-int contexts_remaining = client->export_meta.base_allocation +
-client->export_meta.allocation_depth +
-client->export_meta.bitmap;
+int contexts_remaining = client->export_meta.count;
if (client->export_meta.base_allocation) {
ret = nbd_co_send_block_status(client, request->handle,
--
2.29.0
[PATCH v5 10/12] block: Return depth level during bdrv_is_allocated_above
When checking for allocation across a chain, it's already easy to
count the depth within the chain at which the allocation is found.
Instead of throwing that information away, return it to the caller.
Existing callers only cared about allocated/non-allocated, but having
a depth available will be used by NBD in the next patch.
Note that the previous code (since commit 188a7bbf94 in 2012) was
lazy: for each layer deeper in the backing chain, it was issuing a
bdrv_is_allocated request on the original 'bytes' amount, rather than
on any smaller amount 'n' learned from an upper layer. These
semantics happened to work, since if you have:
Base <- Active
XX---XX-
the consecutive results are offset 0: '11' with *pnum 2, followed by
offset 2: '1' with *pnum 1, followed by offset 3: '0' with *pnum 1;
the resulting sequence 1110 matches reality (the first three clusters
are indeed allocated somewhere in the given range). But post-patch,
we correctly give the results offset 0: '2' with *pnum 1, followed by
offset 1: '11' with *pnum 2, followed by offset 3: '0' with *pnum 1
(2110), without over-reporting the length of contributions from the
backing layers.
Signed-off-by: Eric Blake
---
block/io.c | 11 +++
block/commit.c | 2 +-
block/mirror.c | 2 +-
block/stream.c | 2 +-
4 files changed, 10 insertions(+), 7 deletions(-)
diff --git a/block/io.c b/block/io.c
index 54f0968aee27..4a4fa1c9ab1b 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2414,8 +2414,9 @@ int coroutine_fn bdrv_is_allocated(BlockDriverState *bs,
int64_t offset,
/*
* Given an image chain: ... -> [BASE] -> [INTER1] -> [INTER2] -> [TOP]
*
- * Return 1 if (a prefix of) the given range is allocated in any image
- * between BASE and TOP (BASE is only included if include_base is set).
+ * Return a positive depth if (a prefix of) the given range is allocated
+ * in any image between BASE and TOP (BASE is only included if include_base
+ * is set). Depth 1 is TOP, 2 is the first backing layer, and so forth.
* BASE can be NULL to check if the given offset is allocated in any
* image of the chain. Return 0 otherwise, or negative errno on
* failure.
@@ -2435,6 +2436,7 @@ int bdrv_is_allocated_above(BlockDriverState *top,
{
BlockDriverState *intermediate;
int ret;
+int depth = 0;
int64_t n = bytes;
assert(base || !include_base);
@@ -2444,14 +2446,15 @@ int bdrv_is_allocated_above(BlockDriverState *top,
int64_t pnum_inter;
int64_t size_inter;
+depth++;
assert(intermediate);
-ret = bdrv_is_allocated(intermediate, offset, bytes, &pnum_inter);
+ret = bdrv_is_allocated(intermediate, offset, n, &pnum_inter);
if (ret < 0) {
return ret;
}
if (ret) {
*pnum = pnum_inter;
-return 1;
+return depth;
}
size_inter = bdrv_getlength(intermediate);
diff --git a/block/commit.c b/block/commit.c
index 1e85c306cc41..71db7ba7472e 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -156,7 +156,7 @@ static int coroutine_fn commit_run(Job *job, Error **errp)
/* Copy if allocated above the base */
ret = bdrv_is_allocated_above(blk_bs(s->top), s->base_overlay, true,
offset, COMMIT_BUFFER_SIZE, &n);
-copy = (ret == 1);
+copy = (ret > 0);
trace_commit_one_iteration(s, offset, n, ret);
if (copy) {
assert(n < SIZE_MAX);
diff --git a/block/mirror.c b/block/mirror.c
index 26acf4af6fb7..8e1ad6eceb57 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -846,7 +846,7 @@ static int coroutine_fn mirror_dirty_init(MirrorBlockJob *s)
}
assert(count);
-if (ret == 1) {
+if (ret > 0) {
bdrv_set_dirty_bitmap(s->dirty_bitmap, offset, count);
}
offset += count;
diff --git a/block/stream.c b/block/stream.c
index 8ce6729a33da..236384f2f739 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -167,7 +167,7 @@ static int coroutine_fn stream_run(Job *job, Error **errp)
n = len - offset;
}
-copy = (ret == 1);
+copy = (ret > 0);
}
trace_stream_one_iteration(s, offset, n, ret);
if (copy) {
--
2.29.0
[PATCH v5 06/12] nbd: Update qapi to support exporting multiple bitmaps
Since 'nbd-server-add' is deprecated, and 'block-export-add' is new to
5.2, we can still tweak the interface. Allowing 'bitmaps':['str'] is
nicer than 'bitmap':'str'. This wires up the qapi and qemu-nbd
changes to permit passing multiple bitmaps as distinct metadata
contexts that the NBD client may request, but the actual support for
more than one will require a further patch to the server.
Signed-off-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
---
docs/system/deprecated.rst | 4 +++-
qapi/block-export.json | 18 --
blockdev-nbd.c | 13 +
nbd/server.c | 19 +--
qemu-nbd.c | 10 +-
5 files changed, 46 insertions(+), 18 deletions(-)
diff --git a/docs/system/deprecated.rst b/docs/system/deprecated.rst
index 905628f3a0cb..d6cd027ac740 100644
--- a/docs/system/deprecated.rst
+++ b/docs/system/deprecated.rst
@@ -268,7 +268,9 @@ the 'wait' field, which is only applicable to sockets in
server mode
Use the more generic commands ``block-export-add`` and ``block-export-del``
-instead.
+instead. As part of this deprecation, it is now preferred to export a
+list of dirty bitmaps via ``bitmaps``, rather than a single bitmap via
+``bitmap``.
Human Monitor Protocol (HMP) commands
-
diff --git a/qapi/block-export.json b/qapi/block-export.json
index 893d5cde5dfe..c7c749d61097 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -74,10 +74,10 @@
# @description: Free-form description of the export, up to 4096 bytes.
# (Since 5.0)
#
-# @bitmap: Also export the dirty bitmap reachable from @device, so the
-# NBD client can use NBD_OPT_SET_META_CONTEXT with the
-# metadata context name "qemu:dirty-bitmap:NAME" to inspect the
-# bitmap. (since 4.0)
+# @bitmaps: Also export each of the named dirty bitmaps reachable from
+# @device, so the NBD client can use NBD_OPT_SET_META_CONTEXT with
+# the metadata context name "qemu:dirty-bitmap:BITMAP" to inspect
+# each bitmap. (since 5.2)
#
# @allocation-depth: Also export the allocation depth map for @device, so
#the NBD client can use NBD_OPT_SET_META_CONTEXT with
@@ -88,7 +88,7 @@
##
{ 'struct': 'BlockExportOptionsNbd',
'data': { '*name': 'str', '*description': 'str',
-'*bitmap': 'str', '*allocation-depth': 'bool' } }
+'*bitmaps': ['str'], '*allocation-depth': 'bool' } }
##
# @NbdServerAddOptions:
@@ -100,12 +100,18 @@
# @writable: Whether clients should be able to write to the device via the
#NBD connection (default false).
#
+# @bitmap: Also export a single dirty bitmap reachable from @device, so the
+# NBD client can use NBD_OPT_SET_META_CONTEXT with the metadata
+# context name "qemu:dirty-bitmap:BITMAP" to inspect the bitmap
+# (since 4.0). Mutually exclusive with @bitmaps, and newer
+# clients should use that instead.
+#
# Since: 5.0
##
{ 'struct': 'NbdServerAddOptions',
'base': 'BlockExportOptionsNbd',
'data': { 'device': 'str',
-'*writable': 'bool' } }
+'*writable': 'bool', '*bitmap': 'str' } }
##
# @nbd-server-add:
diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index cee9134b12eb..cfd46223bf4d 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -192,6 +192,19 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error
**errp)
return;
}
+/*
+ * New code should use the list 'bitmaps'; but until this code is
+ * gone, we must support the older single 'bitmap'. Use only one.
+ */
+if (arg->has_bitmap) {
+if (arg->has_bitmaps) {
+error_setg(errp, "Can't mix 'bitmap' and 'bitmaps'");
+return;
+}
+arg->has_bitmaps = true;
+QAPI_LIST_ADD(arg->bitmaps, g_strdup(arg->bitmap));
+}
+
/*
* block-export-add would default to the node-name, but we may have to use
* the device name as a default here for compatibility.
diff --git a/nbd/server.c b/nbd/server.c
index 30cfe0eee467..884ffa00f1bd 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1495,6 +1495,7 @@ static int nbd_export_create(BlockExport *blk_exp,
BlockExportOptions *exp_args,
uint64_t perm, shared_perm;
bool readonly = !exp_args->writable;
bool shared = !exp_args->writable;
+strList *bitmaps;
int ret;
assert(exp_args->type == BLOCK_EXPORT_TYPE_NBD);
@@ -1556,12 +1557,18 @@ static int nbd_export_create(BlockExport *blk_exp,
BlockExportOptions *exp_args,
}
exp->size = QEMU_ALIGN_DOWN(size, BDRV_SECTOR_SIZE);
-if (arg->bitmap) {
+/* XXX Allow more than one bitmap */
+if (arg->bitmaps && arg->bitmaps->next) {
+error_setg(errp, "multiple bitmaps per export not supported yet");
+return -EOPNOTSUPP;
+}
+for (bitm
[PATCH v5 07/12] nbd: Simplify qemu bitmap context name
Each dirty bitmap already knows its name; by reducing the scope of the
places where we construct "qemu:dirty-bitmap:NAME" strings, tracking
the name is more localized, and there are fewer per-export fields to
worry about. This in turn will make it easier for an upcoming patch
to export more than one bitmap at once.
Signed-off-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
---
nbd/server.c | 19 +--
1 file changed, 9 insertions(+), 10 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index 884ffa00f1bd..05a8154975a1 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -97,7 +97,6 @@ struct NBDExport {
bool allocation_depth;
BdrvDirtyBitmap *export_bitmap;
-char *export_bitmap_context;
};
static QTAILQ_HEAD(, NBDExport) exports = QTAILQ_HEAD_INITIALIZER(exports);
@@ -882,14 +881,15 @@ static bool nbd_meta_qemu_query(NBDClient *client,
NBDExportMetaContexts *meta,
}
if (nbd_strshift(&query, "dirty-bitmap:")) {
+const char *bm_name;
+
trace_nbd_negotiate_meta_query_parse("dirty-bitmap:");
if (!meta->exp->export_bitmap) {
trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported");
return true;
}
-if (nbd_meta_empty_or_pattern(client,
- meta->exp->export_bitmap_context +
- strlen("qemu:dirty-bitmap:"), query)) {
+bm_name = bdrv_dirty_bitmap_name(meta->exp->export_bitmap);
+if (nbd_meta_empty_or_pattern(client, bm_name, query)) {
meta->bitmap = true;
}
return true;
@@ -1025,8 +1025,11 @@ static int nbd_negotiate_meta_queries(NBDClient *client,
}
if (meta->bitmap) {
-ret = nbd_negotiate_send_meta_context(client,
- meta->exp->export_bitmap_context,
+const char *bm_name = bdrv_dirty_bitmap_name(meta->exp->export_bitmap);
+g_autofree char *context = g_strdup_printf("qemu:dirty-bitmap:%s",
+ bm_name);
+
+ret = nbd_negotiate_send_meta_context(client, context,
NBD_META_ID_DIRTY_BITMAP,
errp);
if (ret < 0) {
@@ -1599,9 +1602,6 @@ static int nbd_export_create(BlockExport *blk_exp,
BlockExportOptions *exp_args,
bdrv_dirty_bitmap_set_busy(bm, true);
exp->export_bitmap = bm;
assert(strlen(bitmap) <= BDRV_BITMAP_MAX_NAME_SIZE);
-exp->export_bitmap_context = g_strdup_printf("qemu:dirty-bitmap:%s",
- bitmap);
-assert(strlen(exp->export_bitmap_context) < NBD_MAX_STRING_SIZE);
}
exp->allocation_depth = arg->allocation_depth;
@@ -1681,7 +1681,6 @@ static void nbd_export_delete(BlockExport *blk_exp)
if (exp->export_bitmap) {
bdrv_dirty_bitmap_set_busy(exp->export_bitmap, false);
-g_free(exp->export_bitmap_context);
}
}
--
2.29.0
[PATCH v5 02/12] qapi: Make QAPI_LIST_ADD() public
We have a useful macro for inserting at the front of any
QAPI-generated list; move it from block.c to qapi/util.h so more
places can use it, including one earlier place in block.c.
Suggested-by: Vladimir Sementsov-Ogievskiy
Signed-off-by: Eric Blake
---
include/qapi/util.h | 8
block.c | 15 +++
2 files changed, 11 insertions(+), 12 deletions(-)
diff --git a/include/qapi/util.h b/include/qapi/util.h
index 50201896c7a4..b6083055ce69 100644
--- a/include/qapi/util.h
+++ b/include/qapi/util.h
@@ -30,4 +30,12 @@ int qapi_enum_parse(const QEnumLookup *lookup, const char
*buf,
int parse_qapi_name(const char *name, bool complete);
+/* For any GenericList @list, insert @element at the front. */
+#define QAPI_LIST_ADD(list, element) do { \
+typeof(list) _tmp = g_new(typeof(*(list)), 1); \
+_tmp->value = (element); \
+_tmp->next = (list); \
+(list) = _tmp; \
+} while (0)
+
#endif
diff --git a/block.c b/block.c
index 430edf79bb10..45bd79299611 100644
--- a/block.c
+++ b/block.c
@@ -39,6 +39,7 @@
#include "qapi/qmp/qstring.h"
#include "qapi/qobject-output-visitor.h"
#include "qapi/qapi-visit-block-core.h"
+#include "qapi/util.h"
#include "sysemu/block-backend.h"
#include "sysemu/sysemu.h"
#include "qemu/notify.h"
@@ -5211,7 +5212,7 @@ BlockDriverState *bdrv_find_node(const char *node_name)
BlockDeviceInfoList *bdrv_named_nodes_list(bool flat,
Error **errp)
{
-BlockDeviceInfoList *list, *entry;
+BlockDeviceInfoList *list;
BlockDriverState *bs;
list = NULL;
@@ -5221,22 +5222,12 @@ BlockDeviceInfoList *bdrv_named_nodes_list(bool flat,
qapi_free_BlockDeviceInfoList(list);
return NULL;
}
-entry = g_malloc0(sizeof(*entry));
-entry->value = info;
-entry->next = list;
-list = entry;
+QAPI_LIST_ADD(list, info);
}
return list;
}
-#define QAPI_LIST_ADD(list, element) do { \
-typeof(list) _tmp = g_new(typeof(*(list)), 1); \
-_tmp->value = (element); \
-_tmp->next = (list); \
-(list) = _tmp; \
-} while (0)
-
typedef struct XDbgBlockGraphConstructor {
XDbgBlockGraph *graph;
GHashTable *graph_nodes;
--
2.29.0
[PATCH v5 04/12] nbd: Add new qemu:allocation-depth metadata context
'qemu-img map' provides a way to determine which extents of an image
come from the top layer vs. inherited from a backing chain. This is
useful information worth exposing over NBD. There is a proposal to
add a QMP command block-dirty-bitmap-populate which can create a dirty
bitmap that reflects allocation information, at which point the
qemu:dirty-bitmap:NAME metadata context can expose that information
via the creation of a temporary bitmap, but we can shorten the effort
by adding a new qemu:allocation-depth metadata context that does the
same thing without an intermediate bitmap (this patch does not
eliminate the need for that proposal, as it will have other uses as
well).
For this patch, I just encoded a tri-state value (DEPTH_CODE_MASK:
unallocated, from this layer, from any of the backing layers); an
obvious extension would be to provide the actual depth in bits 31-4 (a
new DEPTH_RAW_MASK 0xfff0) while keeping bits 1-0 as a tri-state
(leaving bits 3-2 unused, for ease of reading depth from a hex
number). But adding this extension would require
bdrv_is_allocated_above to return a depth number.
While documenting things, remember that although the NBD protocol has
NBD_OPT_SET_META_CONTEXT, the rest of its documentation refers to
'metadata context', which is a more apt description of what is
actually being used by NBD_CMD_BLOCK_STATUS: the user is requesting
metadata by passing one or more context names. So I also touched up
some existing wording to prefer the term 'metadata context' where it
makes sense.
Note that this patch does not actually enable any way to request a
server to enable this context; that will come in the next patch.
Signed-off-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
---
docs/interop/nbd.txt | 27 ++---
include/block/nbd.h | 12 --
nbd/server.c | 92
3 files changed, 116 insertions(+), 15 deletions(-)
diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt
index f3b3cacc9621..7e948bd42218 100644
--- a/docs/interop/nbd.txt
+++ b/docs/interop/nbd.txt
@@ -17,19 +17,35 @@ namespace "qemu".
== "qemu" namespace ==
-The "qemu" namespace currently contains only one type of context,
-related to exposing the contents of a dirty bitmap alongside the
-associated disk contents. That context has the following form:
+The "qemu" namespace currently contains two available metadata context
+types. The first is related to exposing the contents of a dirty
+bitmap alongside the associated disk contents. That metadata context
+is named with the following form:
qemu:dirty-bitmap:
Each dirty-bitmap metadata context defines only one flag for extents
in reply for NBD_CMD_BLOCK_STATUS:
-bit 0: NBD_STATE_DIRTY, means that the extent is "dirty"
+bit 0: NBD_STATE_DIRTY, set when the extent is "dirty"
+
+The second is related to exposing the source of various extents within
+the image, with a single metadata context named:
+
+qemu:allocation-depth
+
+In the allocation depth context, bits 0 and 1 form a tri-state value:
+
+bits 0-1: 00: NBD_STATE_DEPTH_UNALLOC, the extent is unallocated
+ 01: NBD_STATE_DEPTH_LOCAL, the extent is allocated in the
+ top layer of the image
+ 10: NBD_STATE_DEPTH_BACKING, the extent is inherited from a
+ backing layer
+ 11: invalid, never returned
For NBD_OPT_LIST_META_CONTEXT the following queries are supported
-in addition to "qemu:dirty-bitmap:":
+in addition to the specific "qemu:allocation-depth" and
+"qemu:dirty-bitmap:":
* "qemu:" - returns list of all available metadata contexts in the
namespace.
@@ -55,3 +71,4 @@ the operation of that feature.
NBD_CMD_BLOCK_STATUS for "qemu:dirty-bitmap:", NBD_CMD_CACHE
* 4.2: NBD_FLAG_CAN_MULTI_CONN for shareable read-only exports,
NBD_CMD_FLAG_FAST_ZERO
+* 5.2: NBD_CMD_BLOCK_STATUS for "qemu:allocation-depth"
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 3dd9a04546ec..956687f5c368 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2016-2019 Red Hat, Inc.
+ * Copyright (C) 2016-2020 Red Hat, Inc.
* Copyright (C) 2005 Anthony Liguori
*
* Network Block Device
@@ -47,7 +47,7 @@ typedef struct NBDOptionReply NBDOptionReply;
typedef struct NBDOptionReplyMetaContext {
NBDOptionReply h; /* h.type = NBD_REP_META_CONTEXT, h.length > 4 */
uint32_t context_id;
-/* meta context name follows */
+/* metadata context name follows */
} QEMU_PACKED NBDOptionReplyMetaContext;
/* Transmission phase structs
@@ -229,7 +229,7 @@ enum {
#define NBD_MAX_BUFFER_SIZE (32 * 1024 * 1024)
/*
- * Maximum size of a protocol string (export name, meta context name,
+ * Maximum size of a protocol string (export name, metadata context name,
* etc.). Use malloc rather than stack allocation for storage of a
* string.
*/
@@ -259,6 +259,12 @@ enum {
/* Extent f
[PATCH v5 05/12] nbd: Add 'qemu-nbd -A' to expose allocation depth
Allow the server to expose an additional metacontext to be requested
by savvy clients. qemu-nbd adds a new option -A to expose the
qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this
can also be set via QMP when using block-export-add.
qemu as client can be hacked into viewing this new context by using
the now-misnamed x-dirty-bitmap option when creating an NBD blockdev
(even though our x- naming means we could rename it, I did not think
it worth breaking back-compat of tools that have been using it while
waiting for a better solution). It is worth noting the decoding of
how such context information will appear in 'qemu-img map
--output=json':
NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true
NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false
NBD_STATE_DEPTH_BACKING => "zero":true, "data":true
libnbd as client is probably a nicer way to get at the information
without having to decipher such hacks in qemu as client. ;)
Signed-off-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
---
docs/tools/qemu-nbd.rst| 8 -
qapi/block-core.json | 7 ++--
qapi/block-export.json | 12 +--
nbd/server.c | 2 ++
qemu-nbd.c | 26 +-
tests/qemu-iotests/309 | 73 ++
tests/qemu-iotests/309.out | 22
tests/qemu-iotests/group | 1 +
8 files changed, 136 insertions(+), 15 deletions(-)
create mode 100755 tests/qemu-iotests/309
create mode 100644 tests/qemu-iotests/309.out
diff --git a/docs/tools/qemu-nbd.rst b/docs/tools/qemu-nbd.rst
index 667861cb22e9..fe41336dc550 100644
--- a/docs/tools/qemu-nbd.rst
+++ b/docs/tools/qemu-nbd.rst
@@ -72,10 +72,16 @@ driver options if ``--image-opts`` is specified.
Export the disk as read-only.
+.. option:: -A, --allocation-depth
+
+ Expose allocation depth information via the
+ ``qemu:allocation-depth`` metadata context accessible through
+ NBD_OPT_SET_META_CONTEXT.
+
.. option:: -B, --bitmap=NAME
If *filename* has a qcow2 persistent bitmap *NAME*, expose
- that bitmap via the ``qemu:dirty-bitmap:NAME`` context
+ that bitmap via the ``qemu:dirty-bitmap:NAME`` metadata context
accessible through NBD_OPT_SET_META_CONTEXT.
.. option:: -s, --snapshot
diff --git a/qapi/block-core.json b/qapi/block-core.json
index ee5ebef7f2ce..b6368f5fd9a1 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3883,9 +3883,12 @@
#
# @tls-creds: TLS credentials ID
#
-# @x-dirty-bitmap: A "qemu:dirty-bitmap:NAME" string to query in place of
+# @x-dirty-bitmap: A metadata context name such as "qemu:dirty-bitmap:NAME"
+# or "qemu:allocation-depth" to query in place of the
# traditional "base:allocation" block status (see
-# NBD_OPT_LIST_META_CONTEXT in the NBD protocol) (since 3.0)
+# NBD_OPT_LIST_META_CONTEXT in the NBD protocol; and
+# yes, naming this option x-context would have made
+# more sense) (since 3.0)
#
# @reconnect-delay: On an unexpected disconnect, the nbd client tries to
# connect again until succeeding or encountering a serious
diff --git a/qapi/block-export.json b/qapi/block-export.json
index 65804834d905..893d5cde5dfe 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -75,14 +75,20 @@
# (Since 5.0)
#
# @bitmap: Also export the dirty bitmap reachable from @device, so the
-# NBD client can use NBD_OPT_SET_META_CONTEXT with
-# "qemu:dirty-bitmap:NAME" to inspect the bitmap. (since 4.0)
+# NBD client can use NBD_OPT_SET_META_CONTEXT with the
+# metadata context name "qemu:dirty-bitmap:NAME" to inspect the
+# bitmap. (since 4.0)
+#
+# @allocation-depth: Also export the allocation depth map for @device, so
+#the NBD client can use NBD_OPT_SET_META_CONTEXT with
+#the metadata context name "qemu:allocation-depth" to
+#inspect allocation details. (since 5.2)
#
# Since: 5.0
##
{ 'struct': 'BlockExportOptionsNbd',
'data': { '*name': 'str', '*description': 'str',
-'*bitmap': 'str' } }
+'*bitmap': 'str', '*allocation-depth': 'bool' } }
##
# @NbdServerAddOptions:
diff --git a/nbd/server.c b/nbd/server.c
index ae6f8a8e5429..30cfe0eee467 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1597,6 +1597,8 @@ static int nbd_export_create(BlockExport *blk_exp,
BlockExportOptions *exp_args,
assert(strlen(exp->export_bitmap_context) < NBD_MAX_STRING_SIZE);
}
+exp->allocation_depth = arg->allocation_depth;
+
blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp);
QTAILQ_INSERT_TAIL(&exports, exp, next);
diff --git a/qemu-nbd.c b/qemu-nbd.c
index bc644a0670b6..847fde435a7f 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -99,6 +99,7 @@ static void usage(const char *name)
"\n"
"Exposing part of the
[PATCH v5 00/12] Exposing backing-chain allocation over NBD
v4 was here: https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg02708.html Since then: - rebase to master - patches 1, 2, and 12 are new based on Vladimir's observation of QAPI_LIST_ADD - patches 10-11 are new based on prior discussion on exposing actual depth in addition to a tri-state encoding - patch 3 has a nasty bug fixed that was causing iotest failures - patch 6 updated to take advantage of patch 2 - other minor tweaks based on review comments 001/12:[down] 'qapi: Move GenericList to qapi/util.h' 002/12:[down] 'qapi: Make QAPI_LIST_ADD() public' 003/12:[0002] [FC] 'nbd: Utilize QAPI_CLONE for type conversion' 004/12:[0010] [FC] 'nbd: Add new qemu:allocation-depth metadata context' 005/12:[] [--] 'nbd: Add 'qemu-nbd -A' to expose allocation depth' 006/12:[0014] [FC] 'nbd: Update qapi to support exporting multiple bitmaps' 007/12:[] [--] 'nbd: Simplify qemu bitmap context name' 008/12:[] [--] 'nbd: Refactor counting of metadata contexts' 009/12:[0017] [FC] 'nbd: Allow export of multiple bitmaps for one device' 010/12:[down] 'block: Return depth level during bdrv_is_allocated_above' 011/12:[down] 'nbd: Expose actual depth in qemu:allocation-depth' 012/12:[down] 'qapi: Use QAPI_LIST_ADD() where possible' Eric Blake (12): qapi: Move GenericList to qapi/util.h qapi: Make QAPI_LIST_ADD() public nbd: Utilize QAPI_CLONE for type conversion nbd: Add new qemu:allocation-depth metadata context nbd: Add 'qemu-nbd -A' to expose allocation depth nbd: Update qapi to support exporting multiple bitmaps nbd: Simplify qemu bitmap context name nbd: Refactor counting of metadata contexts nbd: Allow export of multiple bitmaps for one device block: Return depth level during bdrv_is_allocated_above nbd: Expose actual depth in qemu:allocation-depth qapi: Use QAPI_LIST_ADD() where possible docs/devel/writing-qmp-commands.txt | 13 +- docs/interop/nbd.txt| 31 +++- docs/system/deprecated.rst | 4 +- docs/tools/qemu-nbd.rst | 8 +- qapi/block-core.json| 7 +- qapi/block-export.json | 22 ++- include/qapi/visitor.h | 9 +- hw/net/rocker/rocker_fp.h | 2 +- include/block/nbd.h | 14 +- include/qapi/util.h | 16 ++ block/io.c | 11 +- block.c | 15 +- block/commit.c | 2 +- block/gluster.c | 19 +-- block/mirror.c | 2 +- block/stream.c | 2 +- blockdev-nbd.c | 28 ++-- chardev/char.c | 21 ++- hw/core/machine.c | 6 +- hw/net/rocker/rocker.c | 8 +- hw/net/rocker/rocker_fp.c | 14 +- hw/net/virtio-net.c | 21 +-- migration/migration.c | 7 +- migration/postcopy-ram.c| 7 +- monitor/hmp-cmds.c | 11 +- nbd/server.c| 222 ++-- qemu-img.c | 5 +- qemu-nbd.c | 30 ++-- qga/commands-posix.c| 13 +- qga/commands-win32.c| 17 +-- qga/commands.c | 6 +- qom/qom-qmp-cmds.c | 29 +--- target/arm/helper.c | 6 +- target/arm/monitor.c| 13 +- target/i386/cpu.c | 6 +- target/mips/helper.c| 6 +- target/s390x/cpu_models.c | 12 +- tests/test-clone-visitor.c | 7 +- tests/test-qobject-output-visitor.c | 42 +++--- tests/test-visitor-serialization.c | 5 +- trace/qmp.c | 22 ++- ui/vnc.c| 21 +-- util/qemu-config.c | 14 +- target/ppc/translate_init.c.inc | 12 +- tests/qemu-iotests/291 | 6 +- tests/qemu-iotests/309 | 73 + tests/qemu-iotests/309.out | 22 +++ tests/qemu-iotests/group| 1 + 48 files changed, 529 insertions(+), 361 deletions(-) create mode 100755 tests/qemu-iotests/309 create mode 100644 tests/qemu-iotests/309.out -- 2.29.0
[PATCH v5 03/12] nbd: Utilize QAPI_CLONE for type conversion
Rather than open-coding the translation from the deprecated
NbdServerAddOptions type to the preferred BlockExportOptionsNbd, it's
better to utilize QAPI_CLONE_MEMBERS. This solves a couple of issues:
first, if we do any more refactoring of the base type (which an
upcoming patch plans to do), we don't have to revisit the open-coding.
Second, our assignment to arg->name is fishy: the generated QAPI code
currently does not visit it if arg->has_name is false, but if it DID
visit it, we would have introduced a double-free situation when arg is
finally freed.
Signed-off-by: Eric Blake
---
blockdev-nbd.c | 15 ++-
1 file changed, 6 insertions(+), 9 deletions(-)
diff --git a/blockdev-nbd.c b/blockdev-nbd.c
index 8174023e5c47..cee9134b12eb 100644
--- a/blockdev-nbd.c
+++ b/blockdev-nbd.c
@@ -14,6 +14,8 @@
#include "sysemu/block-backend.h"
#include "hw/block/block.h"
#include "qapi/error.h"
+#include "qapi/clone-visitor.h"
+#include "qapi/qapi-visit-block-export.h"
#include "qapi/qapi-commands-block-export.h"
#include "block/nbd.h"
#include "io/channel-socket.h"
@@ -195,7 +197,8 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error
**errp)
* the device name as a default here for compatibility.
*/
if (!arg->has_name) {
-arg->name = arg->device;
+arg->has_name = true;
+arg->name = g_strdup(arg->device);
}
export_opts = g_new(BlockExportOptions, 1);
@@ -205,15 +208,9 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error
**errp)
.node_name = g_strdup(bdrv_get_node_name(bs)),
.has_writable = arg->has_writable,
.writable = arg->writable,
-.u.nbd = {
-.has_name = true,
-.name = g_strdup(arg->name),
-.has_description= arg->has_description,
-.description= g_strdup(arg->description),
-.has_bitmap = arg->has_bitmap,
-.bitmap = g_strdup(arg->bitmap),
-},
};
+QAPI_CLONE_MEMBERS(BlockExportOptionsNbd, &export_opts->u.nbd,
+ qapi_NbdServerAddOptions_base(arg));
/*
* nbd-server-add doesn't complain when a read-only device should be
--
2.29.0
[PATCH v5 01/12] qapi: Move GenericList to qapi/util.h
Placing GenericList in util.h will make it easier for the next patch
to promote QAPI_LIST_ADD() into a public macro without requiring more
files to include the unrelated visitor.h.
However, we can't also move GenericAlternate; this is because it would
introduce a circular dependency: qapi-builtin-types.h needs a complete
definition of QEnumLookup (so it includes qapi/util.h), and
GenericAlternate needs a complete definition of QType (declared in
qapi-builtin-types.h). Leaving GenericAlternate in visitor.h breaks
the cycle, and doesn't matter since we don't have any further planned
uses for that type outside of visitors.
Suggested-by: Markus Armbruster
Signed-off-by: Eric Blake
---
include/qapi/visitor.h | 9 +
include/qapi/util.h| 8
2 files changed, 9 insertions(+), 8 deletions(-)
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index ebc19ede7fff..8c2e1c29ad8b 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -16,6 +16,7 @@
#define QAPI_VISITOR_H
#include "qapi/qapi-builtin-types.h"
+#include "qapi/util.h"
/*
* The QAPI schema defines both a set of C data types, and a QMP wire
@@ -228,14 +229,6 @@
/*** Useful types ***/
-/* This struct is layout-compatible with all other *List structs
- * created by the QAPI generator. It is used as a typical
- * singly-linked list. */
-typedef struct GenericList {
-struct GenericList *next;
-char padding[];
-} GenericList;
-
/* This struct is layout-compatible with all Alternate types
* created by the QAPI generator. */
typedef struct GenericAlternate {
diff --git a/include/qapi/util.h b/include/qapi/util.h
index a7c3c6414874..50201896c7a4 100644
--- a/include/qapi/util.h
+++ b/include/qapi/util.h
@@ -11,6 +11,14 @@
#ifndef QAPI_UTIL_H
#define QAPI_UTIL_H
+/* This struct is layout-compatible with all other *List structs
+ * created by the QAPI generator. It is used as a typical
+ * singly-linked list. */
+typedef struct GenericList {
+struct GenericList *next;
+char padding[];
+} GenericList;
+
typedef struct QEnumLookup {
const char *const *array;
int size;
--
2.29.0
Re: [RFC PATCH 0/4] tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC
On 10/23/20 7:42 PM, Bin Meng wrote: Hi Philippe, On Fri, Oct 23, 2020 at 9:18 PM Philippe Mathieu-Daudé wrote: Series meant to help Bin Meng to debug the SD card issue reported by Michael Roth. Thank you for the patches. Philippe Mathieu-Daudé (4): Revert "hw/sd: Fix incorrect populated function switch status data structure" tests/acceptance: Allow running Orange Pi test using cached artifacts tests/acceptance: Extract do_test_arm_orangepi_armbian_uboot() method tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC hw/sd/sd.c | 3 +- tests/acceptance/boot_linux_console.py | 68 +++--- 2 files changed, 50 insertions(+), 21 deletions(-) With this series, I used: $ ARMBIAN_ARTIFACTS_CACHED=1 AVOCADO_ALLOW_LARGE_STORAGE=1 make check-acceptance It looks that the failure still exists? Log below: 13-tests_acceptance_boot_linux_console.py_BootLinuxConsole.test_arm_orangepi_bionic_20_08/debug.log: 01:11:27 DEBUG| => boot 01:11:27 DEBUG| unable to select a mode 01:11:27 DEBUG| Device 0: unknown device 01:11:27 DEBUG| BOOTP broadcast 1 01:11:27 DEBUG| DHCP client bound to address 10.0.2.15 (1 ms) 01:11:27 DEBUG| *** Warning: no boot file name; using '0A00020F.img' 01:11:27 DEBUG| Using ethernet@1c3 device 01:11:27 DEBUG| TFTP from server 10.0.2.2; our IP address is 10.0.2.15 01:11:27 DEBUG| Filename '0A00020F.img'. 01:11:27 DEBUG| Load address: 0x4200 01:11:27 DEBUG| Loading: *^H 01:11:27 DEBUG| TFTP error: 'Access violation' (2) 01:11:27 DEBUG| Not retrying... Have you rebuilt qemu-system-arm with the reverted patch included? Regards, Bin
Re: [RFC PATCH 0/4] tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC
Hi Philippe, On Fri, Oct 23, 2020 at 9:18 PM Philippe Mathieu-Daudé wrote: > > Series meant to help Bin Meng to debug the SD card issue > reported by Michael Roth. Thank you for the patches. > > Philippe Mathieu-Daudé (4): > Revert "hw/sd: Fix incorrect populated function switch status data > structure" > tests/acceptance: Allow running Orange Pi test using cached artifacts > tests/acceptance: Extract do_test_arm_orangepi_armbian_uboot() method > tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC > > hw/sd/sd.c | 3 +- > tests/acceptance/boot_linux_console.py | 68 +++--- > 2 files changed, 50 insertions(+), 21 deletions(-) With this series, I used: $ ARMBIAN_ARTIFACTS_CACHED=1 AVOCADO_ALLOW_LARGE_STORAGE=1 make check-acceptance It looks that the failure still exists? Log below: 13-tests_acceptance_boot_linux_console.py_BootLinuxConsole.test_arm_orangepi_bionic_20_08/debug.log: 01:11:27 DEBUG| => boot 01:11:27 DEBUG| unable to select a mode 01:11:27 DEBUG| Device 0: unknown device 01:11:27 DEBUG| BOOTP broadcast 1 01:11:27 DEBUG| DHCP client bound to address 10.0.2.15 (1 ms) 01:11:27 DEBUG| *** Warning: no boot file name; using '0A00020F.img' 01:11:27 DEBUG| Using ethernet@1c3 device 01:11:27 DEBUG| TFTP from server 10.0.2.2; our IP address is 10.0.2.15 01:11:27 DEBUG| Filename '0A00020F.img'. 01:11:27 DEBUG| Load address: 0x4200 01:11:27 DEBUG| Loading: *^H 01:11:27 DEBUG| TFTP error: 'Access violation' (2) 01:11:27 DEBUG| Not retrying... Regards, Bin
[PATCH v2 4/6] qapi: Optionally parse simple unions as flat
This extends the Visitor interface with an option that can enable flat
representation (without the 'data' wrapper) for simple unions. This way,
a command line parser can enable a more user friendly syntax while QMP
doesn't enable the option and uses the same representation as before.
We need to disable flat representation for ChardevSpiceChannel, which
has a 'type' option that conflicts with the ChardevBackend 'type'. This
variant will get nesting even when flat parsing is enabled in the
Visitor.
Signed-off-by: Kevin Wolf
---
qapi/char.json | 3 ++-
docs/devel/qapi-code-gen.txt | 11 ++-
include/qapi/visitor-impl.h | 3 +++
include/qapi/visitor.h | 10 ++
qapi/qapi-visit-core.c | 10 ++
scripts/qapi/expr.py | 7 ++-
scripts/qapi/schema.py | 38
scripts/qapi/visit.py| 20 +--
8 files changed, 85 insertions(+), 17 deletions(-)
diff --git a/qapi/char.json b/qapi/char.json
index 43486d1daa..57ec18220b 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -414,7 +414,8 @@
'stdio': 'ChardevStdio',
'console': 'ChardevCommon',
'spicevmc': { 'type': 'ChardevSpiceChannel',
- 'if': 'defined(CONFIG_SPICE)' },
+ 'if': 'defined(CONFIG_SPICE)',
+ 'allow-flat': false },
'spiceport': { 'type': 'ChardevSpicePort',
'if': 'defined(CONFIG_SPICE)' },
'vc': 'ChardevVC',
diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index 9722c1a204..ee34d39a20 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -295,7 +295,9 @@ Syntax:
'*features': FEATURES }
BRANCHES = { BRANCH, ... }
BRANCH = STRING : TYPE-REF
- | STRING : { 'type': TYPE-REF, '*if': COND }
+ | STRING : { 'type': TYPE-REF,
+'*if': COND,
+'*allow-flat': BOOL }
Member 'union' names the union type.
@@ -334,6 +336,13 @@ values to data types like in this example:
'data': { 'file': 'BlockdevOptionsFile',
'qcow2': 'BlockdevOptionsQcow2' } }
+Simple unions can support both wrapped and flat representation of
+branches that have a struct type, unless it is explicitly disabled in
+the schema with 'allow-flat': false. Branches of other types are
+always wrapped. Which representation is used in the generated visitor
+C code can be configured per visitor. Flat representation is
+appropriate when parsing command line options.
+
In the Client JSON Protocol, all simple union branches have wrapped
representation, as in these examples:
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 7362c043be..f628b6eb36 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -121,6 +121,9 @@ struct Visitor
/* Must be set */
void (*free)(Visitor *v);
+
+/* Set to true to make simple unions look like flat unions */
+bool flat_simple_unions;
};
#endif
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index ebc19ede7f..d41be4df48 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -496,6 +496,16 @@ bool visit_is_input(Visitor *v);
*/
bool visit_is_dealloc(Visitor *v);
+/*
+ * Check if simple unions should be treated as flat.
+ */
+bool visit_flat_simple_unions(Visitor *v);
+
+/*
+ * Set if simple unions should be treated as flat.
+ */
+void visit_set_flat_simple_unions(Visitor *v, bool flat);
+
/*** Visiting built-in types ***/
/*
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 7e5f40e7f0..dc6fd78b8c 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -145,6 +145,16 @@ bool visit_is_dealloc(Visitor *v)
return v->type == VISITOR_DEALLOC;
}
+bool visit_flat_simple_unions(Visitor *v)
+{
+return v->flat_simple_unions;
+}
+
+void visit_set_flat_simple_unions(Visitor *v, bool flat)
+{
+v->flat_simple_unions = flat;
+}
+
bool visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp)
{
assert(obj);
diff --git a/scripts/qapi/expr.py b/scripts/qapi/expr.py
index 2fcaaa2497..a39092e4a9 100644
--- a/scripts/qapi/expr.py
+++ b/scripts/qapi/expr.py
@@ -248,7 +248,12 @@ def check_union(expr, info):
for (key, value) in members.items():
source = "'data' member '%s'" % key
check_name_str(key, info, source)
-check_keys(value, info, source, ['type'], ['if'])
+check_keys(value, info, source, ['type'], ['if', 'allow-flat'])
+if 'allow-flat' in value:
+if discriminator is not None:
+raise QAPISemError(info, "'allow-flat' requires simple union")
+if not isinstance(value['allow-flat'], bool):
+raise QAPISemError(info, "'allow-flat' must be a boolean")
check_if(value, info, s
[PATCH v2 1/6] char/stdio: Fix QMP default for 'signal'
Commit 02c4bdf1 tried to make signal=on the default for stdio chardevs
except for '-serial mon:stdio', but it forgot about QMP and accidentally
switched the QMP default from true (except for -nographic) to false
(always). The QMP documentation was kept unchanged and still describes
the opposite of the old behaviour (which is an even older documentation
bug).
Fix all of this by making signal=true the default in ChardevStdio and
documenting it as such.
Fixes: 02c4bdf1d2ca8c02a9bae16398f260b5c08d08bf
Signed-off-by: Kevin Wolf
Reviewed-by: Markus Armbruster
Reviewed-by: Marc-André Lureau
---
qapi/char.json | 3 +--
chardev/char-stdio.c | 4 +---
2 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/qapi/char.json b/qapi/char.json
index b4d66ec90b..43486d1daa 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -321,8 +321,7 @@
# Configuration info for stdio chardevs.
#
# @signal: Allow signals (such as SIGINT triggered by ^C)
-# be delivered to qemu. Default: true in -nographic mode,
-# false otherwise.
+# be delivered to qemu. Default: true.
#
# Since: 1.5
##
diff --git a/chardev/char-stdio.c b/chardev/char-stdio.c
index 82eaebc1db..403da308c9 100644
--- a/chardev/char-stdio.c
+++ b/chardev/char-stdio.c
@@ -112,9 +112,7 @@ static void qemu_chr_open_stdio(Chardev *chr,
qemu_chr_open_fd(chr, 0, 1);
-if (opts->has_signal) {
-stdio_allow_signal = opts->signal;
-}
+stdio_allow_signal = !opts->has_signal || opts->signal;
qemu_chr_set_echo_stdio(chr, false);
}
#endif
--
2.28.0
[PATCH v2 3/6] qapi: Remove wrapper struct for simple unions
Variants of simple unions are always contained in a wrapper object
called 'data' that serves no real use. When mapping a QAPI object to the
command line, this becomes very visible to users because they have to
add one or more useless 'data.' to many option names.
As a first step towards avoiding this painful CLI experience, this gets
rid of the 'data' indirection internally: The QAPI parser doesn't use a
wrapper object as the variant type any more, so the indirection is
removed from the generated C types. As a result, the C type looks the
same for flat and simple unions now. A large part of this patch is
mechanical conversion of C code to remove the 'data' indirection.
Conceptually, the important change is that variants can now have flat
and wrapped representations. For a transitioning period, we'll allow
variants to support both representations in a later patch. Eventually,
the plan is to deprecate and remove wrapped representations entirely,
unifying simple and flat unions.
The externally visible interfaces stay unchanged: Visitors still expect
the 'data' wrappers, and introspection still shows it.
Signed-off-by: Kevin Wolf
---
docs/devel/qapi-code-gen.txt | 32
docs/sphinx/qapidoc.py | 14 +++---
backends/tpm/tpm_emulator.c| 3 +-
backends/tpm/tpm_passthrough.c | 4 +-
block/crypto.c | 3 +-
block/qcow2.c | 9 ++--
block/vmdk.c | 14 ++
blockdev.c | 36 ++---
chardev/char-file.c| 5 +-
chardev/char-mux.c | 5 +-
chardev/char-parallel.c| 5 +-
chardev/char-pipe.c| 7 ++-
chardev/char-ringbuf.c | 5 +-
chardev/char-serial.c | 5 +-
chardev/char-socket.c | 13 ++---
chardev/char-stdio.c | 5 +-
chardev/char-udp.c | 11 ++--
chardev/char.c | 10 ++--
chardev/msmouse.c | 4 +-
chardev/spice.c| 10 ++--
chardev/wctablet.c | 4 +-
hw/core/numa.c | 6 +--
hw/display/xenfb.c | 8 +--
hw/input/hid.c | 8 +--
hw/input/ps2.c | 6 +--
hw/input/virtio-input-hid.c| 8 +--
hw/mem/pc-dimm.c | 18 +++
hw/virtio/virtio-mem-pci.c | 3 +-
hw/virtio/virtio-pmem-pci.c| 3 +-
monitor/hmp-cmds.c | 14 +++---
qemu-keymap.c | 2 +-
replay/replay-input.c | 46 +++--
tests/test-char.c | 12 ++---
tests/test-clone-visitor.c | 14 +++---
tests/test-qmp-cmds.c | 2 +-
tests/test-qobject-input-visitor.c | 24 -
tests/test-qobject-output-visitor.c| 24 -
ui/console.c | 5 +-
ui/input-keymap.c | 12 ++---
ui/input-legacy.c | 12 ++---
ui/input.c | 43
ui/spice-app.c | 5 +-
util/qemu-sockets.c| 8 +--
scripts/qapi/introspect.py | 7 ++-
scripts/qapi/schema.py | 48 ++
scripts/qapi/visit.py | 28 +++
tests/qapi-schema/doc-good.out | 8 +--
tests/qapi-schema/qapi-schema-test.out | 70 +++---
48 files changed, 322 insertions(+), 326 deletions(-)
diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index c6438c6aa9..9722c1a204 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -310,7 +310,20 @@ The BRANCH's STRING name is the branch name.
The BRANCH's value defines the branch's properties, in particular its
type. The form TYPE-REF is shorthand for { 'type': TYPE-REF }.
-A simple union type defines a mapping from automatic discriminator
+Each branch can have flat or wrapped representation in the Client JSON
+Protocol.
+
+Wrapped representation of a branch means that it is represented by a
+'data' member that is of the specified data type corresponding to the
+discriminator value.
+
+Flat representation means that the union forms a single object that
+contains both common members (from the base type) and the selected
+branch's members, without any additional nesting. The two sets of
+member names must be disjoint.
+
+A simple union type automatically contains the 'type' member as a
+discriminator and defines a mapping from automatic discriminator
values to data types like in this example:
{ 'struct': 'BlockdevOptionsFile', 'data': { 'filename': 'str' } }
@@ -321,10 +334,8 @@ values to data types like in this example:
'data': { 'file': 'BlockdevOptionsFile',
'qcow2': 'Bl
[PATCH v2 6/6] qemu-storage-daemon: Use qmp_chardev_add() for --chardev
While this makes the code quite a bit longer and arguably isn't very
elegant, it removes the dependency on QemuOpts from the --chardev option
of the storage daemon.
Going through qmp_chardev_add() already now ensures that semantics and
accessible features won't change incompatibly once we QAPIfy it fully.
Note that there are a few differences between the command line option
-chardev in the system emulator and chardev-add in QMP. The --chardev
option of qemu-storage-daemon will follow QMP in these respects:
* All chardev types using ChardevHostdev accept a "path" option on the
command line, but QMP renamed it to "device"
* ChardevSocket:
- Intentionally different defaults (documented as such): server=false
and wait=true (if server=true) on the command line, but server=true
and wait=false in QMP.
- Accidentally different defaults: tight=true on the command line, but
tight=false in QMP (this is a bug in commit 776b97d3).
- QMP has a nested SocketAddressLegacy field, whereas the command line
accepts "path"/"host"/"port"/"fd"/etc. on the top level.
- The command line has a "delay" option, but QMP has "nodelay"
* ChardevUdp has two SocketAddressLegacy fields, the command line has
"host"/"port"/"localaddr"/"localport" on the top level with defaults
for values that are mandatory in SocketAddressLegacy
Signed-off-by: Kevin Wolf
---
storage-daemon/qemu-storage-daemon.c | 47 ++--
1 file changed, 38 insertions(+), 9 deletions(-)
diff --git a/storage-daemon/qemu-storage-daemon.c
b/storage-daemon/qemu-storage-daemon.c
index e419ba9f19..b543c30951 100644
--- a/storage-daemon/qemu-storage-daemon.c
+++ b/storage-daemon/qemu-storage-daemon.c
@@ -37,6 +37,7 @@
#include "qapi/error.h"
#include "qapi/qapi-visit-block-core.h"
#include "qapi/qapi-visit-block-export.h"
+#include "qapi/qapi-visit-char.h"
#include "qapi/qapi-visit-control.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qstring.h"
@@ -207,18 +208,46 @@ static void process_options(int argc, char *argv[])
}
case OPTION_CHARDEV:
{
-/* TODO This interface is not stable until we QAPIfy it */
-QemuOpts *opts = qemu_opts_parse_noisily(&qemu_chardev_opts,
- optarg, true);
-if (opts == NULL) {
-exit(EXIT_FAILURE);
-}
+QDict *args;
+Visitor *v;
+ChardevBackend *chr_options;
+char *id;
+bool help;
-if (!qemu_chr_new_from_opts(opts, NULL, &error_fatal)) {
-/* No error, but NULL returned means help was printed */
+args = keyval_parse(optarg, "type", &help, &error_fatal);
+if (help) {
+if (qdict_haskey(args, "type")) {
+/* TODO Print help based on the QAPI schema */
+qemu_opts_print_help(&qemu_chardev_opts, true);
+} else {
+qemu_chr_print_types();
+}
exit(EXIT_SUCCESS);
}
-qemu_opts_del(opts);
+
+/*
+ * TODO Flattened version of chardev-add in the QAPI schema
+ *
+ * While it's not there, parse 'id' manually from the QDict
+ * and treat everything else as the 'backend' option for
+ * chardev-add.
+ */
+id = g_strdup(qdict_get_try_str(args, "id"));
+if (!id) {
+error_report("Parameter 'id' is missing");
+exit(EXIT_FAILURE);
+}
+qdict_del(args, "id");
+
+v = qobject_input_visitor_new_keyval(QOBJECT(args));
+visit_set_flat_simple_unions(v, true);
+visit_type_ChardevBackend(v, NULL, &chr_options, &error_fatal);
+visit_free(v);
+
+qmp_chardev_add(id, chr_options, &error_fatal);
+qapi_free_ChardevBackend(chr_options);
+g_free(id);
+qobject_unref(args);
break;
}
case OPTION_EXPORT:
--
2.28.0
Re: [PATCH 0/6] qemu-storage-daemon: QAPIfy --chardev
On 23/10/20 15:40, Markus Armbruster wrote: >> >> The benefit of the user creatable object approach is that we dont >> have to add custom CLI args for different types of object, nor write >> code to populate QOM from QAPI. The downside is that we're divorced >> from the QAPI schema, so loose introspection, and have a different >> type of tedious boilerplate code to write. > Loss of QAPI introspection is the killer. > > We have QOM introspection, but it's far too weak to serve as > replacement. Besides, two introspection facilities is one too many. Wouldn't Eduardo+Kevin's work on object-add provide that too? > Nevertheless, we need Kevin's work now to get a decent storage daemon > CLI while that's still easy to do. We'll have to promise stability > soon, and then changes get much harder. I think we haven't answered the question of whether qsd needs a CLI at all. I looked recently at qemu_init and it struck me that, in principle, the only _really_ necessary command line options for QEMU are -sandbox, -name and possibly -trace (only to be able to trace the monitor). For everything else, one could use LISTEN_FDS socket activation mechanism, or if there's no LISTEN_FDS environment variable open a QMP socket on stdin/stdout. For qemu-standard-daemon, that would be _really_ true and not just in principle I understand that having a command-line can be useful to developers as it's less unwieldy than JSON, but why does it have to be stable? Could we default to only 2-3 command-line options in the same fashion, and only accept --blockdev and friends if the user starts the command line with "qemu-storage-daemon --i-am-not-a-script"? Paolo
Re: [PATCH v12 06/14] copy-on-read: pass bottom node name to COR driver
23.10.2020 18:31, Andrey Shinkevich wrote:
On 23.10.2020 17:45, Vladimir Sementsov-Ogievskiy wrote:
22.10.2020 21:13, Andrey Shinkevich wrote:
We are going to use the COR-filter for a block-stream job.
To limit COR operations by the base node in the backing chain during
stream job, pass the bottom node name, that is the first non-filter
overlay of the base, to the copy-on-read driver as the base node itself
may change due to possible concurrent jobs.
The rest of the functionality will be implemented in the patch that
follows.
Signed-off-by: Andrey Shinkevich
---
block/copy-on-read.c | 16
1 file changed, 16 insertions(+)
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index 618c4c4..3d8e4db 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -24,18 +24,24 @@
#include "block/block_int.h"
#include "qemu/module.h"
#include "qapi/error.h"
+#include "qapi/qmp/qerror.h"
+#include "qapi/qmp/qdict.h"
#include "block/copy-on-read.h"
typedef struct BDRVStateCOR {
bool active;
+ BlockDriverState *bottom_bs;
} BDRVStateCOR;
static int cor_open(BlockDriverState *bs, QDict *options, int flags,
Error **errp)
{
+ BlockDriverState *bottom_bs = NULL;
BDRVStateCOR *state = bs->opaque;
+ /* Find a bottom node name, if any */
+ const char *bottom_node = qdict_get_try_str(options, "bottom");
bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
@@ -51,7 +57,17 @@ static int cor_open(BlockDriverState *bs, QDict *options,
int flags,
((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
bs->file->bs->supported_zero_flags);
+ if (bottom_node) {
+ bottom_bs = bdrv_lookup_bs(NULL, bottom_node, errp);
+ if (!bottom_bs) {
+ error_setg(errp, QERR_BASE_NOT_FOUND, bottom_node);
QERR_BASE_NOT_FOUND is unrelated here. Also, I see a comment in qerror.h that
such macros should not be used in new code. And don't forget to drop qerror.h
include line.
I have been surprized because I don't have it in my branch and instead I do:
error_setg(errp, "Bottom node '%s' not found", bottom_node);
OK, with it:
Reviewed-by: Vladimir Sementsov-Ogievskiy
+ qdict_del(options, "bottom");
this may be moved above "bottom_bs = ..", to not call it after "if" in separate.
Please, see the "Re: [PATCH v11 04/13] copy-on-read: pass overlay base node name to
COR driver".
Hm, assume that it may free the string itself, OK then.
+ return -EINVAL;
+ }
+ qdict_del(options, "bottom");
+ }
state->active = true;
+ state->bottom_bs = bottom_bs;
/*
* We don't need to call bdrv_child_refresh_perms() now as the permissions
--
Best regards,
Vladimir
[PATCH v2 0/6] qemu-storage-daemon: QAPIfy --chardev
While the qemu-storage-daemon command line is still considered unstable, let's change --chardev from the old QemuOpts-based parser to QAPI, so that it becomes a simple mapping of chardev-add to the command line and will fit in a future fully QAPIfied command line without later incompatible changes or additional compatibility glue. v2: - Use g_autoptr in qemu_chr_print_types() [Marc-André] - Fixed build failures with spice [Marc-André] - Fixed documentation generation [Markus] - Make sure that no simple union variant contains a 'data' branch that would make conversion to flat representation much more complicated Kevin Wolf (6): char/stdio: Fix QMP default for 'signal' char: Factor out qemu_chr_print_types() qapi: Remove wrapper struct for simple unions qapi: Optionally parse simple unions as flat tests/qapi-schema: Flat representation of simple unions qemu-storage-daemon: Use qmp_chardev_add() for --chardev qapi/char.json| 6 +- docs/devel/qapi-code-gen.txt | 43 +++--- docs/sphinx/qapidoc.py| 14 ++-- include/chardev/char.h| 1 + include/qapi/visitor-impl.h | 3 + include/qapi/visitor.h| 10 +++ backends/tpm/tpm_emulator.c | 3 +- backends/tpm/tpm_passthrough.c| 4 +- block/crypto.c| 3 +- block/qcow2.c | 9 +- block/vmdk.c | 14 ++-- blockdev.c| 36 chardev/char-file.c | 5 +- chardev/char-mux.c| 5 +- chardev/char-parallel.c | 5 +- chardev/char-pipe.c | 7 +- chardev/char-ringbuf.c| 5 +- chardev/char-serial.c | 5 +- chardev/char-socket.c | 13 ++- chardev/char-stdio.c | 9 +- chardev/char-udp.c| 11 +-- chardev/char.c| 26 +++--- chardev/msmouse.c | 4 +- chardev/spice.c | 10 +-- chardev/wctablet.c| 4 +- hw/core/numa.c| 6 +- hw/display/xenfb.c| 8 +- hw/input/hid.c| 8 +- hw/input/ps2.c| 6 +- hw/input/virtio-input-hid.c | 8 +- hw/mem/pc-dimm.c | 18 ++-- hw/virtio/virtio-mem-pci.c| 3 +- hw/virtio/virtio-pmem-pci.c | 3 +- monitor/hmp-cmds.c| 14 ++-- qapi/qapi-visit-core.c| 10 +++ qemu-keymap.c | 2 +- replay/replay-input.c | 46 +- storage-daemon/qemu-storage-daemon.c | 47 +-- tests/test-char.c | 12 ++- tests/test-clone-visitor.c| 14 ++-- tests/test-qmp-cmds.c | 2 +- tests/test-qobject-input-visitor.c| 24 +++--- tests/test-qobject-output-visitor.c | 24 +++--- ui/console.c | 5 +- ui/input-keymap.c | 12 +-- ui/input-legacy.c | 12 +-- ui/input.c| 43 +- ui/spice-app.c| 5 +- util/qemu-sockets.c | 8 +- scripts/qapi/expr.py | 7 +- scripts/qapi/introspect.py| 7 +- scripts/qapi/schema.py| 74 +--- scripts/qapi/visit.py | 36 tests/qapi-schema/doc-good.out| 8 +- tests/qapi-schema/flat-union-allow-flat.err | 2 + tests/qapi-schema/flat-union-allow-flat.json | 10 +++ tests/qapi-schema/flat-union-allow-flat.out | 0 tests/qapi-schema/meson.build | 5 ++ tests/qapi-schema/qapi-schema-test.json | 10 +++ tests/qapi-schema/qapi-schema-test.out| 84 +++ tests/qapi-schema/union-allow-flat-bad.err| 2 + tests/qapi-schema/union-allow-flat-bad.json | 5 ++ tests/qapi-schema/union-allow-flat-bad.out| 0 .../union-allow-flat-builtin-type.err | 2 + .../union-allow-flat-builtin-type.json| 5 ++ .../union-allow-flat-builtin-type.out | 0 tests/qapi-schema/union-clash-member-data.err | 2 + .../qapi-schema/union-clash-member-data.json | 6 ++ tests/qapi-schema/union-clash-member-data.out | 0 tests/qapi-schema/union-clash-member-type.err | 2 + .../qapi-schema/union-clash-member-type.json | 6 ++ tests/qapi-schema/union-clash-member-type.out | 0 72 files change
Re: [PULL v3 00/28] Block patches
Patchew URL:
https://patchew.org/QEMU/20201023152147.1016281-1-stefa...@redhat.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Type: series
Message-id: 20201023152147.1016281-1-stefa...@redhat.com
Subject: [PULL v3 00/28] Block patches
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
* [new tag] patchew/20201023152147.1016281-1-stefa...@redhat.com ->
patchew/20201023152147.1016281-1-stefa...@redhat.com
Switched to a new branch 'test'
38310db iotests: add commit top->base cases to 274
f0671dd block/io: fix bdrv_is_allocated_above
d9c08b8 block/io: bdrv_common_block_status_above: support bs == base
c837bb3 block/io: bdrv_common_block_status_above: support include_base
6451e449 block/io: fix bdrv_co_block_status_above
b1b2eaa block/export: add vhost-user-blk multi-queue support
4cfdc0f block/export: add iothread and fixed-iothread options
21b0ebf block: move block exports to libblockdev
8e28b19 qemu-storage-daemon: avoid compiling blockdev_ss twice
d78e439 util/vhost-user-server: use static library in meson.build
5ed1b1e util/vhost-user-server: move header to include/
4fdb4dc block/export: convert vhost-user-blk server to block export API
1d7b5ba block/export: report flush errors
70c003e util/vhost-user-server: rework vu_client_trip() coroutine lifecycle
6b44703 util/vhost-user-server: check EOF when reading payload
2712e58 util/vhost-user-server: fix memory leak in vu_message_read()
0e740c0 util/vhost-user-server: drop unused DevicePanicNotifier
4f37c8e block/export: consolidate request structs into VuBlockReq
326d4b4 util/vhost-user-server: drop unnecessary watch deletion
f072aef util/vhost-user-server: drop unnecessary QOM cast
dfd5a40 util/vhost-user-server: s/fileds/fields/ typo fix
106a79f MAINTAINERS: Add vhost-user block device backend server maintainer
bcb59a3 block/export: vhost-user block device backend server
c5e218a block: move logical block size check function to a common utility
function
c4829fd util/vhost-user-server: generic vhost user server
b7acac2 libvhost-user: remove watch for kick_fd when de-initialize vu-dev
d236c9a libvhost-user: Allow vu_message_read to be replaced
098c6e0 block/nvme: Add driver statistics for access alignment and hw errors
=== OUTPUT BEGIN ===
1/28 Checking commit 098c6e074f94 (block/nvme: Add driver statistics for access
alignment and hw errors)
2/28 Checking commit d236c9abc147 (libvhost-user: Allow vu_message_read to be
replaced)
WARNING: Block comments use a leading /* on a separate line
#130: FILE: contrib/libvhost-user/libvhost-user.h:395:
+/* @read_msg: custom method to read vhost-user message
total: 0 errors, 1 warnings, 139 lines checked
Patch 2/28 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/28 Checking commit b7acac2640e0 (libvhost-user: remove watch for kick_fd when
de-initialize vu-dev)
4/28 Checking commit c4829fd6b453 (util/vhost-user-server: generic vhost user
server)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#35:
new file mode 100644
WARNING: line over 80 characters
#87: FILE: util/vhost-user-server.c:48:
+/* When this is set vu_client_trip will stop new processing vhost-user
message */
total: 0 errors, 2 warnings, 500 lines checked
Patch 4/28 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
5/28 Checking commit c5e218aa5fc3 (block: move logical block size check
function to a common utility function)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#85:
new file mode 100644
total: 0 errors, 1 warnings, 129 lines checked
Patch 5/28 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/28 Checking commit bcb59a3a37a8 (block/export: vhost-user block device
backend server)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#30:
new file mode 100644
WARNING: line over 80 characters
#476: FILE: block/export/vhost-user-blk-server.c:442:
+blk_remove_aio_context_notifier(vu_block_device->backend,
blk_aio_attached,
ERROR: g_free(NULL) is safe this check is probably not required
#536: FILE: block/export/vhost-user-blk-server.c:502:
+if (vus->node_name) {
+g_free(vus->node_name);
total: 1 errors, 2 warnings, 714 lines checked
Patch 6/28 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/28 Ch
Re: [PATCH v4 1/7] nbd: Utilize QAPI_CLONE for type conversion
On 10/9/20 4:55 PM, Eric Blake wrote:
> Rather than open-coding the translation from the deprecated
> NbdServerAddOptions type to the preferred BlockExportOptionsNbd, it's
> better to utilize QAPI_CLONE_MEMBERS. This solves a couple of issues:
> first, if we do any more refactoring of the base type (which an
> upcoming patch plans to do), we don't have to revisit the open-coding.
> Second, our assignment to arg->name is fishy: the generated QAPI code
> currently does not visit it if arg->has_name is false, but if it DID
> visit it, we would have introduced a double-free situation when arg is
> finally freed.
>
> Signed-off-by: Eric Blake
> ---
> blockdev-nbd.c | 15 ++-
> 1 file changed, 6 insertions(+), 9 deletions(-)
v5 will fix this nasty bug:
> @@ -195,7 +197,8 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error
> **errp)
> * the device name as a default here for compatibility.
> */
> if (!arg->has_name) {
> -arg->name = arg->device;
> +arg->has_name = true;
> +arg->name = g_steal_pointer(&arg->device);
> }
This causes assertion failures visible in at least iotest 149 and 192,
because arg->device was left NULL. Using g_strdup() instead fixes that.
>
> export_opts = g_new(BlockExportOptions, 1);
> @@ -205,15 +208,9 @@ void qmp_nbd_server_add(NbdServerAddOptions *arg, Error
> **errp)
> .node_name = g_strdup(bdrv_get_node_name(bs)),
> .has_writable = arg->has_writable,
> .writable = arg->writable,
> -.u.nbd = {
> -.has_name = true,
> -.name = g_strdup(arg->name),
> -.has_description= arg->has_description,
> -.description= g_strdup(arg->description),
> -.has_bitmap = arg->has_bitmap,
> -.bitmap = g_strdup(arg->bitmap),
> -},
> };
> +QAPI_CLONE_MEMBERS(BlockExportOptionsNbd, &export_opts->u.nbd,
> + qapi_NbdServerAddOptions_base(arg));
>
> /*
> * nbd-server-add doesn't complain when a read-only device should be
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
[PATCH v2 5/6] tests/qapi-schema: Flat representation of simple unions
This adds some test cases related to flat representation of simple
unions and the 'allow-flat' option for union branches.
Signed-off-by: Kevin Wolf
---
tests/qapi-schema/flat-union-allow-flat.err| 2 ++
tests/qapi-schema/flat-union-allow-flat.json | 10 ++
tests/qapi-schema/flat-union-allow-flat.out| 0
tests/qapi-schema/meson.build | 5 +
tests/qapi-schema/qapi-schema-test.json| 10 ++
tests/qapi-schema/qapi-schema-test.out | 14 ++
tests/qapi-schema/union-allow-flat-bad.err | 2 ++
tests/qapi-schema/union-allow-flat-bad.json| 5 +
tests/qapi-schema/union-allow-flat-bad.out | 0
.../qapi-schema/union-allow-flat-builtin-type.err | 2 ++
.../qapi-schema/union-allow-flat-builtin-type.json | 5 +
.../qapi-schema/union-allow-flat-builtin-type.out | 0
tests/qapi-schema/union-clash-member-data.err | 2 ++
tests/qapi-schema/union-clash-member-data.json | 6 ++
tests/qapi-schema/union-clash-member-data.out | 0
tests/qapi-schema/union-clash-member-type.err | 2 ++
tests/qapi-schema/union-clash-member-type.json | 6 ++
tests/qapi-schema/union-clash-member-type.out | 0
18 files changed, 71 insertions(+)
create mode 100644 tests/qapi-schema/flat-union-allow-flat.err
create mode 100644 tests/qapi-schema/flat-union-allow-flat.json
create mode 100644 tests/qapi-schema/flat-union-allow-flat.out
create mode 100644 tests/qapi-schema/union-allow-flat-bad.err
create mode 100644 tests/qapi-schema/union-allow-flat-bad.json
create mode 100644 tests/qapi-schema/union-allow-flat-bad.out
create mode 100644 tests/qapi-schema/union-allow-flat-builtin-type.err
create mode 100644 tests/qapi-schema/union-allow-flat-builtin-type.json
create mode 100644 tests/qapi-schema/union-allow-flat-builtin-type.out
create mode 100644 tests/qapi-schema/union-clash-member-data.err
create mode 100644 tests/qapi-schema/union-clash-member-data.json
create mode 100644 tests/qapi-schema/union-clash-member-data.out
create mode 100644 tests/qapi-schema/union-clash-member-type.err
create mode 100644 tests/qapi-schema/union-clash-member-type.json
create mode 100644 tests/qapi-schema/union-clash-member-type.out
diff --git a/tests/qapi-schema/flat-union-allow-flat.err
b/tests/qapi-schema/flat-union-allow-flat.err
new file mode 100644
index 00..f3f3d2384b
--- /dev/null
+++ b/tests/qapi-schema/flat-union-allow-flat.err
@@ -0,0 +1,2 @@
+flat-union-allow-flat.json: In union 'MyUnion':
+flat-union-allow-flat.json:6: 'allow-flat' requires simple union
diff --git a/tests/qapi-schema/flat-union-allow-flat.json
b/tests/qapi-schema/flat-union-allow-flat.json
new file mode 100644
index 00..114acaad86
--- /dev/null
+++ b/tests/qapi-schema/flat-union-allow-flat.json
@@ -0,0 +1,10 @@
+# Flat unions must not use allow-flat
+{ 'enum': 'Enum', 'data': [ 'one', 'two' ] }
+{ 'struct': 'Base',
+ 'data': { 'switch': 'Enum' } }
+{ 'struct': 'Branch', 'data': { 'name': 'str' } }
+{ 'union': 'MyUnion',
+ 'base': 'Base',
+ 'discriminator': 'switch',
+ 'data': { 'one': { 'type': 'Branch', 'allow-flat': true },
+'two': 'Branch' } }
diff --git a/tests/qapi-schema/flat-union-allow-flat.out
b/tests/qapi-schema/flat-union-allow-flat.out
new file mode 100644
index 00..e69de29bb2
diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index 304ef939bd..a5138d0c0f 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -103,6 +103,7 @@ schemas = [
'features-name-bad-type.json',
'features-no-list.json',
'features-unknown-key.json',
+ 'flat-union-allow-flat.json',
'flat-union-array-branch.json',
'flat-union-bad-base.json',
'flat-union-bad-discriminator.json',
@@ -184,12 +185,16 @@ schemas = [
'unclosed-list.json',
'unclosed-object.json',
'unclosed-string.json',
+ 'union-allow-flat-bad.json',
+ 'union-allow-flat-builtin-type.json',
'union-base-empty.json',
'union-base-no-discriminator.json',
'union-branch-case.json',
'union-branch-if-invalid.json',
'union-branch-invalid-dict.json',
'union-clash-branches.json',
+ 'union-clash-member-data.json',
+ 'union-clash-member-type.json',
'union-empty.json',
'union-invalid-base.json',
'union-optional-branch.json',
diff --git a/tests/qapi-schema/qapi-schema-test.json
b/tests/qapi-schema/qapi-schema-test.json
index 63f92adf68..1c29576ae9 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -110,6 +110,16 @@
{ 'struct': 'UserDefC',
'data': { 'string1': 'str', 'string2': 'str' } }
+{ 'struct': 'UserDefD',
+ 'data': { 'type': 'str' } }
+
+{ 'union': 'UserDefSimpleUnion',
+ 'data': { 'value1' : {'type': 'UserDefA'},
+'value2' : 'UserDefB',
+'value3' : 'UserDefB',
+'value4' : { 'type': 'UserDefD', 'allow-flat': false }
+ } }
+
[PATCH v2 2/6] char: Factor out qemu_chr_print_types()
We'll want to call the same from a non-QemuOpts code path.
Signed-off-by: Kevin Wolf
Reviewed-by: Marc-André Lureau
Reviewed-by: Markus Armbruster
---
include/chardev/char.h | 1 +
chardev/char.c | 16 ++--
2 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/include/chardev/char.h b/include/chardev/char.h
index db42f0a8c6..3b91645081 100644
--- a/include/chardev/char.h
+++ b/include/chardev/char.h
@@ -212,6 +212,7 @@ void qemu_chr_be_update_read_handlers(Chardev *s,
*/
void qemu_chr_be_event(Chardev *s, QEMUChrEvent event);
+void qemu_chr_print_types(void);
int qemu_chr_add_client(Chardev *s, int fd);
Chardev *qemu_chr_find(const char *name);
diff --git a/chardev/char.c b/chardev/char.c
index 78553125d3..4d188bfa6f 100644
--- a/chardev/char.c
+++ b/chardev/char.c
@@ -633,6 +633,15 @@ ChardevBackend *qemu_chr_parse_opts(QemuOpts *opts, Error
**errp)
return backend;
}
+void qemu_chr_print_types(void)
+{
+g_autoptr(GString) str = g_string_new("");
+
+chardev_name_foreach(help_string_append, str);
+
+qemu_printf("Available chardev backend types: %s\n", str->str);
+}
+
Chardev *qemu_chr_new_from_opts(QemuOpts *opts, GMainContext *context,
Error **errp)
{
@@ -644,12 +653,7 @@ Chardev *qemu_chr_new_from_opts(QemuOpts *opts,
GMainContext *context,
char *bid = NULL;
if (name && is_help_option(name)) {
-GString *str = g_string_new("");
-
-chardev_name_foreach(help_string_append, str);
-
-qemu_printf("Available chardev backend types: %s\n", str->str);
-g_string_free(str, true);
+qemu_chr_print_types();
return NULL;
}
--
2.28.0
Re: [PATCH v12 06/14] copy-on-read: pass bottom node name to COR driver
On 23.10.2020 17:45, Vladimir Sementsov-Ogievskiy wrote:
22.10.2020 21:13, Andrey Shinkevich wrote:
We are going to use the COR-filter for a block-stream job.
To limit COR operations by the base node in the backing chain during
stream job, pass the bottom node name, that is the first non-filter
overlay of the base, to the copy-on-read driver as the base node itself
may change due to possible concurrent jobs.
The rest of the functionality will be implemented in the patch that
follows.
Signed-off-by: Andrey Shinkevich
---
block/copy-on-read.c | 16
1 file changed, 16 insertions(+)
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index 618c4c4..3d8e4db 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -24,18 +24,24 @@
#include "block/block_int.h"
#include "qemu/module.h"
#include "qapi/error.h"
+#include "qapi/qmp/qerror.h"
+#include "qapi/qmp/qdict.h"
#include "block/copy-on-read.h"
typedef struct BDRVStateCOR {
bool active;
+ BlockDriverState *bottom_bs;
} BDRVStateCOR;
static int cor_open(BlockDriverState *bs, QDict *options, int flags,
Error **errp)
{
+ BlockDriverState *bottom_bs = NULL;
BDRVStateCOR *state = bs->opaque;
+ /* Find a bottom node name, if any */
+ const char *bottom_node = qdict_get_try_str(options, "bottom");
bs->file = bdrv_open_child(NULL, options, "file", bs,
&child_of_bds,
BDRV_CHILD_FILTERED |
BDRV_CHILD_PRIMARY,
@@ -51,7 +57,17 @@ static int cor_open(BlockDriverState *bs, QDict
*options, int flags,
((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
bs->file->bs->supported_zero_flags);
+ if (bottom_node) {
+ bottom_bs = bdrv_lookup_bs(NULL, bottom_node, errp);
+ if (!bottom_bs) {
+ error_setg(errp, QERR_BASE_NOT_FOUND, bottom_node);
QERR_BASE_NOT_FOUND is unrelated here. Also, I see a comment in qerror.h
that such macros should not be used in new code. And don't forget to
drop qerror.h include line.
I have been surprized because I don't have it in my branch and instead I do:
error_setg(errp, "Bottom node '%s' not found", bottom_node);
+ qdict_del(options, "bottom");
this may be moved above "bottom_bs = ..", to not call it after "if" in
separate.
Please, see the "Re: [PATCH v11 04/13] copy-on-read: pass overlay base
node name to COR driver".
+ return -EINVAL;
+ }
+ qdict_del(options, "bottom");
+ }
state->active = true;
+ state->bottom_bs = bottom_bs;
/*
* We don't need to call bdrv_child_refresh_perms() now as the
permissions
[PULL v3 22/28] block/export: add iothread and fixed-iothread options
Make it possible to specify the iothread where the export will run. By
default the block node can be moved to other AioContexts later and the
export will follow. The fixed-iothread option forces strict behavior
that prevents changing AioContext while the export is active. See the
QAPI docs for details.
Signed-off-by: Stefan Hajnoczi
Message-id: 20200929125516.186715-5-stefa...@redhat.com
[Fix stray '#' character in block-export.json and add missing "(since:
5.2)" as suggested by Eric Blake.
--Stefan]
Signed-off-by: Stefan Hajnoczi
---
qapi/block-export.json | 11 ++
block/export/export.c| 31 +++-
block/export/vhost-user-blk-server.c | 5 -
nbd/server.c | 2 --
4 files changed, 45 insertions(+), 4 deletions(-)
diff --git a/qapi/block-export.json b/qapi/block-export.json
index a793e34af9..8a4ced817f 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -219,11 +219,22 @@
#export before completion is signalled. (since: 5.2;
#default: false)
#
+# @iothread: The name of the iothread object where the export will run. The
+#default is to use the thread currently associated with the
+#block node. (since: 5.2)
+#
+# @fixed-iothread: True prevents the block node from being moved to another
+# thread while the export is active. If true and @iothread is
+# given, export creation fails if the block node cannot be
+# moved to the iothread. The default is false. (since: 5.2)
+#
# Since: 4.2
##
{ 'union': 'BlockExportOptions',
'base': { 'type': 'BlockExportType',
'id': 'str',
+ '*fixed-iothread': 'bool',
+ '*iothread': 'str',
'node-name': 'str',
'*writable': 'bool',
'*writethrough': 'bool' },
diff --git a/block/export/export.c b/block/export/export.c
index a24fadcfb5..c3478c6c97 100644
--- a/block/export/export.c
+++ b/block/export/export.c
@@ -15,6 +15,7 @@
#include "block/block.h"
#include "sysemu/block-backend.h"
+#include "sysemu/iothread.h"
#include "block/export.h"
#include "block/nbd.h"
#include "qapi/error.h"
@@ -63,10 +64,11 @@ static const BlockExportDriver
*blk_exp_find_driver(BlockExportType type)
BlockExport *blk_exp_add(BlockExportOptions *export, Error **errp)
{
+bool fixed_iothread = export->has_fixed_iothread && export->fixed_iothread;
const BlockExportDriver *drv;
BlockExport *exp = NULL;
BlockDriverState *bs;
-BlockBackend *blk;
+BlockBackend *blk = NULL;
AioContext *ctx;
uint64_t perm;
int ret;
@@ -102,6 +104,28 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Error
**errp)
ctx = bdrv_get_aio_context(bs);
aio_context_acquire(ctx);
+if (export->has_iothread) {
+IOThread *iothread;
+AioContext *new_ctx;
+
+iothread = iothread_by_id(export->iothread);
+if (!iothread) {
+error_setg(errp, "iothread \"%s\" not found", export->iothread);
+goto fail;
+}
+
+new_ctx = iothread_get_aio_context(iothread);
+
+ret = bdrv_try_set_aio_context(bs, new_ctx, errp);
+if (ret == 0) {
+aio_context_release(ctx);
+aio_context_acquire(new_ctx);
+ctx = new_ctx;
+} else if (fixed_iothread) {
+goto fail;
+}
+}
+
/*
* Block exports are used for non-shared storage migration. Make sure
* that BDRV_O_INACTIVE is cleared and the image is ready for write
@@ -116,6 +140,11 @@ BlockExport *blk_exp_add(BlockExportOptions *export, Error
**errp)
}
blk = blk_new(ctx, perm, BLK_PERM_ALL);
+
+if (!fixed_iothread) {
+blk_set_allow_aio_context_change(blk, true);
+}
+
ret = blk_insert_bs(blk, bs, errp);
if (ret < 0) {
goto fail;
diff --git a/block/export/vhost-user-blk-server.c
b/block/export/vhost-user-blk-server.c
index f7021cbd7b..286eb5fb9a 100644
--- a/block/export/vhost-user-blk-server.c
+++ b/block/export/vhost-user-blk-server.c
@@ -323,13 +323,17 @@ static const VuDevIface vu_blk_iface = {
static void blk_aio_attached(AioContext *ctx, void *opaque)
{
VuBlkExport *vexp = opaque;
+
+vexp->export.ctx = ctx;
vhost_user_server_attach_aio_context(&vexp->vu_server, ctx);
}
static void blk_aio_detach(void *opaque)
{
VuBlkExport *vexp = opaque;
+
vhost_user_server_detach_aio_context(&vexp->vu_server);
+vexp->export.ctx = NULL;
}
static void
@@ -384,7 +388,6 @@ static int vu_blk_exp_create(BlockExport *exp,
BlockExportOptions *opts,
vu_blk_initialize_config(blk_bs(exp->blk), &vexp->blkcfg,
logical_block_size);
-blk_set_allow_aio_context_change(exp->blk, true);
blk_add_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detach,
vexp);
[PULL v3 28/28] iotests: add commit top->base cases to 274
From: Vladimir Sementsov-Ogievskiy
These cases are fixed by previous patches around block_status and
is_allocated.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
Reviewed-by: Alberto Garcia
Message-id: 20200924194003.22080-6-vsement...@virtuozzo.com
Signed-off-by: Stefan Hajnoczi
---
tests/qemu-iotests/274 | 20 +++
tests/qemu-iotests/274.out | 68 ++
2 files changed, 88 insertions(+)
diff --git a/tests/qemu-iotests/274 b/tests/qemu-iotests/274
index d4571c5465..76b1ba6a52 100755
--- a/tests/qemu-iotests/274
+++ b/tests/qemu-iotests/274
@@ -115,6 +115,26 @@ with iotests.FilePath('base') as base, \
iotests.qemu_io_log('-c', 'read -P 1 0 %d' % size_short, mid)
iotests.qemu_io_log('-c', 'read -P 0 %d %d' % (size_short, size_diff), mid)
+iotests.log('=== Testing qemu-img commit (top -> base) ===')
+
+create_chain()
+iotests.qemu_img_log('commit', '-b', base, top)
+iotests.img_info_log(base)
+iotests.qemu_io_log('-c', 'read -P 1 0 %d' % size_short, base)
+iotests.qemu_io_log('-c', 'read -P 0 %d %d' % (size_short, size_diff),
base)
+
+iotests.log('=== Testing QMP active commit (top -> base) ===')
+
+create_chain()
+with create_vm() as vm:
+vm.launch()
+vm.qmp_log('block-commit', device='top', base_node='base',
+ job_id='job0', auto_dismiss=False)
+vm.run_job('job0', wait=5)
+
+iotests.img_info_log(mid)
+iotests.qemu_io_log('-c', 'read -P 1 0 %d' % size_short, base)
+iotests.qemu_io_log('-c', 'read -P 0 %d %d' % (size_short, size_diff),
base)
iotests.log('== Resize tests ==')
diff --git a/tests/qemu-iotests/274.out b/tests/qemu-iotests/274.out
index bf5abd4c10..cfe17a8659 100644
--- a/tests/qemu-iotests/274.out
+++ b/tests/qemu-iotests/274.out
@@ -135,6 +135,74 @@ read 1048576/1048576 bytes at offset 0
read 1048576/1048576 bytes at offset 1048576
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+=== Testing qemu-img commit (top -> base) ===
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off
compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
+
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off
compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base
backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off
compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid
backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+
+wrote 2097152/2097152 bytes at offset 0
+2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+Image committed.
+
+image: TEST_IMG
+file format: IMGFMT
+virtual size: 2 MiB (2097152 bytes)
+cluster_size: 65536
+Format specific information:
+compat: 1.1
+compression type: zlib
+lazy refcounts: false
+refcount bits: 16
+corrupt: false
+extended l2: false
+
+read 1048576/1048576 bytes at offset 0
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+read 1048576/1048576 bytes at offset 1048576
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+=== Testing QMP active commit (top -> base) ===
+Formatting 'TEST_DIR/PID-base', fmt=qcow2 cluster_size=65536 extended_l2=off
compression_type=zlib size=2097152 lazy_refcounts=off refcount_bits=16
+
+Formatting 'TEST_DIR/PID-mid', fmt=qcow2 cluster_size=65536 extended_l2=off
compression_type=zlib size=1048576 backing_file=TEST_DIR/PID-base
backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+
+Formatting 'TEST_DIR/PID-top', fmt=qcow2 cluster_size=65536 extended_l2=off
compression_type=zlib size=2097152 backing_file=TEST_DIR/PID-mid
backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
+
+wrote 2097152/2097152 bytes at offset 0
+2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+{"execute": "block-commit", "arguments": {"auto-dismiss": false, "base-node":
"base", "device": "top", "job-id": "job0"}}
+{"return": {}}
+{"execute": "job-complete", "arguments": {"id": "job0"}}
+{"return": {}}
+{"data": {"device": "job0", "len": 1048576, "offset": 1048576, "speed": 0,
"type": "commit"}, "event": "BLOCK_JOB_READY", "timestamp": {"microseconds":
"USECS", "seconds": "SECS"}}
+{"data": {"device": "job0", "len": 1048576, "offset": 1048576, "speed": 0,
"type": "commit"}, "event": "BLOCK_JOB_COMPLETED", "timestamp":
{"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "job0"}}
+{"return": {}}
+image: TEST_IMG
+file format: IMGFMT
+virtual size: 1 MiB (1048576 bytes)
+cluster_size: 65536
+backing file: TEST_DIR/PID-base
+backing file format: IMGFMT
+Format specific information:
+compat: 1.1
+compression type: zlib
+lazy refcounts: false
+refcount bits: 16
+corrupt: false
+extended l2: false
+
+read 1048576/1048576 bytes at offset 0
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+rea
[PULL v3 23/28] block/export: add vhost-user-blk multi-queue support
Allow the number of queues to be configured using --export
vhost-user-blk,num-queues=N. This setting should match the QEMU --device
vhost-user-blk-pci,num-queues=N setting but QEMU vhost-user-blk.c lowers
its own value if the vhost-user-blk backend offers fewer queues than
QEMU.
The vhost-user-blk-server.c code is already capable of multi-queue. All
virtqueue processing runs in the same AioContext. No new locking is
needed.
Add the num-queues=N option and set the VIRTIO_BLK_F_MQ feature bit.
Note that the feature bit only announces the presence of the num_queues
configuration space field. It does not promise that there is more than 1
virtqueue, so we can set it unconditionally.
I tested multi-queue by running a random read fio test with numjobs=4 on
an -smp 4 guest. After the benchmark finished the guest /proc/interrupts
file showed activity on all 4 virtio-blk MSI-X. The /sys/block/vda/mq/
directory shows that Linux blk-mq has 4 queues configured.
An automated test is included in the next commit.
Signed-off-by: Stefan Hajnoczi
Acked-by: Markus Armbruster
Message-id: 20201001144604.559733-2-stefa...@redhat.com
[Fixed accidental tab characters as suggested by Markus Armbruster
--Stefan]
Signed-off-by: Stefan Hajnoczi
---
qapi/block-export.json | 10 +++---
block/export/vhost-user-blk-server.c | 24 ++--
2 files changed, 25 insertions(+), 9 deletions(-)
diff --git a/qapi/block-export.json b/qapi/block-export.json
index 8a4ced817f..480c497690 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -93,11 +93,15 @@
#SocketAddress types are supported. Passed fds must be UNIX domain
#sockets.
# @logical-block-size: Logical block size in bytes. Defaults to 512 bytes.
+# @num-queues: Number of request virtqueues. Must be greater than 0. Defaults
+# to 1.
#
# Since: 5.2
##
{ 'struct': 'BlockExportOptionsVhostUserBlk',
- 'data': { 'addr': 'SocketAddress', '*logical-block-size': 'size' } }
+ 'data': { 'addr': 'SocketAddress',
+ '*logical-block-size': 'size',
+'*num-queues': 'uint16'} }
##
# @NbdServerAddOptions:
@@ -233,8 +237,8 @@
{ 'union': 'BlockExportOptions',
'base': { 'type': 'BlockExportType',
'id': 'str',
- '*fixed-iothread': 'bool',
- '*iothread': 'str',
+'*fixed-iothread': 'bool',
+'*iothread': 'str',
'node-name': 'str',
'*writable': 'bool',
'*writethrough': 'bool' },
diff --git a/block/export/vhost-user-blk-server.c
b/block/export/vhost-user-blk-server.c
index 286eb5fb9a..41f4933d6e 100644
--- a/block/export/vhost-user-blk-server.c
+++ b/block/export/vhost-user-blk-server.c
@@ -21,7 +21,7 @@
#include "util/block-helpers.h"
enum {
-VHOST_USER_BLK_MAX_QUEUES = 1,
+VHOST_USER_BLK_NUM_QUEUES_DEFAULT = 1,
};
struct virtio_blk_inhdr {
unsigned char status;
@@ -242,6 +242,7 @@ static uint64_t vu_blk_get_features(VuDev *dev)
1ull << VIRTIO_BLK_F_DISCARD |
1ull << VIRTIO_BLK_F_WRITE_ZEROES |
1ull << VIRTIO_BLK_F_CONFIG_WCE |
+ 1ull << VIRTIO_BLK_F_MQ |
1ull << VIRTIO_F_VERSION_1 |
1ull << VIRTIO_RING_F_INDIRECT_DESC |
1ull << VIRTIO_RING_F_EVENT_IDX |
@@ -338,7 +339,9 @@ static void blk_aio_detach(void *opaque)
static void
vu_blk_initialize_config(BlockDriverState *bs,
- struct virtio_blk_config *config, uint32_t blk_size)
+ struct virtio_blk_config *config,
+ uint32_t blk_size,
+ uint16_t num_queues)
{
config->capacity = bdrv_getlength(bs) >> BDRV_SECTOR_BITS;
config->blk_size = blk_size;
@@ -346,7 +349,7 @@ vu_blk_initialize_config(BlockDriverState *bs,
config->seg_max = 128 - 2;
config->min_io_size = 1;
config->opt_io_size = 1;
-config->num_queues = VHOST_USER_BLK_MAX_QUEUES;
+config->num_queues = num_queues;
config->max_discard_sectors = 32768;
config->max_discard_seg = 1;
config->discard_sector_alignment = config->blk_size >> 9;
@@ -368,6 +371,7 @@ static int vu_blk_exp_create(BlockExport *exp,
BlockExportOptions *opts,
BlockExportOptionsVhostUserBlk *vu_opts = &opts->u.vhost_user_blk;
Error *local_err = NULL;
uint64_t logical_block_size;
+uint16_t num_queues = VHOST_USER_BLK_NUM_QUEUES_DEFAULT;
vexp->writable = opts->writable;
vexp->blkcfg.wce = 0;
@@ -385,15 +389,23 @@ static int vu_blk_exp_create(BlockExport *exp,
BlockExportOptions *opts,
}
vexp->blk_size = logical_block_size;
blk_set_guest_block_size(exp->blk, logical_block_size);
+
+if (vu_opts->has_num_queues) {
+num_queues = vu_opts->num_queues;
+}
+if (num_queues == 0) {
+error_setg(errp, "num-queues must be greater than 0");
+return -EINVAL;
+}
+
vu_blk_initialize
[PULL v3 17/28] block/export: convert vhost-user-blk server to block export API
Use the new QAPI block exports API instead of defining our own QOM
objects.
This is a large change because the lifecycle of VuBlockDev needs to
follow BlockExportDriver. QOM properties are replaced by QAPI options
objects.
VuBlockDev is renamed VuBlkExport and contains a BlockExport field.
Several fields can be dropped since BlockExport already has equivalents.
The file names and meson build integration will be adjusted in a future
patch. libvhost-user should probably be built as a static library that
is linked into QEMU instead of as a .c file that results in duplicate
compilation.
The new command-line syntax is:
$ qemu-storage-daemon \
--blockdev file,node-name=drive0,filename=test.img \
--export
vhost-user-blk,node-name=drive0,id=export0,unix-socket=/tmp/vhost-user-blk.sock
Note that unix-socket is optional because we may wish to accept chardevs
too in the future.
Markus noted that supported address families are not explicit in the
QAPI schema. It is unlikely that support for more address families will
be added since file descriptor passing is required and few address
families support it. If a new address family needs to be added, then the
QAPI 'features' syntax can be used to advertize them.
Signed-off-by: Stefan Hajnoczi
Acked-by: Markus Armbruster
Message-id: 20200924151549.913737-12-stefa...@redhat.com
[Skip test on big-endian host architectures because this device doesn't
support them yet (as already mentioned in a code comment).
--Stefan]
Signed-off-by: Stefan Hajnoczi
---
qapi/block-export.json | 21 +-
block/export/vhost-user-blk-server.h | 23 +-
block/export/export.c| 6 +
block/export/vhost-user-blk-server.c | 452 +++
util/vhost-user-server.c | 10 +-
block/export/meson.build | 1 +
block/meson.build| 1 -
7 files changed, 156 insertions(+), 358 deletions(-)
diff --git a/qapi/block-export.json b/qapi/block-export.json
index 65804834d9..a793e34af9 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -84,6 +84,21 @@
'data': { '*name': 'str', '*description': 'str',
'*bitmap': 'str' } }
+##
+# @BlockExportOptionsVhostUserBlk:
+#
+# A vhost-user-blk block export.
+#
+# @addr: The vhost-user socket on which to listen. Both 'unix' and 'fd'
+#SocketAddress types are supported. Passed fds must be UNIX domain
+#sockets.
+# @logical-block-size: Logical block size in bytes. Defaults to 512 bytes.
+#
+# Since: 5.2
+##
+{ 'struct': 'BlockExportOptionsVhostUserBlk',
+ 'data': { 'addr': 'SocketAddress', '*logical-block-size': 'size' } }
+
##
# @NbdServerAddOptions:
#
@@ -180,11 +195,12 @@
# An enumeration of block export types
#
# @nbd: NBD export
+# @vhost-user-blk: vhost-user-blk export (since 5.2)
#
# Since: 4.2
##
{ 'enum': 'BlockExportType',
- 'data': [ 'nbd' ] }
+ 'data': [ 'nbd', 'vhost-user-blk' ] }
##
# @BlockExportOptions:
@@ -213,7 +229,8 @@
'*writethrough': 'bool' },
'discriminator': 'type',
'data': {
- 'nbd': 'BlockExportOptionsNbd'
+ 'nbd': 'BlockExportOptionsNbd',
+ 'vhost-user-blk': 'BlockExportOptionsVhostUserBlk'
} }
##
diff --git a/block/export/vhost-user-blk-server.h
b/block/export/vhost-user-blk-server.h
index f06f37c4c8..fcf46fc8a5 100644
--- a/block/export/vhost-user-blk-server.h
+++ b/block/export/vhost-user-blk-server.h
@@ -10,27 +10,10 @@
#ifndef VHOST_USER_BLK_SERVER_H
#define VHOST_USER_BLK_SERVER_H
-#include "util/vhost-user-server.h"
-typedef struct VuBlockDev VuBlockDev;
-#define TYPE_VHOST_USER_BLK_SERVER "vhost-user-blk-server"
-#define VHOST_USER_BLK_SERVER(obj) \
- OBJECT_CHECK(VuBlockDev, obj, TYPE_VHOST_USER_BLK_SERVER)
+#include "block/export.h"
-/* vhost user block device */
-struct VuBlockDev {
-Object parent_obj;
-char *node_name;
-SocketAddress *addr;
-AioContext *ctx;
-VuServer vu_server;
-bool running;
-uint32_t blk_size;
-BlockBackend *backend;
-QIOChannelSocket *sioc;
-QTAILQ_ENTRY(VuBlockDev) next;
-struct virtio_blk_config blkcfg;
-bool writable;
-};
+/* For block/export/export.c */
+extern const BlockExportDriver blk_exp_vhost_user_blk;
#endif /* VHOST_USER_BLK_SERVER_H */
diff --git a/block/export/export.c b/block/export/export.c
index f2c00d13bf..bd7cac241f 100644
--- a/block/export/export.c
+++ b/block/export/export.c
@@ -17,6 +17,9 @@
#include "sysemu/block-backend.h"
#include "block/export.h"
#include "block/nbd.h"
+#if CONFIG_LINUX
+#include "block/export/vhost-user-blk-server.h"
+#endif
#include "qapi/error.h"
#include "qapi/qapi-commands-block-export.h"
#include "qapi/qapi-events-block-export.h"
@@ -24,6 +27,9 @@
static const BlockExportDriver *blk_exp_drivers[] = {
&blk_exp_nbd,
+#if CONFIG_LINUX
+&blk_exp_vhost_user_blk,
+#endif
};
/* Only accessed from the main thread */
diff --git a/block/export/vhost-user-blk-server.c
b/block
[PULL v3 27/28] block/io: fix bdrv_is_allocated_above
From: Vladimir Sementsov-Ogievskiy
bdrv_is_allocated_above wrongly handles short backing files: it reports
after-EOF space as UNALLOCATED which is wrong, as on read the data is
generated on the level of short backing file (if all overlays have
unallocated areas at that place).
Reusing bdrv_common_block_status_above fixes the issue and unifies code
path.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
Reviewed-by: Alberto Garcia
Message-id: 20200924194003.22080-5-vsement...@virtuozzo.com
[Fix s/has/have/ as suggested by Eric Blake. Fix s/area/areas/.
--Stefan]
Signed-off-by: Stefan Hajnoczi
---
block/io.c | 43 +--
1 file changed, 5 insertions(+), 38 deletions(-)
diff --git a/block/io.c b/block/io.c
index b616bc4ada..02528b3823 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2477,52 +2477,19 @@ int coroutine_fn bdrv_is_allocated(BlockDriverState
*bs, int64_t offset,
* at 'offset + *pnum' may return the same allocation status (in other
* words, the result is not necessarily the maximum possible range);
* but 'pnum' will only be 0 when end of file is reached.
- *
*/
int bdrv_is_allocated_above(BlockDriverState *top,
BlockDriverState *base,
bool include_base, int64_t offset,
int64_t bytes, int64_t *pnum)
{
-BlockDriverState *intermediate;
-int ret;
-int64_t n = bytes;
-
-assert(base || !include_base);
-
-intermediate = top;
-while (include_base || intermediate != base) {
-int64_t pnum_inter;
-int64_t size_inter;
-
-assert(intermediate);
-ret = bdrv_is_allocated(intermediate, offset, bytes, &pnum_inter);
-if (ret < 0) {
-return ret;
-}
-if (ret) {
-*pnum = pnum_inter;
-return 1;
-}
-
-size_inter = bdrv_getlength(intermediate);
-if (size_inter < 0) {
-return size_inter;
-}
-if (n > pnum_inter &&
-(intermediate == top || offset + pnum_inter < size_inter)) {
-n = pnum_inter;
-}
-
-if (intermediate == base) {
-break;
-}
-
-intermediate = bdrv_filter_or_cow_bs(intermediate);
+int ret = bdrv_common_block_status_above(top, base, include_base, false,
+ offset, bytes, pnum, NULL, NULL);
+if (ret < 0) {
+return ret;
}
-*pnum = n;
-return 0;
+return !!(ret & BDRV_BLOCK_ALLOCATED);
}
int coroutine_fn
--
2.26.2
[PULL v3 20/28] qemu-storage-daemon: avoid compiling blockdev_ss twice
Introduce libblkdev.fa to avoid recompiling blockdev_ss twice.
Suggested-by: Paolo Bonzini
Reviewed-by: Paolo Bonzini
Signed-off-by: Stefan Hajnoczi
Message-id: 20200929125516.186715-3-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
meson.build| 12 ++--
storage-daemon/meson.build | 3 +--
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/meson.build b/meson.build
index 2ec4f114ce..880683407f 100644
--- a/meson.build
+++ b/meson.build
@@ -1464,7 +1464,6 @@ blockdev_ss.add(files(
# os-win32.c does not
blockdev_ss.add(when: 'CONFIG_POSIX', if_true: files('os-posix.c'))
softmmu_ss.add(when: 'CONFIG_WIN32', if_true: [files('os-win32.c')])
-softmmu_ss.add_all(blockdev_ss)
common_ss.add(files('cpus-common.c'))
@@ -1596,6 +1595,15 @@ block = declare_dependency(link_whole: [libblock],
link_args: '@block.syms',
dependencies: [crypto, io])
+blockdev_ss = blockdev_ss.apply(config_host, strict: false)
+libblockdev = static_library('blockdev', blockdev_ss.sources() + genh,
+ dependencies: blockdev_ss.dependencies(),
+ name_suffix: 'fa',
+ build_by_default: false)
+
+blockdev = declare_dependency(link_whole: [libblockdev],
+ dependencies: [block])
+
qmp_ss = qmp_ss.apply(config_host, strict: false)
libqmp = static_library('qmp', qmp_ss.sources() + genh,
dependencies: qmp_ss.dependencies(),
@@ -1628,7 +1636,7 @@ foreach m : block_mods + softmmu_mods
install_dir: config_host['qemu_moddir'])
endforeach
-softmmu_ss.add(authz, block, chardev, crypto, io, qmp)
+softmmu_ss.add(authz, blockdev, chardev, crypto, io, qmp)
common_ss.add(qom, qemuutil)
common_ss.add_all(when: 'CONFIG_SOFTMMU', if_true: [softmmu_ss])
diff --git a/storage-daemon/meson.build b/storage-daemon/meson.build
index 0409acc3f5..c5adce81c3 100644
--- a/storage-daemon/meson.build
+++ b/storage-daemon/meson.build
@@ -1,7 +1,6 @@
qsd_ss = ss.source_set()
qsd_ss.add(files('qemu-storage-daemon.c'))
-qsd_ss.add(block, chardev, qmp, qom, qemuutil)
-qsd_ss.add_all(blockdev_ss)
+qsd_ss.add(blockdev, chardev, qmp, qom, qemuutil)
subdir('qapi')
--
2.26.2
[PULL v3 19/28] util/vhost-user-server: use static library in meson.build
Don't compile contrib/libvhost-user/libvhost-user.c again. Instead build
the static library once and then reuse it throughout QEMU.
Also switch from CONFIG_LINUX to CONFIG_VHOST_USER, which is what the
vhost-user tools (vhost-user-gpu, etc) do.
Signed-off-by: Stefan Hajnoczi
Message-id: 20200924151549.913737-14-stefa...@redhat.com
[Added CONFIG_LINUX again because libvhost-user doesn't build on macOS.
--Stefan]
Signed-off-by: Stefan Hajnoczi
---
block/export/export.c | 8
block/export/meson.build | 2 +-
contrib/libvhost-user/meson.build | 1 +
meson.build | 6 +-
util/meson.build | 4 +++-
5 files changed, 14 insertions(+), 7 deletions(-)
diff --git a/block/export/export.c b/block/export/export.c
index bd7cac241f..a24fadcfb5 100644
--- a/block/export/export.c
+++ b/block/export/export.c
@@ -17,17 +17,17 @@
#include "sysemu/block-backend.h"
#include "block/export.h"
#include "block/nbd.h"
-#if CONFIG_LINUX
-#include "block/export/vhost-user-blk-server.h"
-#endif
#include "qapi/error.h"
#include "qapi/qapi-commands-block-export.h"
#include "qapi/qapi-events-block-export.h"
#include "qemu/id.h"
+#if defined(CONFIG_LINUX) && defined(CONFIG_VHOST_USER)
+#include "vhost-user-blk-server.h"
+#endif
static const BlockExportDriver *blk_exp_drivers[] = {
&blk_exp_nbd,
-#if CONFIG_LINUX
+#if defined(CONFIG_LINUX) && defined(CONFIG_VHOST_USER)
&blk_exp_vhost_user_blk,
#endif
};
diff --git a/block/export/meson.build b/block/export/meson.build
index ef3a9576f7..fffe6b09e8 100644
--- a/block/export/meson.build
+++ b/block/export/meson.build
@@ -1,2 +1,2 @@
block_ss.add(files('export.c'))
-block_ss.add(when: 'CONFIG_LINUX', if_true: files('vhost-user-blk-server.c',
'../../contrib/libvhost-user/libvhost-user.c'))
+block_ss.add(when: ['CONFIG_LINUX', 'CONFIG_VHOST_USER'], if_true:
files('vhost-user-blk-server.c'))
diff --git a/contrib/libvhost-user/meson.build
b/contrib/libvhost-user/meson.build
index e68dd1a581..a261e7665f 100644
--- a/contrib/libvhost-user/meson.build
+++ b/contrib/libvhost-user/meson.build
@@ -1,3 +1,4 @@
libvhost_user = static_library('vhost-user',
files('libvhost-user.c',
'libvhost-user-glib.c'),
build_by_default: false)
+vhost_user = declare_dependency(link_with: libvhost_user)
diff --git a/meson.build b/meson.build
index 7627a0ae46..2ec4f114ce 100644
--- a/meson.build
+++ b/meson.build
@@ -1398,6 +1398,11 @@ trace_events_subdirs += [
'util',
]
+vhost_user = not_found
+if 'CONFIG_VHOST_USER' in config_host
+ subdir('contrib/libvhost-user')
+endif
+
subdir('qapi')
subdir('qobject')
subdir('stubs')
@@ -1830,7 +1835,6 @@ if have_tools
install: true)
if 'CONFIG_VHOST_USER' in config_host
-subdir('contrib/libvhost-user')
subdir('contrib/vhost-user-blk')
subdir('contrib/vhost-user-gpu')
subdir('contrib/vhost-user-input')
diff --git a/util/meson.build b/util/meson.build
index 2296e81b34..c5159ad79d 100644
--- a/util/meson.build
+++ b/util/meson.build
@@ -66,7 +66,9 @@ if have_block
util_ss.add(files('main-loop.c'))
util_ss.add(files('nvdimm-utils.c'))
util_ss.add(files('qemu-coroutine.c', 'qemu-coroutine-lock.c',
'qemu-coroutine-io.c'))
- util_ss.add(when: 'CONFIG_LINUX', if_true: files('vhost-user-server.c'))
+ util_ss.add(when: ['CONFIG_LINUX', 'CONFIG_VHOST_USER'], if_true: [
+files('vhost-user-server.c'), vhost_user
+ ])
util_ss.add(files('block-helpers.c'))
util_ss.add(files('qemu-coroutine-sleep.c'))
util_ss.add(files('qemu-co-shared-resource.c'))
--
2.26.2
[PULL v3 25/28] block/io: bdrv_common_block_status_above: support include_base
From: Vladimir Sementsov-Ogievskiy
In order to reuse bdrv_common_block_status_above in
bdrv_is_allocated_above, let's support include_base parameter.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Alberto Garcia
Reviewed-by: Eric Blake
Message-id: 20200924194003.22080-3-vsement...@virtuozzo.com
Signed-off-by: Stefan Hajnoczi
---
block/coroutines.h | 2 ++
block/io.c | 21 ++---
2 files changed, 16 insertions(+), 7 deletions(-)
diff --git a/block/coroutines.h b/block/coroutines.h
index f69179f5ef..1cb3128b94 100644
--- a/block/coroutines.h
+++ b/block/coroutines.h
@@ -41,6 +41,7 @@ bdrv_pwritev(BdrvChild *child, int64_t offset, unsigned int
bytes,
int coroutine_fn
bdrv_co_common_block_status_above(BlockDriverState *bs,
BlockDriverState *base,
+ bool include_base,
bool want_zero,
int64_t offset,
int64_t bytes,
@@ -50,6 +51,7 @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
int generated_co_wrapper
bdrv_common_block_status_above(BlockDriverState *bs,
BlockDriverState *base,
+ bool include_base,
bool want_zero,
int64_t offset,
int64_t bytes,
diff --git a/block/io.c b/block/io.c
index a718d50ca2..86f76d04bf 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2343,6 +2343,7 @@ early_out:
int coroutine_fn
bdrv_co_common_block_status_above(BlockDriverState *bs,
BlockDriverState *base,
+ bool include_base,
bool want_zero,
int64_t offset,
int64_t bytes,
@@ -2354,10 +2355,11 @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
BlockDriverState *p;
int64_t eof = 0;
-assert(bs != base);
+assert(include_base || bs != base);
+assert(!include_base || base); /* Can't include NULL base */
ret = bdrv_co_block_status(bs, want_zero, offset, bytes, pnum, map, file);
-if (ret < 0 || *pnum == 0 || ret & BDRV_BLOCK_ALLOCATED) {
+if (ret < 0 || *pnum == 0 || ret & BDRV_BLOCK_ALLOCATED || bs == base) {
return ret;
}
@@ -2368,7 +2370,7 @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
assert(*pnum <= bytes);
bytes = *pnum;
-for (p = bdrv_filter_or_cow_bs(bs); p != base;
+for (p = bdrv_filter_or_cow_bs(bs); include_base || p != base;
p = bdrv_filter_or_cow_bs(p))
{
ret = bdrv_co_block_status(p, want_zero, offset, bytes, pnum, map,
@@ -2406,6 +2408,11 @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
break;
}
+if (p == base) {
+assert(include_base);
+break;
+}
+
/*
* OK, [offset, offset + *pnum) region is unallocated on this layer,
* let's continue the diving.
@@ -2425,7 +2432,7 @@ int bdrv_block_status_above(BlockDriverState *bs,
BlockDriverState *base,
int64_t offset, int64_t bytes, int64_t *pnum,
int64_t *map, BlockDriverState **file)
{
-return bdrv_common_block_status_above(bs, base, true, offset, bytes,
+return bdrv_common_block_status_above(bs, base, false, true, offset, bytes,
pnum, map, file);
}
@@ -2442,9 +2449,9 @@ int coroutine_fn bdrv_is_allocated(BlockDriverState *bs,
int64_t offset,
int ret;
int64_t dummy;
-ret = bdrv_common_block_status_above(bs, bdrv_filter_or_cow_bs(bs), false,
- offset, bytes, pnum ? pnum : &dummy,
- NULL, NULL);
+ret = bdrv_common_block_status_above(bs, bs, true, false, offset,
+ bytes, pnum ? pnum : &dummy, NULL,
+ NULL);
if (ret < 0) {
return ret;
}
--
2.26.2
[PULL v3 21/28] block: move block exports to libblockdev
Block exports are used by softmmu, qemu-storage-daemon, and qemu-nbd.
They are not used by other programs and are not otherwise needed in
libblock.
Undo the recent move of blockdev-nbd.c from blockdev_ss into block_ss.
Since bdrv_close_all() (libblock) calls blk_exp_close_all()
(libblockdev) a stub function is required..
Make qemu-nbd.c use signal handling utility functions instead of
duplicating the code. This helps because os-posix.c is in libblockdev
and it depends on a qemu_system_killed() symbol that qemu-nbd.c lacks.
Once we use the signal handling utility functions we also end up
providing the necessary symbol.
Signed-off-by: Stefan Hajnoczi
Reviewed-by: Paolo Bonzini
Reviewed-by: Eric Blake
Message-id: 20200929125516.186715-4-stefa...@redhat.com
[Fixed s/ndb/nbd/ typo in commit description as suggested by Eric Blake
--Stefan]
Signed-off-by: Stefan Hajnoczi
---
qemu-nbd.c| 21 -
stubs/blk-exp-close-all.c | 7 +++
block/export/meson.build | 4 ++--
meson.build | 4 ++--
nbd/meson.build | 2 ++
stubs/meson.build | 1 +
6 files changed, 22 insertions(+), 17 deletions(-)
create mode 100644 stubs/blk-exp-close-all.c
diff --git a/qemu-nbd.c b/qemu-nbd.c
index bc644a0670..a0701cdf36 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -25,6 +25,7 @@
#include "qapi/error.h"
#include "qemu/cutils.h"
#include "sysemu/block-backend.h"
+#include "sysemu/runstate.h" /* for qemu_system_killed() prototype */
#include "block/block_int.h"
#include "block/nbd.h"
#include "qemu/main-loop.h"
@@ -155,7 +156,11 @@ QEMU_COPYRIGHT "\n"
}
#ifdef CONFIG_POSIX
-static void termsig_handler(int signum)
+/*
+ * The client thread uses SIGTERM to interrupt the server. A signal
+ * handler ensures that "qemu-nbd -v -c" exits with a nice status code.
+ */
+void qemu_system_killed(int signum, pid_t pid)
{
qatomic_cmpxchg(&state, RUNNING, TERMINATE);
qemu_notify_event();
@@ -582,18 +587,8 @@ int main(int argc, char **argv)
BlockExportOptions *export_opts;
#ifdef CONFIG_POSIX
-/*
- * Exit gracefully on various signals, which includes SIGTERM used
- * by 'qemu-nbd -v -c'.
- */
-struct sigaction sa_sigterm;
-memset(&sa_sigterm, 0, sizeof(sa_sigterm));
-sa_sigterm.sa_handler = termsig_handler;
-sigaction(SIGTERM, &sa_sigterm, NULL);
-sigaction(SIGINT, &sa_sigterm, NULL);
-sigaction(SIGHUP, &sa_sigterm, NULL);
-
-signal(SIGPIPE, SIG_IGN);
+os_setup_early_signal_handling();
+os_setup_signal_handling();
#endif
socket_init();
diff --git a/stubs/blk-exp-close-all.c b/stubs/blk-exp-close-all.c
new file mode 100644
index 00..1c71316763
--- /dev/null
+++ b/stubs/blk-exp-close-all.c
@@ -0,0 +1,7 @@
+#include "qemu/osdep.h"
+#include "block/export.h"
+
+/* Only used in programs that support block exports (libblockdev.fa) */
+void blk_exp_close_all(void)
+{
+}
diff --git a/block/export/meson.build b/block/export/meson.build
index fffe6b09e8..9fb4fbf81d 100644
--- a/block/export/meson.build
+++ b/block/export/meson.build
@@ -1,2 +1,2 @@
-block_ss.add(files('export.c'))
-block_ss.add(when: ['CONFIG_LINUX', 'CONFIG_VHOST_USER'], if_true:
files('vhost-user-blk-server.c'))
+blockdev_ss.add(files('export.c'))
+blockdev_ss.add(when: ['CONFIG_LINUX', 'CONFIG_VHOST_USER'], if_true:
files('vhost-user-blk-server.c'))
diff --git a/meson.build b/meson.build
index 880683407f..b349c9bda8 100644
--- a/meson.build
+++ b/meson.build
@@ -1443,7 +1443,6 @@ subdir('dump')
block_ss.add(files(
'block.c',
- 'blockdev-nbd.c',
'blockjob.c',
'job.c',
'qemu-io-cmds.c',
@@ -1456,6 +1455,7 @@ subdir('block')
blockdev_ss.add(files(
'blockdev.c',
+ 'blockdev-nbd.c',
'iothread.c',
'job-qmp.c',
))
@@ -1832,7 +1832,7 @@ if have_tools
qemu_io = executable('qemu-io', files('qemu-io.c'),
dependencies: [block, qemuutil], install: true)
qemu_nbd = executable('qemu-nbd', files('qemu-nbd.c'),
- dependencies: [block, qemuutil], install: true)
+ dependencies: [blockdev, qemuutil], install: true)
subdir('storage-daemon')
subdir('contrib/rdmacm-mux')
diff --git a/nbd/meson.build b/nbd/meson.build
index 0c00a776d3..2baaa36948 100644
--- a/nbd/meson.build
+++ b/nbd/meson.build
@@ -1,5 +1,7 @@
block_ss.add(files(
'client.c',
'common.c',
+))
+blockdev_ss.add(files(
'server.c',
))
diff --git a/stubs/meson.build b/stubs/meson.build
index 67f2a8c069..7b733fadb7 100644
--- a/stubs/meson.build
+++ b/stubs/meson.build
@@ -1,6 +1,7 @@
stub_ss.add(files('arch_type.c'))
stub_ss.add(files('bdrv-next-monitor-owned.c'))
stub_ss.add(files('blk-commit-all.c'))
+stub_ss.add(files('blk-exp-close-all.c'))
stub_ss.add(files('blockdev-close-all-bdrv-states.c'))
stub_ss.add(files('change-state-handler.c'))
stub_ss.add(files('cmos.c'))
--
2.26.2
[PULL v3 05/28] block: move logical block size check function to a common utility function
From: Coiby Xu
Move the constants from hw/core/qdev-properties.c to
util/block-helpers.h so that knowledge of the min/max values is
Signed-off-by: Stefan Hajnoczi
Signed-off-by: Coiby Xu
Reviewed-by: Stefan Hajnoczi
Reviewed-by: Marc-André Lureau
Acked-by: Eduardo Habkost
Message-id: 20200918080912.321299-5-coiby...@gmail.com
Signed-off-by: Stefan Hajnoczi
---
util/block-helpers.h | 19 +
hw/core/qdev-properties-system.c | 31 -
util/block-helpers.c | 46
util/meson.build | 1 +
4 files changed, 71 insertions(+), 26 deletions(-)
create mode 100644 util/block-helpers.h
create mode 100644 util/block-helpers.c
diff --git a/util/block-helpers.h b/util/block-helpers.h
new file mode 100644
index 00..b53295a529
--- /dev/null
+++ b/util/block-helpers.h
@@ -0,0 +1,19 @@
+#ifndef BLOCK_HELPERS_H
+#define BLOCK_HELPERS_H
+
+#include "qemu/units.h"
+
+/* lower limit is sector size */
+#define MIN_BLOCK_SIZE INT64_C(512)
+#define MIN_BLOCK_SIZE_STR "512 B"
+/*
+ * upper limit is arbitrary, 2 MiB looks sufficient for all sensible uses, and
+ * matches qcow2 cluster size limit
+ */
+#define MAX_BLOCK_SIZE (2 * MiB)
+#define MAX_BLOCK_SIZE_STR "2 MiB"
+
+void check_block_size(const char *id, const char *name, int64_t value,
+ Error **errp);
+
+#endif /* BLOCK_HELPERS_H */
diff --git a/hw/core/qdev-properties-system.c b/hw/core/qdev-properties-system.c
index 49bdd12581..b81a4e8d14 100644
--- a/hw/core/qdev-properties-system.c
+++ b/hw/core/qdev-properties-system.c
@@ -30,6 +30,7 @@
#include "sysemu/blockdev.h"
#include "net/net.h"
#include "hw/pci/pci.h"
+#include "util/block-helpers.h"
static bool check_prop_still_unset(DeviceState *dev, const char *name,
const void *old_val, const char *new_val,
@@ -576,16 +577,6 @@ const PropertyInfo qdev_prop_losttickpolicy = {
/* --- blocksize --- */
-/* lower limit is sector size */
-#define MIN_BLOCK_SIZE 512
-#define MIN_BLOCK_SIZE_STR "512 B"
-/*
- * upper limit is arbitrary, 2 MiB looks sufficient for all sensible uses, and
- * matches qcow2 cluster size limit
- */
-#define MAX_BLOCK_SIZE (2 * MiB)
-#define MAX_BLOCK_SIZE_STR "2 MiB"
-
static void set_blocksize(Object *obj, Visitor *v, const char *name,
void *opaque, Error **errp)
{
@@ -593,6 +584,7 @@ static void set_blocksize(Object *obj, Visitor *v, const
char *name,
Property *prop = opaque;
uint32_t *ptr = qdev_get_prop_ptr(dev, prop);
uint64_t value;
+Error *local_err = NULL;
if (dev->realized) {
qdev_prop_set_after_realize(dev, name, errp);
@@ -602,24 +594,11 @@ static void set_blocksize(Object *obj, Visitor *v, const
char *name,
if (!visit_type_size(v, name, &value, errp)) {
return;
}
-/* value of 0 means "unset" */
-if (value && (value < MIN_BLOCK_SIZE || value > MAX_BLOCK_SIZE)) {
-error_setg(errp,
- "Property %s.%s doesn't take value %" PRIu64
- " (minimum: " MIN_BLOCK_SIZE_STR
- ", maximum: " MAX_BLOCK_SIZE_STR ")",
- dev->id ? : "", name, value);
+check_block_size(dev->id ? : "", name, value, &local_err);
+if (local_err) {
+error_propagate(errp, local_err);
return;
}
-
-/* We rely on power-of-2 blocksizes for bitmasks */
-if ((value & (value - 1)) != 0) {
-error_setg(errp,
- "Property %s.%s doesn't take value '%" PRId64 "', "
- "it's not a power of 2", dev->id ?: "", name,
(int64_t)value);
-return;
-}
-
*ptr = value;
}
diff --git a/util/block-helpers.c b/util/block-helpers.c
new file mode 100644
index 00..c4851432f5
--- /dev/null
+++ b/util/block-helpers.c
@@ -0,0 +1,46 @@
+/*
+ * Block utility functions
+ *
+ * Copyright IBM, Corp. 2011
+ * Copyright (c) 2020 Coiby Xu
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qapi/qmp/qerror.h"
+#include "block-helpers.h"
+
+/**
+ * check_block_size:
+ * @id: The unique ID of the object
+ * @name: The name of the property being validated
+ * @value: The block size in bytes
+ * @errp: A pointer to an area to store an error
+ *
+ * This function checks that the block size meets the following conditions:
+ * 1. At least MIN_BLOCK_SIZE
+ * 2. No larger than MAX_BLOCK_SIZE
+ * 3. A power of 2
+ */
+void check_block_size(const char *id, const char *name, int64_t value,
+ Error **errp)
+{
+/* value of 0 means "unset" */
+if (value && (value < MIN_BLOCK_SIZE || value > MAX_BLOCK_SIZE)) {
+error_setg(errp, QERR_PROPERTY_VALUE_OUT_OF_RANGE,
+ id, name, value
[PULL v3 16/28] block/export: report flush errors
Propagate the flush return value since errors are possible.
Signed-off-by: Stefan Hajnoczi
Message-id: 20200924151549.913737-11-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
block/export/vhost-user-blk-server.c | 11 +++
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/block/export/vhost-user-blk-server.c
b/block/export/vhost-user-blk-server.c
index 4d35232bf3..faefcfcaea 100644
--- a/block/export/vhost-user-blk-server.c
+++ b/block/export/vhost-user-blk-server.c
@@ -78,11 +78,11 @@ vu_block_discard_write_zeroes(VuBlockReq *req, struct iovec
*iov,
return -EINVAL;
}
-static void coroutine_fn vu_block_flush(VuBlockReq *req)
+static int coroutine_fn vu_block_flush(VuBlockReq *req)
{
VuBlockDev *vdev_blk = get_vu_block_device_by_server(req->server);
BlockBackend *backend = vdev_blk->backend;
-blk_co_flush(backend);
+return blk_co_flush(backend);
}
static void coroutine_fn vu_block_virtio_process_req(void *opaque)
@@ -152,8 +152,11 @@ static void coroutine_fn vu_block_virtio_process_req(void
*opaque)
break;
}
case VIRTIO_BLK_T_FLUSH:
-vu_block_flush(req);
-req->in->status = VIRTIO_BLK_S_OK;
+if (vu_block_flush(req) == 0) {
+req->in->status = VIRTIO_BLK_S_OK;
+} else {
+req->in->status = VIRTIO_BLK_S_IOERR;
+}
break;
case VIRTIO_BLK_T_GET_ID: {
size_t size = MIN(iov_size(&elem->in_sg[0], in_num),
--
2.26.2
[PULL v3 08/28] util/vhost-user-server: s/fileds/fields/ typo fix
Signed-off-by: Stefan Hajnoczi
Message-id: 20200924151549.913737-3-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
util/vhost-user-server.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c
index b189944856..9bd33e0fdb 100644
--- a/util/vhost-user-server.c
+++ b/util/vhost-user-server.c
@@ -407,7 +407,7 @@ bool vhost_user_server_start(VuServer *server,
return false;
}
-/* zero out unspecified fileds */
+/* zero out unspecified fields */
*server = (VuServer) {
.listener = listener,
.vu_iface = vu_iface,
--
2.26.2
[PULL v3 15/28] util/vhost-user-server: rework vu_client_trip() coroutine lifecycle
The vu_client_trip() coroutine is leaked during AioContext switching. It
is also unsafe to destroy the vu_dev in panic_cb() since its callers
still access it in some cases.
Rework the lifecycle to solve these safety issues.
Signed-off-by: Stefan Hajnoczi
Message-id: 20200924151549.913737-10-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
util/vhost-user-server.h | 29 ++--
block/export/vhost-user-blk-server.c | 9 +-
util/vhost-user-server.c | 245 +++
3 files changed, 155 insertions(+), 128 deletions(-)
diff --git a/util/vhost-user-server.h b/util/vhost-user-server.h
index 92177fc911..0da4c2cc4c 100644
--- a/util/vhost-user-server.h
+++ b/util/vhost-user-server.h
@@ -19,34 +19,36 @@
#include "qapi/error.h"
#include "standard-headers/linux/virtio_blk.h"
+/* A kick fd that we monitor on behalf of libvhost-user */
typedef struct VuFdWatch {
VuDev *vu_dev;
int fd; /*kick fd*/
void *pvt;
vu_watch_cb cb;
-bool processing;
QTAILQ_ENTRY(VuFdWatch) next;
} VuFdWatch;
-typedef struct VuServer VuServer;
-
-struct VuServer {
+/**
+ * VuServer:
+ * A vhost-user server instance with user-defined VuDevIface callbacks.
+ * Vhost-user device backends can be implemented using VuServer. VuDevIface
+ * callbacks and virtqueue kicks run in the given AioContext.
+ */
+typedef struct {
QIONetListener *listener;
+QEMUBH *restart_listener_bh;
AioContext *ctx;
int max_queues;
const VuDevIface *vu_iface;
+
+/* Protected by ctx lock */
VuDev vu_dev;
QIOChannel *ioc; /* The I/O channel with the client */
QIOChannelSocket *sioc; /* The underlying data channel with the client */
-/* IOChannel for fd provided via VHOST_USER_SET_SLAVE_REQ_FD */
-QIOChannel *ioc_slave;
-QIOChannelSocket *sioc_slave;
-Coroutine *co_trip; /* coroutine for processing VhostUserMsg */
QTAILQ_HEAD(, VuFdWatch) vu_fd_watches;
-/* restart coroutine co_trip if AIOContext is changed */
-bool aio_context_changed;
-bool processing_msg;
-};
+
+Coroutine *co_trip; /* coroutine for processing VhostUserMsg */
+} VuServer;
bool vhost_user_server_start(VuServer *server,
SocketAddress *unix_socket,
@@ -57,6 +59,7 @@ bool vhost_user_server_start(VuServer *server,
void vhost_user_server_stop(VuServer *server);
-void vhost_user_server_set_aio_context(VuServer *server, AioContext *ctx);
+void vhost_user_server_attach_aio_context(VuServer *server, AioContext *ctx);
+void vhost_user_server_detach_aio_context(VuServer *server);
#endif /* VHOST_USER_SERVER_H */
diff --git a/block/export/vhost-user-blk-server.c
b/block/export/vhost-user-blk-server.c
index c8fa4ecba9..4d35232bf3 100644
--- a/block/export/vhost-user-blk-server.c
+++ b/block/export/vhost-user-blk-server.c
@@ -313,18 +313,13 @@ static const VuDevIface vu_block_iface = {
static void blk_aio_attached(AioContext *ctx, void *opaque)
{
VuBlockDev *vub_dev = opaque;
-aio_context_acquire(ctx);
-vhost_user_server_set_aio_context(&vub_dev->vu_server, ctx);
-aio_context_release(ctx);
+vhost_user_server_attach_aio_context(&vub_dev->vu_server, ctx);
}
static void blk_aio_detach(void *opaque)
{
VuBlockDev *vub_dev = opaque;
-AioContext *ctx = vub_dev->vu_server.ctx;
-aio_context_acquire(ctx);
-vhost_user_server_set_aio_context(&vub_dev->vu_server, NULL);
-aio_context_release(ctx);
+vhost_user_server_detach_aio_context(&vub_dev->vu_server);
}
static void
diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c
index 981908fef0..c448800e58 100644
--- a/util/vhost-user-server.c
+++ b/util/vhost-user-server.c
@@ -9,8 +9,50 @@
*/
#include "qemu/osdep.h"
#include "qemu/main-loop.h"
+#include "block/aio-wait.h"
#include "vhost-user-server.h"
+/*
+ * Theory of operation:
+ *
+ * VuServer is started and stopped by vhost_user_server_start() and
+ * vhost_user_server_stop() from the main loop thread. Starting the server
+ * opens a vhost-user UNIX domain socket and listens for incoming connections.
+ * Only one connection is allowed at a time.
+ *
+ * The connection is handled by the vu_client_trip() coroutine in the
+ * VuServer->ctx AioContext. The coroutine consists of a vu_dispatch() loop
+ * where libvhost-user calls vu_message_read() to receive the next vhost-user
+ * protocol messages over the UNIX domain socket.
+ *
+ * When virtqueues are set up libvhost-user calls set_watch() to monitor kick
+ * fds. These fds are also handled in the VuServer->ctx AioContext.
+ *
+ * Both vu_client_trip() and kick fd monitoring can be stopped by shutting down
+ * the socket connection. Shutting down the socket connection causes
+ * vu_message_read() to fail since no more data can be received from the
socket.
+ * After vu_dispatch() fails, vu_client_trip() calls vu_deinit() to stop
+ * libvhost-user before terminating the coroutine. vu_deinit() calls
+ * remove
[PULL v3 13/28] util/vhost-user-server: fix memory leak in vu_message_read()
fds[] is leaked when qio_channel_readv_full() fails.
Use vmsg->fds[] instead of keeping a local fds[] array. Then we can
reuse goto fail to clean up fds. vmsg->fd_num must be zeroed before the
loop to make this safe.
Signed-off-by: Stefan Hajnoczi
Message-id: 20200924151549.913737-8-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
util/vhost-user-server.c | 50 ++--
1 file changed, 23 insertions(+), 27 deletions(-)
diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c
index 73a1667b54..a7b7a9897f 100644
--- a/util/vhost-user-server.c
+++ b/util/vhost-user-server.c
@@ -100,21 +100,11 @@ vu_message_read(VuDev *vu_dev, int conn_fd, VhostUserMsg
*vmsg)
};
int rc, read_bytes = 0;
Error *local_err = NULL;
-/*
- * Store fds/nfds returned from qio_channel_readv_full into
- * temporary variables.
- *
- * VhostUserMsg is a packed structure, gcc will complain about passing
- * pointer to a packed structure member if we pass &VhostUserMsg.fd_num
- * and &VhostUserMsg.fds directly when calling qio_channel_readv_full,
- * thus two temporary variables nfds and fds are used here.
- */
-size_t nfds = 0, nfds_t = 0;
const size_t max_fds = G_N_ELEMENTS(vmsg->fds);
-int *fds_t = NULL;
VuServer *server = container_of(vu_dev, VuServer, vu_dev);
QIOChannel *ioc = server->ioc;
+vmsg->fd_num = 0;
if (!ioc) {
error_report_err(local_err);
goto fail;
@@ -122,41 +112,47 @@ vu_message_read(VuDev *vu_dev, int conn_fd, VhostUserMsg
*vmsg)
assert(qemu_in_coroutine());
do {
+size_t nfds = 0;
+int *fds = NULL;
+
/*
* qio_channel_readv_full may have short reads, keeping calling it
* until getting VHOST_USER_HDR_SIZE or 0 bytes in total
*/
-rc = qio_channel_readv_full(ioc, &iov, 1, &fds_t, &nfds_t, &local_err);
+rc = qio_channel_readv_full(ioc, &iov, 1, &fds, &nfds, &local_err);
if (rc < 0) {
if (rc == QIO_CHANNEL_ERR_BLOCK) {
+assert(local_err == NULL);
qio_channel_yield(ioc, G_IO_IN);
continue;
} else {
error_report_err(local_err);
-return false;
+goto fail;
}
}
-read_bytes += rc;
-if (nfds_t > 0) {
-if (nfds + nfds_t > max_fds) {
+
+if (nfds > 0) {
+if (vmsg->fd_num + nfds > max_fds) {
error_report("A maximum of %zu fds are allowed, "
"however got %zu fds now",
- max_fds, nfds + nfds_t);
+ max_fds, vmsg->fd_num + nfds);
+g_free(fds);
goto fail;
}
-memcpy(vmsg->fds + nfds, fds_t,
- nfds_t *sizeof(vmsg->fds[0]));
-nfds += nfds_t;
-g_free(fds_t);
+memcpy(vmsg->fds + vmsg->fd_num, fds, nfds * sizeof(vmsg->fds[0]));
+vmsg->fd_num += nfds;
+g_free(fds);
}
-if (read_bytes == VHOST_USER_HDR_SIZE || rc == 0) {
-break;
+
+if (rc == 0) { /* socket closed */
+goto fail;
}
-iov.iov_base = (char *)vmsg + read_bytes;
-iov.iov_len = VHOST_USER_HDR_SIZE - read_bytes;
-} while (true);
-vmsg->fd_num = nfds;
+iov.iov_base += rc;
+iov.iov_len -= rc;
+read_bytes += rc;
+} while (read_bytes != VHOST_USER_HDR_SIZE);
+
/* qio_channel_readv_full will make socket fds blocking, unblock them */
vmsg_unblock_fds(vmsg);
if (vmsg->size > sizeof(vmsg->payload)) {
--
2.26.2
[PULL v3 14/28] util/vhost-user-server: check EOF when reading payload
Unexpected EOF is an error that must be reported.
Signed-off-by: Stefan Hajnoczi
Message-id: 20200924151549.913737-9-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
util/vhost-user-server.c | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c
index a7b7a9897f..981908fef0 100644
--- a/util/vhost-user-server.c
+++ b/util/vhost-user-server.c
@@ -169,8 +169,10 @@ vu_message_read(VuDev *vu_dev, int conn_fd, VhostUserMsg
*vmsg)
};
if (vmsg->size) {
rc = qio_channel_readv_all_eof(ioc, &iov_payload, 1, &local_err);
-if (rc == -1) {
-error_report_err(local_err);
+if (rc != 1) {
+if (local_err) {
+error_report_err(local_err);
+}
goto fail;
}
}
--
2.26.2
[PULL v3 11/28] block/export: consolidate request structs into VuBlockReq
Only one struct is needed per request. Drop req_data and the separate
VuBlockReq instance. Instead let vu_queue_pop() allocate everything at
once.
This fixes the req_data memory leak in vu_block_virtio_process_req().
Signed-off-by: Stefan Hajnoczi
Message-id: 20200924151549.913737-6-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
block/export/vhost-user-blk-server.c | 68 +---
1 file changed, 21 insertions(+), 47 deletions(-)
diff --git a/block/export/vhost-user-blk-server.c
b/block/export/vhost-user-blk-server.c
index 2ba1613cc9..d227b468d8 100644
--- a/block/export/vhost-user-blk-server.c
+++ b/block/export/vhost-user-blk-server.c
@@ -25,7 +25,7 @@ struct virtio_blk_inhdr {
};
typedef struct VuBlockReq {
-VuVirtqElement *elem;
+VuVirtqElement elem;
int64_t sector_num;
size_t size;
struct virtio_blk_inhdr *in;
@@ -39,14 +39,10 @@ static void vu_block_req_complete(VuBlockReq *req)
VuDev *vu_dev = &req->server->vu_dev;
/* IO size with 1 extra status byte */
-vu_queue_push(vu_dev, req->vq, req->elem, req->size + 1);
+vu_queue_push(vu_dev, req->vq, &req->elem, req->size + 1);
vu_queue_notify(vu_dev, req->vq);
-if (req->elem) {
-free(req->elem);
-}
-
-g_free(req);
+free(req);
}
static VuBlockDev *get_vu_block_device_by_server(VuServer *server)
@@ -89,20 +85,12 @@ static void coroutine_fn vu_block_flush(VuBlockReq *req)
blk_co_flush(backend);
}
-struct req_data {
-VuServer *server;
-VuVirtq *vq;
-VuVirtqElement *elem;
-};
-
static void coroutine_fn vu_block_virtio_process_req(void *opaque)
{
-struct req_data *data = opaque;
-VuServer *server = data->server;
-VuVirtq *vq = data->vq;
-VuVirtqElement *elem = data->elem;
+VuBlockReq *req = opaque;
+VuServer *server = req->server;
+VuVirtqElement *elem = &req->elem;
uint32_t type;
-VuBlockReq *req;
VuBlockDev *vdev_blk = get_vu_block_device_by_server(server);
BlockBackend *backend = vdev_blk->backend;
@@ -111,18 +99,13 @@ static void coroutine_fn vu_block_virtio_process_req(void
*opaque)
struct iovec *out_iov = elem->out_sg;
unsigned in_num = elem->in_num;
unsigned out_num = elem->out_num;
+
/* refer to hw/block/virtio_blk.c */
if (elem->out_num < 1 || elem->in_num < 1) {
error_report("virtio-blk request missing headers");
-free(elem);
-return;
+goto err;
}
-req = g_new0(VuBlockReq, 1);
-req->server = server;
-req->vq = vq;
-req->elem = elem;
-
if (unlikely(iov_to_buf(out_iov, out_num, 0, &req->out,
sizeof(req->out)) != sizeof(req->out))) {
error_report("virtio-blk request outhdr too short");
@@ -202,36 +185,27 @@ static void coroutine_fn vu_block_virtio_process_req(void
*opaque)
err:
free(elem);
-g_free(req);
-return;
}
static void vu_block_process_vq(VuDev *vu_dev, int idx)
{
-VuServer *server;
-VuVirtq *vq;
-struct req_data *req_data;
+VuServer *server = container_of(vu_dev, VuServer, vu_dev);
+VuVirtq *vq = vu_get_queue(vu_dev, idx);
-server = container_of(vu_dev, VuServer, vu_dev);
-assert(server);
-
-vq = vu_get_queue(vu_dev, idx);
-assert(vq);
-VuVirtqElement *elem;
while (1) {
-elem = vu_queue_pop(vu_dev, vq, sizeof(VuVirtqElement) +
-sizeof(VuBlockReq));
-if (elem) {
-req_data = g_new0(struct req_data, 1);
-req_data->server = server;
-req_data->vq = vq;
-req_data->elem = elem;
-Coroutine *co = qemu_coroutine_create(vu_block_virtio_process_req,
- req_data);
-aio_co_enter(server->ioc->ctx, co);
-} else {
+VuBlockReq *req;
+
+req = vu_queue_pop(vu_dev, vq, sizeof(VuBlockReq));
+if (!req) {
break;
}
+
+req->server = server;
+req->vq = vq;
+
+Coroutine *co =
+qemu_coroutine_create(vu_block_virtio_process_req, req);
+qemu_coroutine_enter(co);
}
}
--
2.26.2
[PULL v3 12/28] util/vhost-user-server: drop unused DevicePanicNotifier
The device panic notifier callback is not used. Drop it.
Signed-off-by: Stefan Hajnoczi
Message-id: 20200924151549.913737-7-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
util/vhost-user-server.h | 3 ---
block/export/vhost-user-blk-server.c | 3 +--
util/vhost-user-server.c | 6 --
3 files changed, 1 insertion(+), 11 deletions(-)
diff --git a/util/vhost-user-server.h b/util/vhost-user-server.h
index 5232f96718..92177fc911 100644
--- a/util/vhost-user-server.h
+++ b/util/vhost-user-server.h
@@ -29,12 +29,10 @@ typedef struct VuFdWatch {
} VuFdWatch;
typedef struct VuServer VuServer;
-typedef void DevicePanicNotifierFn(VuServer *server);
struct VuServer {
QIONetListener *listener;
AioContext *ctx;
-DevicePanicNotifierFn *device_panic_notifier;
int max_queues;
const VuDevIface *vu_iface;
VuDev vu_dev;
@@ -54,7 +52,6 @@ bool vhost_user_server_start(VuServer *server,
SocketAddress *unix_socket,
AioContext *ctx,
uint16_t max_queues,
- DevicePanicNotifierFn *device_panic_notifier,
const VuDevIface *vu_iface,
Error **errp);
diff --git a/block/export/vhost-user-blk-server.c
b/block/export/vhost-user-blk-server.c
index d227b468d8..c8fa4ecba9 100644
--- a/block/export/vhost-user-blk-server.c
+++ b/block/export/vhost-user-blk-server.c
@@ -439,8 +439,7 @@ static void vhost_user_blk_server_start(VuBlockDev
*vu_block_device,
ctx = bdrv_get_aio_context(blk_bs(vu_block_device->backend));
if (!vhost_user_server_start(&vu_block_device->vu_server, addr, ctx,
- VHOST_USER_BLK_MAX_QUEUES,
- NULL, &vu_block_iface,
+ VHOST_USER_BLK_MAX_QUEUES, &vu_block_iface,
errp)) {
goto error;
}
diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c
index 6efe2279fd..73a1667b54 100644
--- a/util/vhost-user-server.c
+++ b/util/vhost-user-server.c
@@ -81,10 +81,6 @@ static void panic_cb(VuDev *vu_dev, const char *buf)
close_client(server);
}
-if (server->device_panic_notifier) {
-server->device_panic_notifier(server);
-}
-
/*
* Set the callback function for network listener so another
* vhost-user client can connect to this server
@@ -385,7 +381,6 @@ bool vhost_user_server_start(VuServer *server,
SocketAddress *socket_addr,
AioContext *ctx,
uint16_t max_queues,
- DevicePanicNotifierFn *device_panic_notifier,
const VuDevIface *vu_iface,
Error **errp)
{
@@ -402,7 +397,6 @@ bool vhost_user_server_start(VuServer *server,
.vu_iface = vu_iface,
.max_queues= max_queues,
.ctx = ctx,
-.device_panic_notifier = device_panic_notifier,
};
qio_net_listener_set_name(server->listener, "vhost-user-backend-listener");
--
2.26.2
[PULL v3 06/28] block/export: vhost-user block device backend server
From: Coiby Xu
By making use of libvhost-user, block device drive can be shared to
the connected vhost-user client. Only one client can connect to the
server one time.
Since vhost-user-server needs a block drive to be created first, delay
the creation of this object.
Suggested-by: Kevin Wolf
Signed-off-by: Stefan Hajnoczi
Signed-off-by: Coiby Xu
Reviewed-by: Stefan Hajnoczi
Reviewed-by: Marc-André Lureau
Message-id: 20200918080912.321299-6-coiby...@gmail.com
[Shorten "vhost_user_blk_server" string to "vhost_user_blk" to avoid the
following compiler warning:
../block/export/vhost-user-blk-server.c:178:50: error: ‘%s’ directive output
truncated writing 21 bytes into a region of size 20 [-Werror=format-truncation=]
and fix "Invalid size %ld ..." ssize_t format string arguments for
32-bit hosts.
--Stefan]
Signed-off-by: Stefan Hajnoczi
---
block/export/vhost-user-blk-server.h | 36 ++
block/export/vhost-user-blk-server.c | 661 +++
softmmu/vl.c | 4 +
block/meson.build| 1 +
4 files changed, 702 insertions(+)
create mode 100644 block/export/vhost-user-blk-server.h
create mode 100644 block/export/vhost-user-blk-server.c
diff --git a/block/export/vhost-user-blk-server.h
b/block/export/vhost-user-blk-server.h
new file mode 100644
index 00..f06f37c4c8
--- /dev/null
+++ b/block/export/vhost-user-blk-server.h
@@ -0,0 +1,36 @@
+/*
+ * Sharing QEMU block devices via vhost-user protocal
+ *
+ * Copyright (c) Coiby Xu .
+ * Copyright (c) 2020 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+
+#ifndef VHOST_USER_BLK_SERVER_H
+#define VHOST_USER_BLK_SERVER_H
+#include "util/vhost-user-server.h"
+
+typedef struct VuBlockDev VuBlockDev;
+#define TYPE_VHOST_USER_BLK_SERVER "vhost-user-blk-server"
+#define VHOST_USER_BLK_SERVER(obj) \
+ OBJECT_CHECK(VuBlockDev, obj, TYPE_VHOST_USER_BLK_SERVER)
+
+/* vhost user block device */
+struct VuBlockDev {
+Object parent_obj;
+char *node_name;
+SocketAddress *addr;
+AioContext *ctx;
+VuServer vu_server;
+bool running;
+uint32_t blk_size;
+BlockBackend *backend;
+QIOChannelSocket *sioc;
+QTAILQ_ENTRY(VuBlockDev) next;
+struct virtio_blk_config blkcfg;
+bool writable;
+};
+
+#endif /* VHOST_USER_BLK_SERVER_H */
diff --git a/block/export/vhost-user-blk-server.c
b/block/export/vhost-user-blk-server.c
new file mode 100644
index 00..2ba1613cc9
--- /dev/null
+++ b/block/export/vhost-user-blk-server.c
@@ -0,0 +1,661 @@
+/*
+ * Sharing QEMU block devices via vhost-user protocal
+ *
+ * Parts of the code based on nbd/server.c.
+ *
+ * Copyright (c) Coiby Xu .
+ * Copyright (c) 2020 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "block/block.h"
+#include "vhost-user-blk-server.h"
+#include "qapi/error.h"
+#include "qom/object_interfaces.h"
+#include "sysemu/block-backend.h"
+#include "util/block-helpers.h"
+
+enum {
+VHOST_USER_BLK_MAX_QUEUES = 1,
+};
+struct virtio_blk_inhdr {
+unsigned char status;
+};
+
+typedef struct VuBlockReq {
+VuVirtqElement *elem;
+int64_t sector_num;
+size_t size;
+struct virtio_blk_inhdr *in;
+struct virtio_blk_outhdr out;
+VuServer *server;
+struct VuVirtq *vq;
+} VuBlockReq;
+
+static void vu_block_req_complete(VuBlockReq *req)
+{
+VuDev *vu_dev = &req->server->vu_dev;
+
+/* IO size with 1 extra status byte */
+vu_queue_push(vu_dev, req->vq, req->elem, req->size + 1);
+vu_queue_notify(vu_dev, req->vq);
+
+if (req->elem) {
+free(req->elem);
+}
+
+g_free(req);
+}
+
+static VuBlockDev *get_vu_block_device_by_server(VuServer *server)
+{
+return container_of(server, VuBlockDev, vu_server);
+}
+
+static int coroutine_fn
+vu_block_discard_write_zeroes(VuBlockReq *req, struct iovec *iov,
+ uint32_t iovcnt, uint32_t type)
+{
+struct virtio_blk_discard_write_zeroes desc;
+ssize_t size = iov_to_buf(iov, iovcnt, 0, &desc, sizeof(desc));
+if (unlikely(size != sizeof(desc))) {
+error_report("Invalid size %zd, expect %zu", size, sizeof(desc));
+return -EINVAL;
+}
+
+VuBlockDev *vdev_blk = get_vu_block_device_by_server(req->server);
+uint64_t range[2] = { le64_to_cpu(desc.sector) << 9,
+ le32_to_cpu(desc.num_sectors) << 9 };
+if (type == VIRTIO_BLK_T_DISCARD) {
+if (blk_co_pdiscard(vdev_blk->backend, range[0], range[1]) == 0) {
+return 0;
+}
+} else if (type == VIRTIO_BLK_T_WRITE_ZEROES) {
+if (blk_co_pwrite_zeroes(vdev_blk->backend,
+ range[0], range[1], 0) == 0) {
+return 0;
+}
+}
+
+return -EINVAL;
+}
+
+static
[PULL v3 03/28] libvhost-user: remove watch for kick_fd when de-initialize vu-dev
From: Coiby Xu
When the client is running in gdb and quit command is run in gdb,
QEMU will still dispatch the event which will cause segment fault in
the callback function.
Signed-off-by: Coiby Xu
Reviewed-by: Stefan Hajnoczi
Reviewed-by: Marc-André Lureau
Message-id: 20200918080912.321299-3-coiby...@gmail.com
Signed-off-by: Stefan Hajnoczi
---
contrib/libvhost-user/libvhost-user.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/contrib/libvhost-user/libvhost-user.c
b/contrib/libvhost-user/libvhost-user.c
index 09bdff18f3..bfec8a881a 100644
--- a/contrib/libvhost-user/libvhost-user.c
+++ b/contrib/libvhost-user/libvhost-user.c
@@ -1918,6 +1918,7 @@ vu_deinit(VuDev *dev)
}
if (vq->kick_fd != -1) {
+dev->remove_watch(dev, vq->kick_fd);
close(vq->kick_fd);
vq->kick_fd = -1;
}
--
2.26.2
[PULL v3 04/28] util/vhost-user-server: generic vhost user server
From: Coiby Xu
Sharing QEMU devices via vhost-user protocol.
Only one vhost-user client can connect to the server one time.
Suggested-by: Kevin Wolf
Signed-off-by: Stefan Hajnoczi
Signed-off-by: Coiby Xu
Reviewed-by: Stefan Hajnoczi
Reviewed-by: Marc-André Lureau
Message-id: 20200918080912.321299-4-coiby...@gmail.com
[Fixed size_t %lu -> %zu format string compiler error.
--Stefan]
Signed-off-by: Stefan Hajnoczi
---
util/vhost-user-server.h | 65 ++
util/vhost-user-server.c | 428 +++
util/meson.build | 1 +
3 files changed, 494 insertions(+)
create mode 100644 util/vhost-user-server.h
create mode 100644 util/vhost-user-server.c
diff --git a/util/vhost-user-server.h b/util/vhost-user-server.h
new file mode 100644
index 00..5232f96718
--- /dev/null
+++ b/util/vhost-user-server.h
@@ -0,0 +1,65 @@
+/*
+ * Sharing QEMU devices via vhost-user protocol
+ *
+ * Copyright (c) Coiby Xu .
+ * Copyright (c) 2020 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+
+#ifndef VHOST_USER_SERVER_H
+#define VHOST_USER_SERVER_H
+
+#include "contrib/libvhost-user/libvhost-user.h"
+#include "io/channel-socket.h"
+#include "io/channel-file.h"
+#include "io/net-listener.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "standard-headers/linux/virtio_blk.h"
+
+typedef struct VuFdWatch {
+VuDev *vu_dev;
+int fd; /*kick fd*/
+void *pvt;
+vu_watch_cb cb;
+bool processing;
+QTAILQ_ENTRY(VuFdWatch) next;
+} VuFdWatch;
+
+typedef struct VuServer VuServer;
+typedef void DevicePanicNotifierFn(VuServer *server);
+
+struct VuServer {
+QIONetListener *listener;
+AioContext *ctx;
+DevicePanicNotifierFn *device_panic_notifier;
+int max_queues;
+const VuDevIface *vu_iface;
+VuDev vu_dev;
+QIOChannel *ioc; /* The I/O channel with the client */
+QIOChannelSocket *sioc; /* The underlying data channel with the client */
+/* IOChannel for fd provided via VHOST_USER_SET_SLAVE_REQ_FD */
+QIOChannel *ioc_slave;
+QIOChannelSocket *sioc_slave;
+Coroutine *co_trip; /* coroutine for processing VhostUserMsg */
+QTAILQ_HEAD(, VuFdWatch) vu_fd_watches;
+/* restart coroutine co_trip if AIOContext is changed */
+bool aio_context_changed;
+bool processing_msg;
+};
+
+bool vhost_user_server_start(VuServer *server,
+ SocketAddress *unix_socket,
+ AioContext *ctx,
+ uint16_t max_queues,
+ DevicePanicNotifierFn *device_panic_notifier,
+ const VuDevIface *vu_iface,
+ Error **errp);
+
+void vhost_user_server_stop(VuServer *server);
+
+void vhost_user_server_set_aio_context(VuServer *server, AioContext *ctx);
+
+#endif /* VHOST_USER_SERVER_H */
diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c
new file mode 100644
index 00..b189944856
--- /dev/null
+++ b/util/vhost-user-server.c
@@ -0,0 +1,428 @@
+/*
+ * Sharing QEMU devices via vhost-user protocol
+ *
+ * Copyright (c) Coiby Xu .
+ * Copyright (c) 2020 Red Hat, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or
+ * later. See the COPYING file in the top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "qemu/main-loop.h"
+#include "vhost-user-server.h"
+
+static void vmsg_close_fds(VhostUserMsg *vmsg)
+{
+int i;
+for (i = 0; i < vmsg->fd_num; i++) {
+close(vmsg->fds[i]);
+}
+}
+
+static void vmsg_unblock_fds(VhostUserMsg *vmsg)
+{
+int i;
+for (i = 0; i < vmsg->fd_num; i++) {
+qemu_set_nonblock(vmsg->fds[i]);
+}
+}
+
+static void vu_accept(QIONetListener *listener, QIOChannelSocket *sioc,
+ gpointer opaque);
+
+static void close_client(VuServer *server)
+{
+/*
+ * Before closing the client
+ *
+ * 1. Let vu_client_trip stop processing new vhost-user msg
+ *
+ * 2. remove kick_handler
+ *
+ * 3. wait for the kick handler to be finished
+ *
+ * 4. wait for the current vhost-user msg to be finished processing
+ */
+
+QIOChannelSocket *sioc = server->sioc;
+/* When this is set vu_client_trip will stop new processing vhost-user
message */
+server->sioc = NULL;
+
+VuFdWatch *vu_fd_watch, *next;
+QTAILQ_FOREACH_SAFE(vu_fd_watch, &server->vu_fd_watches, next, next) {
+aio_set_fd_handler(server->ioc->ctx, vu_fd_watch->fd, true, NULL,
+ NULL, NULL, NULL);
+}
+
+while (!QTAILQ_EMPTY(&server->vu_fd_watches)) {
+QTAILQ_FOREACH_SAFE(vu_fd_watch, &server->vu_fd_watches, next, next) {
+if (!vu_fd_watch->processing) {
+QTAILQ_REMOVE(&server->vu_fd_watches, vu_fd_watch, next);
+g_free(vu_fd_watch);
+}
+
[PULL v3 18/28] util/vhost-user-server: move header to include/
Headers used by other subsystems are located in include/. Also add the
vhost-user-server and vhost-user-blk-server headers to MAINTAINERS.
Signed-off-by: Stefan Hajnoczi
Message-id: 20200924151549.913737-13-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
MAINTAINERS| 4 +++-
{util => include/qemu}/vhost-user-server.h | 0
block/export/vhost-user-blk-server.c | 2 +-
util/vhost-user-server.c | 2 +-
4 files changed, 5 insertions(+), 3 deletions(-)
rename {util => include/qemu}/vhost-user-server.h (100%)
diff --git a/MAINTAINERS b/MAINTAINERS
index 2789952a30..ef6f5c7399 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3073,8 +3073,10 @@ Vhost-user block device backend server
M: Coiby Xu
S: Maintained
F: block/export/vhost-user-blk-server.c
-F: util/vhost-user-server.c
+F: block/export/vhost-user-blk-server.h
+F: include/qemu/vhost-user-server.h
F: tests/qtest/libqos/vhost-user-blk.c
+F: util/vhost-user-server.c
Replication
M: Wen Congyang
diff --git a/util/vhost-user-server.h b/include/qemu/vhost-user-server.h
similarity index 100%
rename from util/vhost-user-server.h
rename to include/qemu/vhost-user-server.h
diff --git a/block/export/vhost-user-blk-server.c
b/block/export/vhost-user-blk-server.c
index 3e5bd6caee..f7021cbd7b 100644
--- a/block/export/vhost-user-blk-server.c
+++ b/block/export/vhost-user-blk-server.c
@@ -13,7 +13,7 @@
#include "block/block.h"
#include "contrib/libvhost-user/libvhost-user.h"
#include "standard-headers/linux/virtio_blk.h"
-#include "util/vhost-user-server.h"
+#include "qemu/vhost-user-server.h"
#include "vhost-user-blk-server.h"
#include "qapi/error.h"
#include "qom/object_interfaces.h"
diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c
index 516999b38a..783d847a6d 100644
--- a/util/vhost-user-server.c
+++ b/util/vhost-user-server.c
@@ -9,8 +9,8 @@
*/
#include "qemu/osdep.h"
#include "qemu/main-loop.h"
+#include "qemu/vhost-user-server.h"
#include "block/aio-wait.h"
-#include "vhost-user-server.h"
/*
* Theory of operation:
--
2.26.2
[PULL v3 24/28] block/io: fix bdrv_co_block_status_above
From: Vladimir Sementsov-Ogievskiy
bdrv_co_block_status_above has several design problems with handling
short backing files:
1. With want_zeros=true, it may return ret with BDRV_BLOCK_ZERO but
without BDRV_BLOCK_ALLOCATED flag, when actually short backing file
which produces these after-EOF zeros is inside requested backing
sequence.
2. With want_zero=false, it may return pnum=0 prior to actual EOF,
because of EOF of short backing file.
Fix these things, making logic about short backing files clearer.
With fixed bdrv_block_status_above we also have to improve is_zero in
qcow2 code, otherwise iotest 154 will fail, because with this patch we
stop to merge zeros of different types (produced by fully unallocated
in the whole backing chain regions vs produced by short backing files).
Note also, that this patch leaves for another day the general problem
around block-status: misuse of BDRV_BLOCK_ALLOCATED as is-fs-allocated
vs go-to-backing.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Alberto Garcia
Reviewed-by: Eric Blake
Message-id: 20200924194003.22080-2-vsement...@virtuozzo.com
[Fix s/comes/come/ as suggested by Eric Blake
--Stefan]
Signed-off-by: Stefan Hajnoczi
---
block/io.c| 68 ---
block/qcow2.c | 16 ++--
2 files changed, 68 insertions(+), 16 deletions(-)
diff --git a/block/io.c b/block/io.c
index 54f0968aee..a718d50ca2 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2350,34 +2350,74 @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
int64_t *map,
BlockDriverState **file)
{
+int ret;
BlockDriverState *p;
-int ret = 0;
-bool first = true;
+int64_t eof = 0;
assert(bs != base);
-for (p = bs; p != base; p = bdrv_filter_or_cow_bs(p)) {
+
+ret = bdrv_co_block_status(bs, want_zero, offset, bytes, pnum, map, file);
+if (ret < 0 || *pnum == 0 || ret & BDRV_BLOCK_ALLOCATED) {
+return ret;
+}
+
+if (ret & BDRV_BLOCK_EOF) {
+eof = offset + *pnum;
+}
+
+assert(*pnum <= bytes);
+bytes = *pnum;
+
+for (p = bdrv_filter_or_cow_bs(bs); p != base;
+ p = bdrv_filter_or_cow_bs(p))
+{
ret = bdrv_co_block_status(p, want_zero, offset, bytes, pnum, map,
file);
if (ret < 0) {
-break;
+return ret;
}
-if (ret & BDRV_BLOCK_ZERO && ret & BDRV_BLOCK_EOF && !first) {
+if (*pnum == 0) {
/*
- * Reading beyond the end of the file continues to read
- * zeroes, but we can only widen the result to the
- * unallocated length we learned from an earlier
- * iteration.
+ * The top layer deferred to this layer, and because this layer is
+ * short, any zeroes that we synthesize beyond EOF behave as if
they
+ * were allocated at this layer.
+ *
+ * We don't include BDRV_BLOCK_EOF into ret, as upper layer may be
+ * larger. We'll add BDRV_BLOCK_EOF if needed at function end, see
+ * below.
*/
+assert(ret & BDRV_BLOCK_EOF);
*pnum = bytes;
+if (file) {
+*file = p;
+}
+ret = BDRV_BLOCK_ZERO | BDRV_BLOCK_ALLOCATED;
+break;
}
-if (ret & (BDRV_BLOCK_ZERO | BDRV_BLOCK_DATA)) {
+if (ret & BDRV_BLOCK_ALLOCATED) {
+/*
+ * We've found the node and the status, we must break.
+ *
+ * Drop BDRV_BLOCK_EOF, as it's not for upper layer, which may be
+ * larger. We'll add BDRV_BLOCK_EOF if needed at function end, see
+ * below.
+ */
+ret &= ~BDRV_BLOCK_EOF;
break;
}
-/* [offset, pnum] unallocated on this layer, which could be only
- * the first part of [offset, bytes]. */
-bytes = MIN(bytes, *pnum);
-first = false;
+
+/*
+ * OK, [offset, offset + *pnum) region is unallocated on this layer,
+ * let's continue the diving.
+ */
+assert(*pnum <= bytes);
+bytes = *pnum;
+}
+
+if (offset + *pnum == eof) {
+ret |= BDRV_BLOCK_EOF;
}
+
return ret;
}
diff --git a/block/qcow2.c b/block/qcow2.c
index b05512718c..b6cb4db8bb 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -3860,8 +3860,20 @@ static bool is_zero(BlockDriverState *bs, int64_t
offset, int64_t bytes)
if (!bytes) {
return true;
}
-res = bdrv_block_status_above(bs, NULL, offset, bytes, &nr, NULL, NULL);
-return res >= 0 && (res & BDRV_BLOCK_ZERO) && nr == bytes;
+
+/*
+ * bdrv_block_status_above doesn't merge different types of zeros, for
+ * example, zeros which come from the region which is unallocated in
+ * the whole backing c
[PULL v3 26/28] block/io: bdrv_common_block_status_above: support bs == base
From: Vladimir Sementsov-Ogievskiy
We are going to reuse bdrv_common_block_status_above in
bdrv_is_allocated_above. bdrv_is_allocated_above may be called with
include_base == false and still bs == base (for ex. from img_rebase()).
So, support this corner case.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Kevin Wolf
Reviewed-by: Eric Blake
Reviewed-by: Alberto Garcia
Message-id: 20200924194003.22080-4-vsement...@virtuozzo.com
Signed-off-by: Stefan Hajnoczi
---
block/io.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/block/io.c b/block/io.c
index 86f76d04bf..b616bc4ada 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2355,9 +2355,13 @@ bdrv_co_common_block_status_above(BlockDriverState *bs,
BlockDriverState *p;
int64_t eof = 0;
-assert(include_base || bs != base);
assert(!include_base || base); /* Can't include NULL base */
+if (!include_base && bs == base) {
+*pnum = bytes;
+return 0;
+}
+
ret = bdrv_co_block_status(bs, want_zero, offset, bytes, pnum, map, file);
if (ret < 0 || *pnum == 0 || ret & BDRV_BLOCK_ALLOCATED || bs == base) {
return ret;
--
2.26.2
[PULL v3 09/28] util/vhost-user-server: drop unnecessary QOM cast
We already have access to the value with the correct type (ioc and sioc are the same QIOChannel). Signed-off-by: Stefan Hajnoczi Message-id: 20200924151549.913737-4-stefa...@redhat.com Signed-off-by: Stefan Hajnoczi --- util/vhost-user-server.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c index 9bd33e0fdb..443ab7448c 100644 --- a/util/vhost-user-server.c +++ b/util/vhost-user-server.c @@ -337,7 +337,7 @@ static void vu_accept(QIONetListener *listener, QIOChannelSocket *sioc, server->ioc = QIO_CHANNEL(sioc); object_ref(OBJECT(server->ioc)); qio_channel_attach_aio_context(server->ioc, server->ctx); -qio_channel_set_blocking(QIO_CHANNEL(server->sioc), false, NULL); +qio_channel_set_blocking(server->ioc, false, NULL); vu_client_start(server); } -- 2.26.2
[PULL v3 01/28] block/nvme: Add driver statistics for access alignment and hw errors
From: Philippe Mathieu-Daudé
Keep statistics of some hardware errors, and number of
aligned/unaligned I/O accesses.
QMP example booting a full RHEL 8.3 aarch64 guest:
{ "execute": "query-blockstats" }
{
"return": [
{
"device": "",
"node-name": "drive0",
"stats": {
"flush_total_time_ns": 6026948,
"wr_highest_offset": 3383991230464,
"wr_total_time_ns": 807450995,
"failed_wr_operations": 0,
"failed_rd_operations": 0,
"wr_merged": 3,
"wr_bytes": 50133504,
"failed_unmap_operations": 0,
"failed_flush_operations": 0,
"account_invalid": false,
"rd_total_time_ns": 1846979900,
"flush_operations": 130,
"wr_operations": 659,
"rd_merged": 1192,
"rd_bytes": 218244096,
"account_failed": false,
"idle_time_ns": 2678641497,
"rd_operations": 7406,
},
"driver-specific": {
"driver": "nvme",
"completion-errors": 0,
"unaligned-accesses": 2959,
"aligned-accesses": 4477
},
"qdev": "/machine/peripheral-anon/device[0]/virtio-backend"
}
]
}
Suggested-by: Stefan Hajnoczi
Signed-off-by: Philippe Mathieu-Daudé
Acked-by: Markus Armbruster
Message-id: 20201001162939.1567915-1-phi...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
qapi/block-core.json | 24 +++-
block/nvme.c | 27 +++
2 files changed, 50 insertions(+), 1 deletion(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index ee5ebef7f2..e00fc27b5e 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -947,6 +947,27 @@
'discard-nb-failed': 'uint64',
'discard-bytes-ok': 'uint64' } }
+##
+# @BlockStatsSpecificNvme:
+#
+# NVMe driver statistics
+#
+# @completion-errors: The number of completion errors.
+#
+# @aligned-accesses: The number of aligned accesses performed by
+#the driver.
+#
+# @unaligned-accesses: The number of unaligned accesses performed by
+# the driver.
+#
+# Since: 5.2
+##
+{ 'struct': 'BlockStatsSpecificNvme',
+ 'data': {
+ 'completion-errors': 'uint64',
+ 'aligned-accesses': 'uint64',
+ 'unaligned-accesses': 'uint64' } }
+
##
# @BlockStatsSpecific:
#
@@ -959,7 +980,8 @@
'discriminator': 'driver',
'data': {
'file': 'BlockStatsSpecificFile',
- 'host_device': 'BlockStatsSpecificFile' } }
+ 'host_device': 'BlockStatsSpecificFile',
+ 'nvme': 'BlockStatsSpecificNvme' } }
##
# @BlockStats:
diff --git a/block/nvme.c b/block/nvme.c
index b48f6f2588..739a0a700c 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -128,6 +128,12 @@ struct BDRVNVMeState {
/* PCI address (required for nvme_refresh_filename()) */
char *device;
+
+struct {
+uint64_t completion_errors;
+uint64_t aligned_accesses;
+uint64_t unaligned_accesses;
+} stats;
};
#define NVME_BLOCK_OPT_DEVICE "device"
@@ -384,6 +390,9 @@ static bool nvme_process_completion(NVMeQueuePair *q)
break;
}
ret = nvme_translate_error(c);
+if (ret) {
+s->stats.completion_errors++;
+}
q->cq.head = (q->cq.head + 1) % NVME_QUEUE_SIZE;
if (!q->cq.head) {
q->cq_phase = !q->cq_phase;
@@ -1155,8 +1164,10 @@ static int nvme_co_prw(BlockDriverState *bs, uint64_t
offset, uint64_t bytes,
assert(QEMU_IS_ALIGNED(bytes, s->page_size));
assert(bytes <= s->max_transfer);
if (nvme_qiov_aligned(bs, qiov)) {
+s->stats.aligned_accesses++;
return nvme_co_prw_aligned(bs, offset, bytes, qiov, is_write, flags);
}
+s->stats.unaligned_accesses++;
trace_nvme_prw_buffered(s, offset, bytes, qiov->niov, is_write);
buf = qemu_try_memalign(s->page_size, bytes);
@@ -1452,6 +1463,21 @@ static void nvme_unregister_buf(BlockDriverState *bs,
void *host)
qemu_vfio_dma_unmap(s->vfio, host);
}
+static BlockStatsSpecific *nvme_get_specific_stats(BlockDriverState *bs)
+{
+BlockStatsSpecific *stats = g_new(BlockStatsSpecific, 1);
+BDRVNVMeState *s = bs->opaque;
+
+stats->driver = BLOCKDEV_DRIVER_NVME;
+stats->u.nvme = (BlockStatsSpecificNvme) {
+.completion_errors = s->stats.completion_errors,
+.aligned_accesses = s->stats.aligned_accesses,
+.unaligned_accesses = s->stats.unaligned_accesses,
+};
+
+return stats;
+}
+
static const char *const nvme_strong_runtime_opts[] = {
NVME_BLOCK_OPT_DEVICE,
NVME_BLOCK_OPT_NAMESPACE,
@@ -1485,6 +1511,7 @@ static BlockDriver bdrv_nvme = {
.bdrv_refresh_filename= nvme_refresh_filename,
.bdrv_refresh_limits = nvme_refresh_limits,
[PULL v3 00/28] Block patches
The following changes since commit 4c5b97bfd0dd54dc27717ae8d1cd10e14eef1430: Merge remote-tracking branch 'remotes/kraxel/tags/modules-20201022-pull-request' into staging (2020-10-22 12:33:21 +0100) are available in the Git repository at: https://gitlab.com/stefanha/qemu.git tags/block-pull-request for you to fetch changes up to 4f193168af097ef664287f4c801724f063a60fc2: iotests: add commit top->base cases to 274 (2020-10-23 13:42:16 +0100) Pull request v3: * Fix linker error on FreeBSD and other !CONFIG_LINUX host OSes [Peter] Coiby Xu (6): libvhost-user: Allow vu_message_read to be replaced libvhost-user: remove watch for kick_fd when de-initialize vu-dev util/vhost-user-server: generic vhost user server block: move logical block size check function to a common utility function block/export: vhost-user block device backend server MAINTAINERS: Add vhost-user block device backend server maintainer Philippe Mathieu-Daudé (1): block/nvme: Add driver statistics for access alignment and hw errors Stefan Hajnoczi (16): util/vhost-user-server: s/fileds/fields/ typo fix util/vhost-user-server: drop unnecessary QOM cast util/vhost-user-server: drop unnecessary watch deletion block/export: consolidate request structs into VuBlockReq util/vhost-user-server: drop unused DevicePanicNotifier util/vhost-user-server: fix memory leak in vu_message_read() util/vhost-user-server: check EOF when reading payload util/vhost-user-server: rework vu_client_trip() coroutine lifecycle block/export: report flush errors block/export: convert vhost-user-blk server to block export API util/vhost-user-server: move header to include/ util/vhost-user-server: use static library in meson.build qemu-storage-daemon: avoid compiling blockdev_ss twice block: move block exports to libblockdev block/export: add iothread and fixed-iothread options block/export: add vhost-user-blk multi-queue support Vladimir Sementsov-Ogievskiy (5): block/io: fix bdrv_co_block_status_above block/io: bdrv_common_block_status_above: support include_base block/io: bdrv_common_block_status_above: support bs == base block/io: fix bdrv_is_allocated_above iotests: add commit top->base cases to 274 MAINTAINERS| 9 + qapi/block-core.json | 24 +- qapi/block-export.json | 36 +- block/coroutines.h | 2 + block/export/vhost-user-blk-server.h | 19 + contrib/libvhost-user/libvhost-user.h | 21 + include/qemu/vhost-user-server.h | 65 +++ util/block-helpers.h | 19 + block/export/export.c | 37 +- block/export/vhost-user-blk-server.c | 431 block/io.c | 132 +++--- block/nvme.c | 27 ++ block/qcow2.c | 16 +- contrib/libvhost-user/libvhost-user-glib.c | 2 +- contrib/libvhost-user/libvhost-user.c | 15 +- hw/core/qdev-properties-system.c | 31 +- nbd/server.c | 2 - qemu-nbd.c | 21 +- softmmu/vl.c | 4 + stubs/blk-exp-close-all.c | 7 + tests/vhost-user-bridge.c | 2 + tools/virtiofsd/fuse_virtio.c | 4 +- util/block-helpers.c | 46 +++ util/vhost-user-server.c | 446 + block/export/meson.build | 3 +- contrib/libvhost-user/meson.build | 1 + meson.build| 22 +- nbd/meson.build| 2 + storage-daemon/meson.build | 3 +- stubs/meson.build | 1 + tests/qemu-iotests/274 | 20 + tests/qemu-iotests/274.out | 68 util/meson.build | 4 + 33 files changed, 1420 insertions(+), 122 deletions(-) create mode 100644 block/export/vhost-user-blk-server.h create mode 100644 include/qemu/vhost-user-server.h create mode 100644 util/block-helpers.h create mode 100644 block/export/vhost-user-blk-server.c create mode 100644 stubs/blk-exp-close-all.c create mode 100644 util/block-helpers.c create mode 100644 util/vhost-user-server.c -- 2.26.2
[PULL v3 02/28] libvhost-user: Allow vu_message_read to be replaced
From: Coiby Xu
Allow vu_message_read to be replaced by one which will make use of the
QIOChannel functions. Thus reading vhost-user message won't stall the
guest. For slave channel, we still use the default vu_message_read.
Reviewed-by: Marc-André Lureau
Signed-off-by: Coiby Xu
Reviewed-by: Stefan Hajnoczi
Message-id: 20200918080912.321299-2-coiby...@gmail.com
Signed-off-by: Stefan Hajnoczi
---
contrib/libvhost-user/libvhost-user.h | 21 +
contrib/libvhost-user/libvhost-user-glib.c | 2 +-
contrib/libvhost-user/libvhost-user.c | 14 +++---
tests/vhost-user-bridge.c | 2 ++
tools/virtiofsd/fuse_virtio.c | 4 ++--
5 files changed, 33 insertions(+), 10 deletions(-)
diff --git a/contrib/libvhost-user/libvhost-user.h
b/contrib/libvhost-user/libvhost-user.h
index 287ac5fec7..3bbeae8587 100644
--- a/contrib/libvhost-user/libvhost-user.h
+++ b/contrib/libvhost-user/libvhost-user.h
@@ -36,6 +36,8 @@
*/
#define VHOST_USER_MAX_RAM_SLOTS 32
+#define VHOST_USER_HDR_SIZE offsetof(VhostUserMsg, payload.u64)
+
typedef enum VhostSetConfigType {
VHOST_SET_CONFIG_TYPE_MASTER = 0,
VHOST_SET_CONFIG_TYPE_MIGRATION = 1,
@@ -221,6 +223,7 @@ typedef uint64_t (*vu_get_features_cb) (VuDev *dev);
typedef void (*vu_set_features_cb) (VuDev *dev, uint64_t features);
typedef int (*vu_process_msg_cb) (VuDev *dev, VhostUserMsg *vmsg,
int *do_reply);
+typedef bool (*vu_read_msg_cb) (VuDev *dev, int sock, VhostUserMsg *vmsg);
typedef void (*vu_queue_set_started_cb) (VuDev *dev, int qidx, bool started);
typedef bool (*vu_queue_is_processed_in_order_cb) (VuDev *dev, int qidx);
typedef int (*vu_get_config_cb) (VuDev *dev, uint8_t *config, uint32_t len);
@@ -389,6 +392,23 @@ struct VuDev {
bool broken;
uint16_t max_queues;
+/* @read_msg: custom method to read vhost-user message
+ *
+ * Read data from vhost_user socket fd and fill up
+ * the passed VhostUserMsg *vmsg struct.
+ *
+ * If reading fails, it should close the received set of file
+ * descriptors as socket message's auxiliary data.
+ *
+ * For the details, please refer to vu_message_read in libvhost-user.c
+ * which will be used by default if not custom method is provided when
+ * calling vu_init
+ *
+ * Returns: true if vhost-user message successfully received,
+ * otherwise return false.
+ *
+ */
+vu_read_msg_cb read_msg;
/* @set_watch: add or update the given fd to the watch set,
* call cb when condition is met */
vu_set_watch_cb set_watch;
@@ -432,6 +452,7 @@ bool vu_init(VuDev *dev,
uint16_t max_queues,
int socket,
vu_panic_cb panic,
+ vu_read_msg_cb read_msg,
vu_set_watch_cb set_watch,
vu_remove_watch_cb remove_watch,
const VuDevIface *iface);
diff --git a/contrib/libvhost-user/libvhost-user-glib.c
b/contrib/libvhost-user/libvhost-user-glib.c
index 53f1ca4cdd..0df2ec9271 100644
--- a/contrib/libvhost-user/libvhost-user-glib.c
+++ b/contrib/libvhost-user/libvhost-user-glib.c
@@ -147,7 +147,7 @@ vug_init(VugDev *dev, uint16_t max_queues, int socket,
g_assert(dev);
g_assert(iface);
-if (!vu_init(&dev->parent, max_queues, socket, panic, set_watch,
+if (!vu_init(&dev->parent, max_queues, socket, panic, NULL, set_watch,
remove_watch, iface)) {
return false;
}
diff --git a/contrib/libvhost-user/libvhost-user.c
b/contrib/libvhost-user/libvhost-user.c
index 9f1285b8a1..09bdff18f3 100644
--- a/contrib/libvhost-user/libvhost-user.c
+++ b/contrib/libvhost-user/libvhost-user.c
@@ -68,8 +68,6 @@
/* The version of inflight buffer */
#define INFLIGHT_VERSION 1
-#define VHOST_USER_HDR_SIZE offsetof(VhostUserMsg, payload.u64)
-
/* The version of the protocol we support */
#define VHOST_USER_VERSION 1
#define LIBVHOST_USER_DEBUG 0
@@ -268,7 +266,7 @@ have_userfault(void)
}
static bool
-vu_message_read(VuDev *dev, int conn_fd, VhostUserMsg *vmsg)
+vu_message_read_default(VuDev *dev, int conn_fd, VhostUserMsg *vmsg)
{
char control[CMSG_SPACE(VHOST_MEMORY_BASELINE_NREGIONS * sizeof(int))] =
{};
struct iovec iov = {
@@ -416,7 +414,7 @@ vu_process_message_reply(VuDev *dev, const VhostUserMsg
*vmsg)
goto out;
}
-if (!vu_message_read(dev, dev->slave_fd, &msg_reply)) {
+if (!vu_message_read_default(dev, dev->slave_fd, &msg_reply)) {
goto out;
}
@@ -907,7 +905,7 @@ vu_set_mem_table_exec_postcopy(VuDev *dev, VhostUserMsg
*vmsg)
/* Wait for QEMU to confirm that it's registered the handler for the
* faults.
*/
-if (!vu_message_read(dev, dev->sock, vmsg) ||
+if (!dev->read_msg(dev, dev->sock, vmsg) ||
vmsg->size != sizeof(vmsg->payload.u64) ||
vmsg->payload.u64 != 0) {
vu_panic(dev, "failed to receive valid ack f
[PULL v3 07/28] MAINTAINERS: Add vhost-user block device backend server maintainer
From: Coiby Xu Suggested-by: Stefano Garzarella Signed-off-by: Coiby Xu Reviewed-by: Stefan Hajnoczi Reviewed-by: Marc-André Lureau Message-id: 20200918080912.321299-8-coiby...@gmail.com [Removed reference to vhost-user-blk-test.c, it will be sent in a separate pull request. --Stefan] Signed-off-by: Stefan Hajnoczi --- MAINTAINERS | 7 +++ 1 file changed, 7 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 6a197bd358..2789952a30 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3069,6 +3069,13 @@ L: qemu-block@nongnu.org S: Supported F: tests/image-fuzzer/ +Vhost-user block device backend server +M: Coiby Xu +S: Maintained +F: block/export/vhost-user-blk-server.c +F: util/vhost-user-server.c +F: tests/qtest/libqos/vhost-user-blk.c + Replication M: Wen Congyang M: Xie Changlong -- 2.26.2
[PULL v3 10/28] util/vhost-user-server: drop unnecessary watch deletion
Explicitly deleting watches is not necessary since libvhost-user calls
remove_watch() during vu_deinit(). Add an assertion to check this
though.
Signed-off-by: Stefan Hajnoczi
Message-id: 20200924151549.913737-5-stefa...@redhat.com
Signed-off-by: Stefan Hajnoczi
---
util/vhost-user-server.c | 19 ---
1 file changed, 4 insertions(+), 15 deletions(-)
diff --git a/util/vhost-user-server.c b/util/vhost-user-server.c
index 443ab7448c..6efe2279fd 100644
--- a/util/vhost-user-server.c
+++ b/util/vhost-user-server.c
@@ -48,21 +48,6 @@ static void close_client(VuServer *server)
/* When this is set vu_client_trip will stop new processing vhost-user
message */
server->sioc = NULL;
-VuFdWatch *vu_fd_watch, *next;
-QTAILQ_FOREACH_SAFE(vu_fd_watch, &server->vu_fd_watches, next, next) {
-aio_set_fd_handler(server->ioc->ctx, vu_fd_watch->fd, true, NULL,
- NULL, NULL, NULL);
-}
-
-while (!QTAILQ_EMPTY(&server->vu_fd_watches)) {
-QTAILQ_FOREACH_SAFE(vu_fd_watch, &server->vu_fd_watches, next, next) {
-if (!vu_fd_watch->processing) {
-QTAILQ_REMOVE(&server->vu_fd_watches, vu_fd_watch, next);
-g_free(vu_fd_watch);
-}
-}
-}
-
while (server->processing_msg) {
if (server->ioc->read_coroutine) {
server->ioc->read_coroutine = NULL;
@@ -73,6 +58,10 @@ static void close_client(VuServer *server)
}
vu_deinit(&server->vu_dev);
+
+/* vu_deinit() should have called remove_watch() */
+assert(QTAILQ_EMPTY(&server->vu_fd_watches));
+
object_unref(OBJECT(sioc));
object_unref(OBJECT(server->ioc));
}
--
2.26.2
[PATCH v2] block: End quiescent sections when a BDS is deleted
If a BDS gets deleted during blk_drain_all(), it might miss a
call to bdrv_do_drained_end(). This means missing a call to
aio_enable_external() and the AIO context remains disabled for
ever. This can cause a device to become irresponsive and to
disrupt the guest execution, ie. hang, loop forever or worse.
This scenario is quite easy to encounter with virtio-scsi
on POWER when punching multiple blockdev-create QMP commands
while the guest is booting and it is still running the SLOF
firmware. This happens because SLOF disables/re-enables PCI
devices multiple times via IO/MEM/MASTER bits of PCI_COMMAND
register after the initial probe/feature negotiation, as it
tends to work with a single device at a time at various stages
like probing and running block/network bootloaders without
doing a full reset in-between. This naturally generates many
dataplane stops and starts, and thus many drain sections that
can race with blockdev_create_run(). In the end, SLOF bails
out.
It is somehow reproducible on x86 but it requires to generate
articial dataplane start/stop activity with stop/cont QMP
commands. In this case, seabios ends up looping for ever,
waiting for the virtio-scsi device to send a response to
a command it never received.
Add a helper that pairs all previously called bdrv_do_drained_begin()
with a bdrv_do_drained_end() and call it from bdrv_close().
While at it, update the "/bdrv-drain/graph-change/drain_all"
test in test-bdrv-drain so that it can catch the issue.
BugId: https://bugzilla.redhat.com/show_bug.cgi?id=1874441
Signed-off-by: Greg Kurz
---
v2: - use g_assert() in core code (checkpatch)
- rename helper to bdrv_drain_all_end_quiesce() (Kevin)
---
block.c |9 +
block/io.c | 13 +
include/block/block.h |6 ++
tests/test-bdrv-drain.c |1 +
4 files changed, 29 insertions(+)
diff --git a/block.c b/block.c
index 430edf79bb10..ee5b28a9798d 100644
--- a/block.c
+++ b/block.c
@@ -4458,6 +4458,15 @@ static void bdrv_close(BlockDriverState *bs)
}
QLIST_INIT(&bs->aio_notifiers);
bdrv_drained_end(bs);
+
+/*
+ * If we're still inside some bdrv_drain_all_begin()/end() sections, end
+ * them now since this BDS won't exist anymore when bdrv_drain_all_end()
+ * gets called.
+ */
+if (bs->quiesce_counter) {
+bdrv_drain_all_end_quiesce(bs);
+}
}
void bdrv_close_all(void)
diff --git a/block/io.c b/block/io.c
index 54f0968aee27..c3a345a911e6 100644
--- a/block/io.c
+++ b/block/io.c
@@ -633,6 +633,19 @@ void bdrv_drain_all_begin(void)
}
}
+void bdrv_drain_all_end_quiesce(BlockDriverState *bs)
+{
+int drained_end_counter = 0;
+
+g_assert(bs->quiesce_counter > 0);
+g_assert(!bs->refcnt);
+
+while (bs->quiesce_counter) {
+bdrv_do_drained_end(bs, false, NULL, true, &drained_end_counter);
+}
+BDRV_POLL_WHILE(bs, qatomic_read(&drained_end_counter) > 0);
+}
+
void bdrv_drain_all_end(void)
{
BlockDriverState *bs = NULL;
diff --git a/include/block/block.h b/include/block/block.h
index d16c401cb44e..809987017631 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -779,6 +779,12 @@ void bdrv_drained_end(BlockDriverState *bs);
*/
void bdrv_drained_end_no_poll(BlockDriverState *bs, int *drained_end_counter);
+/**
+ * End all quiescent sections started by bdrv_drain_all_begin(). This is
+ * only needed when deleting a BDS before bdrv_drain_all_end() is called.
+ */
+void bdrv_drain_all_end_quiesce(BlockDriverState *bs);
+
/**
* End a quiescent section started by bdrv_subtree_drained_begin().
*/
diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index 1595bbc92e9e..8a29e33e004a 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -594,6 +594,7 @@ static void test_graph_change_drain_all(void)
g_assert_cmpint(bs_b->quiesce_counter, ==, 0);
g_assert_cmpint(b_s->drain_count, ==, 0);
+g_assert_cmpint(qemu_get_aio_context()->external_disable_cnt, ==, 0);
bdrv_unref(bs_b);
blk_unref(blk_b);
Re: [PATCH v12 07/14] copy-on-read: limit COR operations to bottom node
22.10.2020 21:13, Andrey Shinkevich wrote:
Limit COR operations to the bottom node (inclusively) in the backing
chain when the bottom node name is given. It will be useful for a block
stream job when the COR-filter is applied. The bottom node is passed as
the base itself may change due to concurrent commit jobs on the same
backing chain.
Signed-off-by: Andrey Shinkevich
---
block/copy-on-read.c | 42 --
1 file changed, 40 insertions(+), 2 deletions(-)
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index 3d8e4db..8178a91 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -123,8 +123,46 @@ static int coroutine_fn
cor_co_preadv_part(BlockDriverState *bs,
size_t qiov_offset,
int flags)
{
-return bdrv_co_preadv_part(bs->file, offset, bytes, qiov, qiov_offset,
- flags | BDRV_REQ_COPY_ON_READ);
+int64_t n = 0;
no need to initialize n actually.
+int local_flags;
+int ret;
+BDRVStateCOR *state = bs->opaque;
+
+if (!state->bottom_bs) {
+return bdrv_co_preadv_part(bs->file, offset, bytes, qiov, qiov_offset,
+ flags | BDRV_REQ_COPY_ON_READ);
+}
+
+while (bytes) {
+local_flags = flags;
+
+/* In case of failure, try to copy-on-read anyway */
+ret = bdrv_is_allocated(bs->file->bs, offset, bytes, &n);
+if (!ret || ret < 0) {
simper: if (ret <= 0) {
+ret =
bdrv_is_allocated_above(bdrv_backing_chain_next(bs->file->bs),
+ state->bottom_bs, true, offset,
+ n, &n);
+if (ret == 1 || ret < 0) {
+local_flags |= BDRV_REQ_COPY_ON_READ;
+}
+/* Finish earlier if the end of a backing file has been reached */
+if (ret == 0 && n == 0) {
"n == 0" should be enough
+break;
+}
+}
+
+ret = bdrv_co_preadv_part(bs->file, offset, n, qiov, qiov_offset,
+ local_flags);
+if (ret < 0) {
+return ret;
+}
+
+offset += n;
+qiov_offset += n;
+bytes -= n;
+}
+
+return 0;
}
My remarks are minor, so:
Reviewed-by: Vladimir Sementsov-Ogievskiy
And yes, I still think that this is good to be merged with two previous patches.
--
Best regards,
Vladimir
Re: [PATCH v12 05/14] qapi: create BlockdevOptionsCor structure for COR driver
22.10.2020 21:13, Andrey Shinkevich wrote:
Create the BlockdevOptionsCor structure for COR driver specific options
splitting it off form the BlockdevOptionsGenericFormat. The only option
'bottom' node in the structure denotes an image file that limits the
COR operations in the backing chain.
Suggested-by: Max Reitz
Signed-off-by: Andrey Shinkevich
---
qapi/block-core.json | 21 -
1 file changed, 20 insertions(+), 1 deletion(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0a64306..bf465f6 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -3938,6 +3938,25 @@
'data': { 'throttle-group': 'str',
'file' : 'BlockdevRef'
} }
+
+##
+# @BlockdevOptionsCor:
+#
+# Driver specific block device options for the copy-on-read driver.
+#
+# @bottom: the name of a non-filter node (allocation-bearing layer) that limits
+# the COR operations in the backing chain (inclusive).
+# For the block-stream job, it will be the first non-filter overlay of
+# the base node. We do not involve the base node into the COR
+# operations because the base may change due to a concurrent
+# block-commit job on the same backing chain.
+#
+# Since: 5.2
+##
+{ 'struct': 'BlockdevOptionsCor',
+ 'base': 'BlockdevOptionsGenericFormat',
+ 'data': { '*bottom': 'str' } }
+
##
# @BlockdevOptions:
#
@@ -3990,7 +4009,7 @@
'bochs': 'BlockdevOptionsGenericFormat',
'cloop': 'BlockdevOptionsGenericFormat',
'compress': 'BlockdevOptionsGenericFormat',
- 'copy-on-read':'BlockdevOptionsGenericFormat',
+ 'copy-on-read':'BlockdevOptionsCor',
'dmg':'BlockdevOptionsGenericFormat',
'file': 'BlockdevOptionsFile',
'ftp':'BlockdevOptionsCurlFtp',
Reviewed-by: Vladimir Sementsov-Ogievskiy
Still, I'd prefer this to be merged with further two patches, to not add
non-working interfaces even if this will be fixed two commits further. We do
similar things sometimes to simplify big commits, but in this case merged
03/04/05 doesn't seem too big.
--
Best regards,
Vladimir
Re: [PATCH v12 06/14] copy-on-read: pass bottom node name to COR driver
22.10.2020 21:13, Andrey Shinkevich wrote:
We are going to use the COR-filter for a block-stream job.
To limit COR operations by the base node in the backing chain during
stream job, pass the bottom node name, that is the first non-filter
overlay of the base, to the copy-on-read driver as the base node itself
may change due to possible concurrent jobs.
The rest of the functionality will be implemented in the patch that
follows.
Signed-off-by: Andrey Shinkevich
---
block/copy-on-read.c | 16
1 file changed, 16 insertions(+)
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index 618c4c4..3d8e4db 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -24,18 +24,24 @@
#include "block/block_int.h"
#include "qemu/module.h"
#include "qapi/error.h"
+#include "qapi/qmp/qerror.h"
+#include "qapi/qmp/qdict.h"
#include "block/copy-on-read.h"
typedef struct BDRVStateCOR {
bool active;
+BlockDriverState *bottom_bs;
} BDRVStateCOR;
static int cor_open(BlockDriverState *bs, QDict *options, int flags,
Error **errp)
{
+BlockDriverState *bottom_bs = NULL;
BDRVStateCOR *state = bs->opaque;
+/* Find a bottom node name, if any */
+const char *bottom_node = qdict_get_try_str(options, "bottom");
bs->file = bdrv_open_child(NULL, options, "file", bs, &child_of_bds,
BDRV_CHILD_FILTERED | BDRV_CHILD_PRIMARY,
@@ -51,7 +57,17 @@ static int cor_open(BlockDriverState *bs, QDict *options,
int flags,
((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
bs->file->bs->supported_zero_flags);
+if (bottom_node) {
+bottom_bs = bdrv_lookup_bs(NULL, bottom_node, errp);
+if (!bottom_bs) {
+error_setg(errp, QERR_BASE_NOT_FOUND, bottom_node);
QERR_BASE_NOT_FOUND is unrelated here. Also, I see a comment in qerror.h that
such macros should not be used in new code. And don't forget to drop qerror.h
include line.
+qdict_del(options, "bottom");
this may be moved above "bottom_bs = ..", to not call it after "if" in separate.
+return -EINVAL;
+}
+qdict_del(options, "bottom");
+}
state->active = true;
+state->bottom_bs = bottom_bs;
/*
* We don't need to call bdrv_child_refresh_perms() now as the permissions
--
Best regards,
Vladimir
Re: [PATCH] block: End quiescent sections when a BDS is deleted
On Fri, 23 Oct 2020 16:18:05 +0200
Kevin Wolf wrote:
> Am 23.10.2020 um 12:41 hat Greg Kurz geschrieben:
> > If a BDS gets deleted during blk_drain_all(), it might miss a
> > call to bdrv_do_drained_end(). This means missing a call to
> > aio_enable_external() and the AIO context remains disabled for
> > ever. This can cause a device to become irresponsive and to
> > disrupt the guest execution, ie. hang, loop forever or worse.
> >
> > This scenario is quite easy to encounter with virtio-scsi
> > on POWER when punching multiple blockdev-create QMP commands
> > while the guest is booting and it is still running the SLOF
> > firmware. This happens because SLOF disables/re-enables PCI
> > devices multiple times via IO/MEM/MASTER bits of PCI_COMMAND
> > register after the initial probe/feature negotiation, as it
> > tends to work with a single device at a time at various stages
> > like probing and running block/network bootloaders without
> > doing a full reset in-between. This naturally generates many
> > dataplane stops and starts, and thus many drain sections that
> > can race with blockdev_create_run(). In the end, SLOF bails
> > out.
> >
> > It is somehow reproducible on x86 but it requires to generate
> > articial dataplane start/stop activity with stop/cont QMP
> > commands. In this case, seabios ends up looping for ever,
> > waiting for the virtio-scsi device to send a response to
> > a command it never received.
> >
> > Add a helper that pairs all previously called bdrv_do_drained_begin()
> > with a bdrv_do_drained_end() and call it from bdrv_close().
> > While at it, update the "/bdrv-drain/graph-change/drain_all"
> > test in test-bdrv-drain so that it can catch the issue.
> >
> > BugId: https://bugzilla.redhat.com/show_bug.cgi?id=1874441
> > Signed-off-by: Greg Kurz
> > ---
> > block.c |9 +
> > block/io.c | 13 +
> > include/block/block.h |6 ++
> > tests/test-bdrv-drain.c |1 +
> > 4 files changed, 29 insertions(+)
> >
> > diff --git a/block.c b/block.c
> > index 430edf79bb10..ddcb36dd48a8 100644
> > --- a/block.c
> > +++ b/block.c
> > @@ -4458,6 +4458,15 @@ static void bdrv_close(BlockDriverState *bs)
> > }
> > QLIST_INIT(&bs->aio_notifiers);
> > bdrv_drained_end(bs);
> > +
> > +/*
> > + * If we're still inside some bdrv_drain_all_begin()/end() sections,
> > end
> > + * them now since this BDS won't exist anymore when
> > bdrv_drain_all_end()
> > + * gets called.
> > + */
> > +if (bs->quiesce_counter) {
> > +bdrv_drained_end_quiesce(bs);
> > +}
> > }
> >
> > void bdrv_close_all(void)
> > diff --git a/block/io.c b/block/io.c
> > index 54f0968aee27..8a0da06bbb14 100644
> > --- a/block/io.c
> > +++ b/block/io.c
> > @@ -633,6 +633,19 @@ void bdrv_drain_all_begin(void)
> > }
> > }
> >
> > +void bdrv_drained_end_quiesce(BlockDriverState *bs)
>
> I think the name should make clear that this is meant as a counterpart
> for bdrv_drain_all_begin(), so maybe bdrv_drain_all_end_quiesce()?
>
Sure.
> (The function is not suitable for any other kinds of drain because the
> parameters it passes to bdrv_do_drained_end() are only the same as for
> bdrv_drain_all_begin().)
>
> > +{
> > +int drained_end_counter = 0;
> > +
> > +g_assert_cmpint(bs->quiesce_counter, >, 0);
> > +g_assert_cmpint(bs->refcnt, ==, 0);
>
> By the way, I didn't know about the problem with these macros either.
>
> > +while (bs->quiesce_counter) {
> > +bdrv_do_drained_end(bs, false, NULL, true, &drained_end_counter);
> > +}
> > +BDRV_POLL_WHILE(bs, qatomic_read(&drained_end_counter) > 0);
> > +}
> > +
> > void bdrv_drain_all_end(void)
> > {
> > BlockDriverState *bs = NULL;
> > diff --git a/include/block/block.h b/include/block/block.h
> > index d16c401cb44e..c0ce6e690081 100644
> > --- a/include/block/block.h
> > +++ b/include/block/block.h
> > @@ -779,6 +779,12 @@ void bdrv_drained_end(BlockDriverState *bs);
> > */
> > void bdrv_drained_end_no_poll(BlockDriverState *bs, int
> > *drained_end_counter);
> >
> > +/**
> > + * End all quiescent sections started by bdrv_drain_all_begin(). This is
> > + * only needed when deleting a BDS before bdrv_drain_all_end() is called.
> > + */
> > +void bdrv_drained_end_quiesce(BlockDriverState *bs);
> > +
> > /**
> > * End a quiescent section started by bdrv_subtree_drained_begin().
> > */
> > diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
> > index 1595bbc92e9e..8a29e33e004a 100644
> > --- a/tests/test-bdrv-drain.c
> > +++ b/tests/test-bdrv-drain.c
> > @@ -594,6 +594,7 @@ static void test_graph_change_drain_all(void)
> >
> > g_assert_cmpint(bs_b->quiesce_counter, ==, 0);
> > g_assert_cmpint(b_s->drain_count, ==, 0);
> > +g_assert_cmpint(qemu_get_aio_context()->external_disable_cnt, ==, 0);
>
> But here in the test case we should keep g_assert_cmpint() because it
> gives better er
Re: [PATCH v12 03/14] copy-on-read: add filter drop function
22.10.2020 21:13, Andrey Shinkevich wrote: Provide API for the COR-filter removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. This function may be considered as an intermediate solution before we are able to use bdrv_remove_node(). It will be possible once the QEMU permission update system has overhauled. To insert the filter, the block generic layer function bdrv_insert_node() can be used. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy -- Best regards, Vladimir
Re: [PATCH v12 02/14] block: add insert/remove node functions
23.10.2020 17:24, Vladimir Sementsov-Ogievskiy wrote:
22.10.2020 21:13, Andrey Shinkevich wrote:
Provide API for a node insertion to and removal from a backing chain.
Suggested-by: Max Reitz
Signed-off-by: Andrey Shinkevich
---
block.c | 49 +
include/block/block.h | 3 +++
2 files changed, 52 insertions(+)
diff --git a/block.c b/block.c
index 430edf7..502b483 100644
--- a/block.c
+++ b/block.c
@@ -4670,6 +4670,55 @@ static void bdrv_delete(BlockDriverState *bs)
g_free(bs);
}
+BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
+ int flags, Error **errp)
+{
+ BlockDriverState *new_node_bs;
+ Error *local_err = NULL;
+
+ new_node_bs = bdrv_open(NULL, NULL, node_options, flags, errp);
+ if (new_node_bs == NULL) {
+ error_prepend(errp, "Could not create node: ");
+ return NULL;
+ }
+
+ bdrv_drained_begin(bs);
+ bdrv_replace_node(bs, new_node_bs, &local_err);
+ bdrv_drained_end(bs);
+
+ if (local_err) {
+ bdrv_unref(new_node_bs);
+ error_propagate(errp, local_err);
+ return NULL;
+ }
+
+ return new_node_bs;
+}
+
+void bdrv_remove_node(BlockDriverState *bs)
+{
+ BdrvChild *child;
+ BlockDriverState *inferior_bs;
+
+ child = bdrv_filter_or_cow_child(bs);
+ if (!child) {
+ return;
+ }
+ inferior_bs = child->bs;
+
+ /* Retain the BDS until we complete the graph change. */
+ bdrv_ref(inferior_bs);
+ /* Hold a guest back from writing while permissions are being reset. */
+ bdrv_drained_begin(inferior_bs);
+ /* Refresh permissions before the graph change. */
+ bdrv_child_refresh_perms(bs, child, &error_abort);
+ bdrv_replace_node(bs, inferior_bs, &error_abort);
+
+ bdrv_drained_end(inferior_bs);
+ bdrv_unref(inferior_bs);
+ bdrv_unref(bs);
+}
The function is unused, and as I understand can't work as intended without
s->active handling. I think it should be dropped
with bdrv_remove_node dropped:
Reviewed-by: Vladimir Sementsov-Ogievskiy
+
/*
* Run consistency checks on an image
*
diff --git a/include/block/block.h b/include/block/block.h
index d16c401..ae7612f 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -350,6 +350,9 @@ void bdrv_append(BlockDriverState *bs_new, BlockDriverState
*bs_top,
Error **errp);
void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
Error **errp);
+BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
+ int flags, Error **errp);
+void bdrv_remove_node(BlockDriverState *bs);
int bdrv_parse_aio(const char *mode, int *flags);
int bdrv_parse_cache_mode(const char *mode, int *flags, bool *writethrough);
--
Best regards,
Vladimir
Re: [PATCH v12 02/14] block: add insert/remove node functions
22.10.2020 21:13, Andrey Shinkevich wrote:
Provide API for a node insertion to and removal from a backing chain.
Suggested-by: Max Reitz
Signed-off-by: Andrey Shinkevich
---
block.c | 49 +
include/block/block.h | 3 +++
2 files changed, 52 insertions(+)
diff --git a/block.c b/block.c
index 430edf7..502b483 100644
--- a/block.c
+++ b/block.c
@@ -4670,6 +4670,55 @@ static void bdrv_delete(BlockDriverState *bs)
g_free(bs);
}
+BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
+ int flags, Error **errp)
+{
+BlockDriverState *new_node_bs;
+Error *local_err = NULL;
+
+new_node_bs = bdrv_open(NULL, NULL, node_options, flags, errp);
+if (new_node_bs == NULL) {
+error_prepend(errp, "Could not create node: ");
+return NULL;
+}
+
+bdrv_drained_begin(bs);
+bdrv_replace_node(bs, new_node_bs, &local_err);
+bdrv_drained_end(bs);
+
+if (local_err) {
+bdrv_unref(new_node_bs);
+error_propagate(errp, local_err);
+return NULL;
+}
+
+return new_node_bs;
+}
+
+void bdrv_remove_node(BlockDriverState *bs)
+{
+BdrvChild *child;
+BlockDriverState *inferior_bs;
+
+child = bdrv_filter_or_cow_child(bs);
+if (!child) {
+return;
+}
+inferior_bs = child->bs;
+
+/* Retain the BDS until we complete the graph change. */
+bdrv_ref(inferior_bs);
+/* Hold a guest back from writing while permissions are being reset. */
+bdrv_drained_begin(inferior_bs);
+/* Refresh permissions before the graph change. */
+bdrv_child_refresh_perms(bs, child, &error_abort);
+bdrv_replace_node(bs, inferior_bs, &error_abort);
+
+bdrv_drained_end(inferior_bs);
+bdrv_unref(inferior_bs);
+bdrv_unref(bs);
+}
The function is unused, and as I understand can't work as intended without
s->active handling. I think it should be dropped
+
/*
* Run consistency checks on an image
*
diff --git a/include/block/block.h b/include/block/block.h
index d16c401..ae7612f 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -350,6 +350,9 @@ void bdrv_append(BlockDriverState *bs_new, BlockDriverState
*bs_top,
Error **errp);
void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
Error **errp);
+BlockDriverState *bdrv_insert_node(BlockDriverState *bs, QDict *node_options,
+ int flags, Error **errp);
+void bdrv_remove_node(BlockDriverState *bs);
int bdrv_parse_aio(const char *mode, int *flags);
int bdrv_parse_cache_mode(const char *mode, int *flags, bool *writethrough);
--
Best regards,
Vladimir
Re: [PATCH] block: End quiescent sections when a BDS is deleted
Am 23.10.2020 um 12:41 hat Greg Kurz geschrieben:
> If a BDS gets deleted during blk_drain_all(), it might miss a
> call to bdrv_do_drained_end(). This means missing a call to
> aio_enable_external() and the AIO context remains disabled for
> ever. This can cause a device to become irresponsive and to
> disrupt the guest execution, ie. hang, loop forever or worse.
>
> This scenario is quite easy to encounter with virtio-scsi
> on POWER when punching multiple blockdev-create QMP commands
> while the guest is booting and it is still running the SLOF
> firmware. This happens because SLOF disables/re-enables PCI
> devices multiple times via IO/MEM/MASTER bits of PCI_COMMAND
> register after the initial probe/feature negotiation, as it
> tends to work with a single device at a time at various stages
> like probing and running block/network bootloaders without
> doing a full reset in-between. This naturally generates many
> dataplane stops and starts, and thus many drain sections that
> can race with blockdev_create_run(). In the end, SLOF bails
> out.
>
> It is somehow reproducible on x86 but it requires to generate
> articial dataplane start/stop activity with stop/cont QMP
> commands. In this case, seabios ends up looping for ever,
> waiting for the virtio-scsi device to send a response to
> a command it never received.
>
> Add a helper that pairs all previously called bdrv_do_drained_begin()
> with a bdrv_do_drained_end() and call it from bdrv_close().
> While at it, update the "/bdrv-drain/graph-change/drain_all"
> test in test-bdrv-drain so that it can catch the issue.
>
> BugId: https://bugzilla.redhat.com/show_bug.cgi?id=1874441
> Signed-off-by: Greg Kurz
> ---
> block.c |9 +
> block/io.c | 13 +
> include/block/block.h |6 ++
> tests/test-bdrv-drain.c |1 +
> 4 files changed, 29 insertions(+)
>
> diff --git a/block.c b/block.c
> index 430edf79bb10..ddcb36dd48a8 100644
> --- a/block.c
> +++ b/block.c
> @@ -4458,6 +4458,15 @@ static void bdrv_close(BlockDriverState *bs)
> }
> QLIST_INIT(&bs->aio_notifiers);
> bdrv_drained_end(bs);
> +
> +/*
> + * If we're still inside some bdrv_drain_all_begin()/end() sections, end
> + * them now since this BDS won't exist anymore when bdrv_drain_all_end()
> + * gets called.
> + */
> +if (bs->quiesce_counter) {
> +bdrv_drained_end_quiesce(bs);
> +}
> }
>
> void bdrv_close_all(void)
> diff --git a/block/io.c b/block/io.c
> index 54f0968aee27..8a0da06bbb14 100644
> --- a/block/io.c
> +++ b/block/io.c
> @@ -633,6 +633,19 @@ void bdrv_drain_all_begin(void)
> }
> }
>
> +void bdrv_drained_end_quiesce(BlockDriverState *bs)
I think the name should make clear that this is meant as a counterpart
for bdrv_drain_all_begin(), so maybe bdrv_drain_all_end_quiesce()?
(The function is not suitable for any other kinds of drain because the
parameters it passes to bdrv_do_drained_end() are only the same as for
bdrv_drain_all_begin().)
> +{
> +int drained_end_counter = 0;
> +
> +g_assert_cmpint(bs->quiesce_counter, >, 0);
> +g_assert_cmpint(bs->refcnt, ==, 0);
By the way, I didn't know about the problem with these macros either.
> +while (bs->quiesce_counter) {
> +bdrv_do_drained_end(bs, false, NULL, true, &drained_end_counter);
> +}
> +BDRV_POLL_WHILE(bs, qatomic_read(&drained_end_counter) > 0);
> +}
> +
> void bdrv_drain_all_end(void)
> {
> BlockDriverState *bs = NULL;
> diff --git a/include/block/block.h b/include/block/block.h
> index d16c401cb44e..c0ce6e690081 100644
> --- a/include/block/block.h
> +++ b/include/block/block.h
> @@ -779,6 +779,12 @@ void bdrv_drained_end(BlockDriverState *bs);
> */
> void bdrv_drained_end_no_poll(BlockDriverState *bs, int
> *drained_end_counter);
>
> +/**
> + * End all quiescent sections started by bdrv_drain_all_begin(). This is
> + * only needed when deleting a BDS before bdrv_drain_all_end() is called.
> + */
> +void bdrv_drained_end_quiesce(BlockDriverState *bs);
> +
> /**
> * End a quiescent section started by bdrv_subtree_drained_begin().
> */
> diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
> index 1595bbc92e9e..8a29e33e004a 100644
> --- a/tests/test-bdrv-drain.c
> +++ b/tests/test-bdrv-drain.c
> @@ -594,6 +594,7 @@ static void test_graph_change_drain_all(void)
>
> g_assert_cmpint(bs_b->quiesce_counter, ==, 0);
> g_assert_cmpint(b_s->drain_count, ==, 0);
> +g_assert_cmpint(qemu_get_aio_context()->external_disable_cnt, ==, 0);
But here in the test case we should keep g_assert_cmpint() because it
gives better error messages when it fails (and checkpatch doesn't warn
about it in tests).
Apart from the naming and checkpatch, this looks good to me.
Kevin
Re: [PATCH 3/6] qapi: Remove wrapper struct for simple unions
Am 23.10.2020 um 14:49 hat Markus Armbruster geschrieben:
> In addition to that, I get
>
> /usr/bin/sphinx-build-3 -Dversion=5.1.50 -Drelease=
> -Ddepfile=docs/interop.d -Ddepfile_stamp=docs/interop.stamp -b html -d
> /work/armbru/qemu/bld-x86/docs/interop.p /work/armbru/qemu/docs/interop
> /work/armbru/qemu/bld-x86/docs/interop
> Running Sphinx v2.2.2
> building [mo]: targets for 0 po files that are out of date
> building [html]: targets for 12 source files that are out of date
> updating environment: [new config] 12 added, 0 changed, 0 removed
> reading sources... [ 8%] bitmaps
> reading sources... [ 16%] dbus
> reading sources... [ 25%] dbus-vmstate
> reading sources... [ 33%] index
> reading sources... [ 41%] live-block-operations
> reading sources... [ 50%] pr-helper
> reading sources... [ 58%] qemu-ga
> reading sources... [ 66%] qemu-ga-ref
> reading sources... [ 75%] qemu-qmp-ref
>
> Exception occurred:
> File "/work/armbru/qemu/docs/sphinx/qapidoc.py", line 187, in
> _nodes_for_members
> assert not v.type.base and not v.type.variants
> AttributeError: 'QAPISchemaBuiltinType' object has no attribute 'base'
FWIW, when I install sphinx on F33 (sphinx-build 3.2.1), master doesn't
even build for me, so I uninstalled it again for now...
But the fix is the following, I'll include it in v2.
Kevin
diff --git a/docs/sphinx/qapidoc.py b/docs/sphinx/qapidoc.py
index e03abcbb95..75f6721e29 100644
--- a/docs/sphinx/qapidoc.py
+++ b/docs/sphinx/qapidoc.py
@@ -183,12 +183,14 @@ class QAPISchemaGenRSTVisitor(QAPISchemaVisitor):
if variants:
for v in variants.variants:
-if v.type.is_implicit():
-assert not v.type.base and not v.type.variants
-for m in v.type.local_members:
-term = self._nodes_for_one_member(m)
-term.extend(self._nodes_for_variant_when(variants, v))
-dlnode += self._make_dlitem(term, None)
+if v.wrapped:
+term = [nodes.literal('', 'data'),
+nodes.Text(': '),
+nodes.literal('', v.type.doc_type())]
+term.extend(self._nodes_for_variant_when(variants, v))
+dlnode += self._make_dlitem(term, None)
+elif v.type.is_implicit():
+assert v.type.name == 'q_empty'
else:
term = [nodes.Text('The members of '),
nodes.literal('', v.type.doc_type())]
Re: [PATCH 0/6] qemu-storage-daemon: QAPIfy --chardev
Daniel P. Berrangé writes: > On Fri, Oct 23, 2020 at 12:12:16PM +0200, Kevin Wolf wrote: >> While the qemu-storage-daemon command line is still considered unstable, >> let's change --chardev from the old QemuOpts-based parser to QAPI, so >> that it becomes a simple mapping of chardev-add to the command line and >> will fit in a future fully QAPIfied command line without later >> incompatible changes or additional compatibility glue. > > NB, I am *not* objecting to this series in what I'm about to write, > but I want to point out that I don't think we should assume that > this proposed QAPIified CLI is neccessarily the long term end point. > Below I outline potential incompatible changes that would end up > turning -chardev into legacy / deprecated syntax. > > > The chardev QAPI-ification is a good example of the gap between QOM > and QAPI. > > The chardev backends are QOM classes, but they lack the "user creatable" > interface. > > Thus instead of configuring them using -object / object_add, we have a > QAPI model, custom args/commands -chardev / chardev_add, and then code > that populates the object instance properties from the QAPI model. Except > we've not really exposed them as object instance properties - they are > mostly just struct fields directly accessed. > > > The benefit of the chardev approach is that we have an introspectable > QAPI model. The downside is that we have a bunch of conversion code > for QAPI to QOM mapping. > > The benefit of the user creatable object approach is that we dont > have to add custom CLI args for different types of object, nor write > code to populate QOM from QAPI. The downside is that we're divorced > from the QAPI schema, so loose introspection, and have a different > type of tedious boilerplate code to write. Loss of QAPI introspection is the killer. We have QOM introspection, but it's far too weak to serve as replacement. Besides, two introspection facilities is one too many. > The fact that we have this custom -chardev arg, instead of just using > -object is related to the ordering problems we have in main() in the > system emulators. > > We have some -object types that need creating before the chardev > types, and some -object types that need creating after the chardevs. > > If -chardev didn't exist, and we used -object for chardev creation, > then the ordering problem would trivially not exist, as we would > simply be processing all -object args in order that we receive them. > > > IOW, if we're considering our ideal long term configuration goals > for QEMU, then I think we should be looking at a solution to bridge > the gap, so that we can get the best of both worlds, and have a single > way todo things. > > IMHO, ideally none of -netdev, -chardev, -device, etc would exist > in any "modern" CLI, they would just be considered legacy syntax. > Essentially everything would be created using the same -object arg > and object_add commands. > > > What I think this means is that we need to use QAPI to represent > the properties of an object, and auto-generate the QOM code to > register properties, provider setters/getters, constructors, etc. Use of a QAPI schema rather than code to define user-facing QOM properties provides QAPI introspection for QOM. Having that would be lovely. Bonus: in QOM, documenting types takes effort. Consequently, user-facing QOM is mostly undocumented. In a QAPI schema, *not* documenting stuff takes effort: the QAPI generator insists on documentation. Eduardo (cc'ed) mentioned he'd like to work towards defining properties in a QAPI schema. > For existing user creatable objects this would largely involve > deleting lots of existing code and defining some simple QAPI > schemas. This is probably quite easy, if tedious. > > For the chardev/netdev/etc objects we already have the QAPI schema, > but would need to add code genration parts to turn them into full > user creatable object classes. This is likely harder more involved > work, especially to keep backcompat working well. I sympathize with this long term vision. Nevertheless, we need Kevin's work now to get a decent storage daemon CLI while that's still easy to do. We'll have to promise stability soon, and then changes get much harder.
[RFC PATCH 2/4] tests/acceptance: Allow running Orange Pi test using cached artifacts
Unfortunately the Armbian 19.11.3 image has been removed from the
dl.armbian.com file server. Developers having the artifact cached
can still run the test. Allow them to, until we find a proper
solution to share binaries with the whole community.
This avoids (when file manually added to cache):
BootLinuxConsole.test_arm_orangepi_bionic_19_11: CANCEL: Missing asset
https://dl.armbian.com/orangepipc/archive/Armbian_19.11.3_Orangepipc_bionic_current_5.3.9.7z
(1.06 s)
Reported-by: Bin Meng
Signed-off-by: Philippe Mathieu-Daudé
---
tests/acceptance/boot_linux_console.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tests/acceptance/boot_linux_console.py
b/tests/acceptance/boot_linux_console.py
index 0118ed59156..032ac85978c 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -780,6 +780,8 @@ def test_arm_orangepi_sd(self):
# Wait for VM to shut down gracefully
self.vm.wait()
+@skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+'Test artifacts fetched from unreliable dl.armbian.com')
@skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
@skipUnless(P7ZIP_AVAILABLE, '7z not installed')
def test_arm_orangepi_bionic(self):
--
2.26.2
[RFC PATCH 4/4] tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC
Test U-Boot and Linux on the recent Armbian release 20.08.
Suggested-by: Cleber Rosa
Signed-off-by: Philippe Mathieu-Daudé
---
tests/acceptance/boot_linux_console.py | 25 +
1 file changed, 25 insertions(+)
diff --git a/tests/acceptance/boot_linux_console.py
b/tests/acceptance/boot_linux_console.py
index 803b4512a26..b604cfe7dab 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -829,6 +829,31 @@ def test_arm_orangepi_bionic_19_11(self):
self.do_test_arm_orangepi_uboot_armbian(image_path)
+@skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+'Test artifacts fetched from unreliable apt.armbian.com')
+@skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
+def test_arm_orangepi_bionic_20_08(self):
+"""
+:avocado: tags=arch:arm
+:avocado: tags=machine:orangepi-pc
+:avocado: tags=device:sd
+"""
+
+# This test download a 275 MiB compressed image and expand it
+# to 1036 MiB, but the underlying filesystem is 1552 MiB...
+# As we expand it to 2 GiB we are safe.
+
+image_url = ('https://dl.armbian.com/orangepipc/archive/'
+ 'Armbian_20.08.1_Orangepipc_bionic_current_5.8.5.img.xz')
+image_hash = ('b4d6775f5673486329e45a0586bf06b6'
+ 'dbe792199fd182ac6b9c7bb6c7d3e6dd')
+image_path_xz = self.fetch_asset(image_url, asset_hash=image_hash,
+ algorithm='sha256')
+image_path = archive.extract(image_path_xz, self.workdir)
+image_pow2ceil_expand(image_path)
+
+self.do_test_arm_orangepi_uboot_armbian(image_path)
+
@skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
def test_arm_orangepi_uboot_netbsd9(self):
"""
--
2.26.2
[RFC PATCH 3/4] tests/acceptance: Extract do_test_arm_orangepi_armbian_uboot() method
As we want to reuse the same U-Boot test for multiple
Armbian releases, extract the common part as
do_test_arm_orangepi_armbian_uboot().
Signed-off-by: Philippe Mathieu-Daudé
---
tests/acceptance/boot_linux_console.py | 45 ++
1 file changed, 24 insertions(+), 21 deletions(-)
diff --git a/tests/acceptance/boot_linux_console.py
b/tests/acceptance/boot_linux_console.py
index 032ac85978c..803b4512a26 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -780,27 +780,7 @@ def test_arm_orangepi_sd(self):
# Wait for VM to shut down gracefully
self.vm.wait()
-@skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
-'Test artifacts fetched from unreliable dl.armbian.com')
-@skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
-@skipUnless(P7ZIP_AVAILABLE, '7z not installed')
-def test_arm_orangepi_bionic(self):
-"""
-:avocado: tags=arch:arm
-:avocado: tags=machine:orangepi-pc
-:avocado: tags=device:sd
-"""
-
-# This test download a 196MB compressed image and expand it to 1GB
-image_url = ('https://dl.armbian.com/orangepipc/archive/'
- 'Armbian_19.11.3_Orangepipc_bionic_current_5.3.9.7z')
-image_hash = '196a8ffb72b0123d92cea4a070894813d305c71e'
-image_path_7z = self.fetch_asset(image_url, asset_hash=image_hash)
-image_name = 'Armbian_19.11.3_Orangepipc_bionic_current_5.3.9.img'
-image_path = os.path.join(self.workdir, image_name)
-process.run("7z e -o%s %s" % (self.workdir, image_path_7z))
-image_pow2ceil_expand(image_path)
-
+def do_test_arm_orangepi_uboot_armbian(self, image_path):
self.vm.set_console()
self.vm.add_args('-drive', 'file=' + image_path + ',if=sd,format=raw',
'-nic', 'user',
@@ -826,6 +806,29 @@ def test_arm_orangepi_bionic(self):
'to ')
self.wait_for_console_pattern('Starting Load Kernel Modules...')
+@skipUnless(os.getenv('ARMBIAN_ARTIFACTS_CACHED'),
+'Test artifacts fetched from unreliable apt.armbian.com')
+@skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
+@skipUnless(P7ZIP_AVAILABLE, '7z not installed')
+def test_arm_orangepi_bionic_19_11(self):
+"""
+:avocado: tags=arch:arm
+:avocado: tags=machine:orangepi-pc
+:avocado: tags=device:sd
+"""
+
+# This test download a 196MB compressed image and expand it to 1GB
+image_url = ('https://dl.armbian.com/orangepipc/archive/'
+ 'Armbian_19.11.3_Orangepipc_bionic_current_5.3.9.7z')
+image_hash = '196a8ffb72b0123d92cea4a070894813d305c71e'
+image_path_7z = self.fetch_asset(image_url, asset_hash=image_hash)
+image_name = 'Armbian_19.11.3_Orangepipc_bionic_current_5.3.9.img'
+image_path = os.path.join(self.workdir, image_name)
+process.run("7z e -o%s %s" % (self.workdir, image_path_7z))
+image_pow2ceil_expand(image_path)
+
+self.do_test_arm_orangepi_uboot_armbian(image_path)
+
@skipUnless(os.getenv('AVOCADO_ALLOW_LARGE_STORAGE'), 'storage limited')
def test_arm_orangepi_uboot_netbsd9(self):
"""
--
2.26.2
[RFC PATCH 0/4] tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC
Series meant to help Bin Meng to debug the SD card issue reported by Michael Roth. Philippe Mathieu-Daudé (4): Revert "hw/sd: Fix incorrect populated function switch status data structure" tests/acceptance: Allow running Orange Pi test using cached artifacts tests/acceptance: Extract do_test_arm_orangepi_armbian_uboot() method tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC hw/sd/sd.c | 3 +- tests/acceptance/boot_linux_console.py | 68 +++--- 2 files changed, 50 insertions(+), 21 deletions(-) -- 2.26.2
[RFC PATCH 1/4] Revert "hw/sd: Fix incorrect populated function switch status data structure"
This reverts commit b638627c723a8d0d2bb73489bc6bf9ff09b8d53a.
Currently booting U-Boot on the Orange Pi PC we get:
console: U-Boot SPL 2020.04-armbian (Sep 02 2020 - 10:16:13 +0200)
...
console: Autoboot in 1 seconds, press to stop
console: =>
console: => setenv extraargs 'printk.time=0 console=ttyS0,115200 loglevel=7
nosmp systemd.default_timeout_start_sec=9000
systemd.mask=armbian-zram-config.service systemd.mask=armbian-ramlog.service'
console: => boot
console: unable to select a mode
console: Device 0: unknown device
console: BOOTP broadcast 1
console: DHCP client bound to address 10.0.2.15 (12 ms)
console: *** Warning: no boot file name; using '0A00020F.img'
console: Using ethernet@1c3 device
console: TFTP from server 10.0.2.2; our IP address is 10.0.2.15
console: Filename '0A00020F.img'.
console: Load address: 0x4200
console: Loading: *
console: TFTP error: 'Access violation' (2)
console: Not retrying...
console: missing environment variable: pxeuuid
console: missing environment variable: bootfile
...
With commit b638627c723 reverted:
console: U-Boot SPL 2020.04-armbian (Sep 02 2020 - 10:16:13 +0200)
...
console: Autoboot in 1 seconds, press to stop
console: =>
console: => setenv extraargs 'printk.time=0 console=ttyS0,115200 loglevel=7
nosmp systemd.default_timeout_start_sec=9000
systemd.mask=armbian-zram-config.service systemd.mask=armbian-ramlog.service'
console: => boot
console: switch to partitions #0, OK
console: mmc0 is current device
console: Scanning mmc 0:1...
console: Found U-Boot script /boot/boot.scr
console: 3967 bytes read in 16 ms (241.2 KiB/s)
console: ## Executing script at 4310
console: U-boot loaded from SD
console: Boot script loaded from mmc
console: 153 bytes read in 10 ms (14.6 KiB/s)
console: 11185760 bytes read in 6698 ms (1.6 MiB/s)
console: 7788240 bytes read in 2966 ms (2.5 MiB/s)
console: Found mainline kernel configuration
console: 32121 bytes read in 50 ms (627 KiB/s)
console: 4185 bytes read in 23 ms (176.8 KiB/s)
console: Applying kernel provided DT fixup script (sun8i-h3-fixup.scr)
console: ## Executing script at 4500
console: ## Loading init Ramdisk from Legacy Image at 4330 ...
console: Image Name: uInitrd
console: Image Type: ARM Linux RAMDisk Image (gzip compressed)
console: Data Size:11185696 Bytes = 10.7 MiB
console: Load Address:
console: Entry Point:
console: Verifying Checksum ... OK
console: ## Flattened Device Tree blob at 4300
console: Booting using the fdt blob at 0x4300
console: Loading Ramdisk to 49555000, end 49fffe20 ... OK
console: Loading Device Tree to 494e4000, end 49554fff ... OK
console: Starting kernel ...
console: Uncompressing Linux... done, booting the kernel.
Fixes: b638627c723 ("hw/sd: Fix incorrect populated function switch status data
structure")
Reported-by: Michael Roth
Signed-off-by: Philippe Mathieu-Daudé
---
hw/sd/sd.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/hw/sd/sd.c b/hw/sd/sd.c
index c3febed2434..c17197785bc 100644
--- a/hw/sd/sd.c
+++ b/hw/sd/sd.c
@@ -823,12 +823,11 @@ static void sd_function_switch(SDState *sd, uint32_t arg)
sd->data[11] = 0x43;
sd->data[12] = 0x80; /* Supported group 1 functions */
sd->data[13] = 0x03;
-
for (i = 0; i < 6; i ++) {
new_func = (arg >> (i * 4)) & 0x0f;
if (mode && new_func != 0x0f)
sd->function_group[i] = new_func;
-sd->data[16 - (i >> 1)] |= new_func << ((i % 2) * 4);
+sd->data[14 + (i >> 1)] = new_func << ((i * 4) & 4);
}
memset(&sd->data[17], 0, 47);
stw_be_p(sd->data + 64, sd_crc16(sd->data, 64));
--
2.26.2
Re: [PATCH 3/6] qapi: Remove wrapper struct for simple unions
Marc-André Lureau writes:
> Hi
>
> On Fri, Oct 23, 2020 at 2:14 PM Kevin Wolf wrote:
>
>> Variants of simple unions are always contained in a wrapper object
>> called 'data' that serves no real use. When mapping a QAPI object to the
>> command line, this becomes very visible to users because they have to
>> add one or more useless 'data.' to many option names.
>>
>> As a first step towards avoiding this painful CLI experience, this gets
>> rid of the 'data' indirection internally: The QAPI parser doesn't use a
>> wrapper object as the variant type any more, so the indirection is
>> removed from the generated C types. As a result, the C type looks the
>> same for flat and simple unions now. A large part of this patch is
>> mechanical conversion of C code to remove the 'data' indirection.
>>
>> Conceptually, the important change is that variants can now have flat
>> and wrapped representations. For a transitioning period, we'll allow
>> variants to support both representations in a later patch. Eventually,
>> the plan is to deprecate and remove wrapped representations entirely,
>> unifying simple and flat unions.
>>
>> The externally visible interfaces stay unchanged: Visitors still expect
>> the 'data' wrappers, and introspection still shows it.
>>
>> Signed-off-by: Kevin Wolf
>>
>
> breaks the build for me:
>
> ../chardev/spice.c: In function ‘qemu_chr_open_spice_vmc’:
> ../chardev/spice.c:261:56: error: ‘ChardevSpiceChannel’ has no member named
> ‘data’
> 261 | ChardevSpiceChannel *spicevmc = backend->u.spicevmc.data;
> |^
> ../chardev/spice.c: In function ‘qemu_chr_open_spice_port’:
> ../chardev/spice.c:297:55: error: ‘ChardevSpicePort’ has no member named
> ‘data’
> 297 | ChardevSpicePort *spiceport = backend->u.spiceport.data;
> | ^
> ../chardev/spice.c: In function ‘qemu_chr_parse_spice_vmc’:
> ../chardev/spice.c:331:35: error: ‘ChardevSpiceChannel’ has no member named
> ‘data’
> 331 | spicevmc = backend->u.spicevmc.data =
> g_new0(ChardevSpiceChannel, 1);
> | ^
> ../chardev/spice.c: In function ‘qemu_chr_parse_spice_port’:
> ../chardev/spice.c:347:37: error: ‘ChardevSpicePort’ has no member named
> ‘data’
> 347 | spiceport = backend->u.spiceport.data =
> g_new0(ChardevSpicePort, 1);
> | ^
> ../ui/spice-app.c: In function ‘chr_spice_backend_new’:
> ../ui/spice-app.c:63:20: error: ‘ChardevSpicePort’ has no member named
> ‘data’
>63 | be->u.spiceport.data = g_new0(ChardevSpicePort, 1);
> |^
> ../ui/spice-app.c: In function ‘vc_chr_open’:
> ../ui/spice-app.c:86:20: error: ‘ChardevSpicePort’ has no member named
> ‘data’
>86 | be->u.spiceport.data->fqdn = fqdn ?
> |^
> ../ui/spice-app.c: In function ‘spice_app_display_init’:
> ../ui/spice-app.c:185:20: error: ‘ChardevSpicePort’ has no member named
> ‘data’
> 185 | be->u.spiceport.data->fqdn = g_strdup("org.qemu.monitor.qmp.0");
> |^
In addition to that, I get
/usr/bin/sphinx-build-3 -Dversion=5.1.50 -Drelease=
-Ddepfile=docs/interop.d -Ddepfile_stamp=docs/interop.stamp -b html -d
/work/armbru/qemu/bld-x86/docs/interop.p /work/armbru/qemu/docs/interop
/work/armbru/qemu/bld-x86/docs/interop
Running Sphinx v2.2.2
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 12 source files that are out of date
updating environment: [new config] 12 added, 0 changed, 0 removed
reading sources... [ 8%] bitmaps
reading sources... [ 16%] dbus
reading sources... [ 25%] dbus-vmstate
reading sources... [ 33%] index
reading sources... [ 41%] live-block-operations
reading sources... [ 50%] pr-helper
reading sources... [ 58%] qemu-ga
reading sources... [ 66%] qemu-ga-ref
reading sources... [ 75%] qemu-qmp-ref
Exception occurred:
File "/work/armbru/qemu/docs/sphinx/qapidoc.py", line 187, in
_nodes_for_members
assert not v.type.base and not v.type.variants
AttributeError: 'QAPISchemaBuiltinType' object has no attribute 'base'
The full traceback has been saved in /tmp/sphinx-err-fq8tq1i7.log, if you
want to report the issue to the developers.
Please also report this if it was a user error, so that a better error
message can be provided next time.
A bug report can be filed in the tracker at
. Thanks!
Re: [PATCH 3/6] qapi: Remove wrapper struct for simple unions
Am 23.10.2020 um 13:06 hat Marc-André Lureau geschrieben:
> On Fri, Oct 23, 2020 at 2:40 PM Marc-André Lureau
> wrote:
> > On Fri, Oct 23, 2020 at 2:14 PM Kevin Wolf wrote:
> >
> >> Variants of simple unions are always contained in a wrapper object
> >> called 'data' that serves no real use. When mapping a QAPI object to the
> >> command line, this becomes very visible to users because they have to
> >> add one or more useless 'data.' to many option names.
> >>
> >> As a first step towards avoiding this painful CLI experience, this gets
> >> rid of the 'data' indirection internally: The QAPI parser doesn't use a
> >> wrapper object as the variant type any more, so the indirection is
> >> removed from the generated C types. As a result, the C type looks the
> >> same for flat and simple unions now. A large part of this patch is
> >> mechanical conversion of C code to remove the 'data' indirection.
> >>
> >> Conceptually, the important change is that variants can now have flat
> >> and wrapped representations. For a transitioning period, we'll allow
> >> variants to support both representations in a later patch. Eventually,
> >> the plan is to deprecate and remove wrapped representations entirely,
> >> unifying simple and flat unions.
> >>
> >> The externally visible interfaces stay unchanged: Visitors still expect
> >> the 'data' wrappers, and introspection still shows it.
> >>
> >> Signed-off-by: Kevin Wolf
> >
> > breaks the build for me:
Oops, apparently I'm not compiling spice code. I've installed the
libraries now and will fix this for v2 (it's just the same mechanical
conversion as elsewhere).
See below for the changes I'm squashing in.
> Other than that, I like the change, and it looks quite
> straightforward. I hope it's acceptable and we are not missing the
> reasons that extra data indirection was necessary.
>
> Having to support both flat and wrapped versions in the externally visible
> interfaces might be tricky though.
It's actually pretty simple, I have some quickly hacked up draft patches
because Markus wanted to see what the path is for eventually getting rid
of the wrapped versions (and with that, the difference between simple
and flat unions).
Essentially what I'm doing is calling visit_optional("data") and if it's
there I do the visit_start/end_struct thing for wrapped representation,
and if not, it's flat.
This would be problematic if we had a simple union with a variant that
contains an option "data" (because then it would be ambiguous), but in
this case we're lucky for a change.
After that, it's setting the "deprecated" feature on the "data" member,
waiting for two releases and eventually removing the support for wrapped
representation.
I'm expecting the first part to happen in the 6.0 release cycle, and the
removal could be done in 6.2 then.
Kevin
diff --git a/chardev/spice.c b/chardev/spice.c
index 1104426e3a..8d8502dca7 100644
--- a/chardev/spice.c
+++ b/chardev/spice.c
@@ -258,7 +258,7 @@ static void qemu_chr_open_spice_vmc(Chardev *chr,
bool *be_opened,
Error **errp)
{
-ChardevSpiceChannel *spicevmc = backend->u.spicevmc.data;
+ChardevSpiceChannel *spicevmc = &backend->u.spicevmc;
const char *type = spicevmc->type;
const char **psubtype = spice_server_char_device_recognized_subtypes();
@@ -294,7 +294,7 @@ static void qemu_chr_open_spice_port(Chardev *chr,
bool *be_opened,
Error **errp)
{
-ChardevSpicePort *spiceport = backend->u.spiceport.data;
+ChardevSpicePort *spiceport = &backend->u.spiceport;
const char *name = spiceport->fqdn;
SpiceChardev *s;
@@ -321,14 +321,13 @@ static void qemu_chr_parse_spice_vmc(QemuOpts *opts,
ChardevBackend *backend,
Error **errp)
{
const char *name = qemu_opt_get(opts, "name");
-ChardevSpiceChannel *spicevmc;
+ChardevSpiceChannel *spicevmc = &backend->u.spicevmc;
if (name == NULL) {
error_setg(errp, "chardev: spice channel: no name given");
return;
}
backend->type = CHARDEV_BACKEND_KIND_SPICEVMC;
-spicevmc = backend->u.spicevmc.data = g_new0(ChardevSpiceChannel, 1);
qemu_chr_parse_common(opts, qapi_ChardevSpiceChannel_base(spicevmc));
spicevmc->type = g_strdup(name);
}
@@ -337,14 +336,13 @@ static void qemu_chr_parse_spice_port(QemuOpts *opts,
ChardevBackend *backend,
Error **errp)
{
const char *name = qemu_opt_get(opts, "name");
-ChardevSpicePort *spiceport;
+ChardevSpicePort *spiceport = &backend->u.spiceport;
if (name == NULL) {
error_setg(errp, "chardev: spice port: no name given");
return;
}
backend->type = CHARDEV_BACKEND_KIND_SPICEPORT;
-spiceport = backend->u.spiceport.data = g_new0(ChardevSpicePort, 1);
qemu_chr_parse_common(opts, qapi_Char
Re: [PATCH 2/6] char: Factor out qemu_chr_print_types()
Marc-André Lureau writes:
> On Fri, Oct 23, 2020 at 2:14 PM Kevin Wolf wrote:
>
>> We'll want to call the same from a non-QemuOpts code path.
>>
>> Signed-off-by: Kevin Wolf
>> ---
>> include/chardev/char.h | 1 +
>> chardev/char.c | 17 +++--
>> 2 files changed, 12 insertions(+), 6 deletions(-)
>>
>> diff --git a/include/chardev/char.h b/include/chardev/char.h
>> index db42f0a8c6..3b91645081 100644
>> --- a/include/chardev/char.h
>> +++ b/include/chardev/char.h
>> @@ -212,6 +212,7 @@ void qemu_chr_be_update_read_handlers(Chardev *s,
>> */
>> void qemu_chr_be_event(Chardev *s, QEMUChrEvent event);
>>
>> +void qemu_chr_print_types(void);
>> int qemu_chr_add_client(Chardev *s, int fd);
>> Chardev *qemu_chr_find(const char *name);
>>
>> diff --git a/chardev/char.c b/chardev/char.c
>> index 78553125d3..028612c333 100644
>> --- a/chardev/char.c
>> +++ b/chardev/char.c
>> @@ -633,6 +633,16 @@ ChardevBackend *qemu_chr_parse_opts(QemuOpts *opts,
>> Error **errp)
>> return backend;
>> }
>>
>> +void qemu_chr_print_types(void)
>> +{
>> +GString *str = g_string_new("");
>>
>
> Suggest g_auto here
g_autoptr(GList) str = ..., I presume.
> +
>> +chardev_name_foreach(help_string_append, str);
>> +
>> +qemu_printf("Available chardev backend types: %s\n", str->str);
>> +g_string_free(str, true);
>> +}
>> +
>> Chardev *qemu_chr_new_from_opts(QemuOpts *opts, GMainContext *context,
>> Error **errp)
>> {
>> @@ -644,12 +654,7 @@ Chardev *qemu_chr_new_from_opts(QemuOpts *opts,
>> GMainContext *context,
>> char *bid = NULL;
>>
>> if (name && is_help_option(name)) {
>> -GString *str = g_string_new("");
>> -
>> -chardev_name_foreach(help_string_append, str);
>> -
>> -qemu_printf("Available chardev backend types: %s\n", str->str);
>> -g_string_free(str, true);
>> +qemu_chr_print_types();
>> return NULL;
>> }
>>
>> --
>> 2.28.0
>>
>>
>>
> anyway
> Reviewed-by: Marc-André Lureau
Reviewed-by: Markus Armbruster
Re: [PATCH 1/6] char/stdio: Fix QMP default for 'signal'
Kevin Wolf writes:
> Commit 02c4bdf1 tried to make signal=on the default for stdio chardevs
> except for '-serial mon:stdio', but it forgot about QMP and accidentally
> switched the QMP default from true (except for -nographic) to false
> (always). The documentation was kept unchanged and still describes the
The QMP documentation, actually. CLI documentation was updated.
> opposite of the old behaviour (which is an even older documentation
> bug).
Hardly surprising, given how vague the documentation was (and is). See
below.
> Fix all of this by making signal=true the default in ChardevStdio and
> documenting it as such.
>
> Fixes: 02c4bdf1d2ca8c02a9bae16398f260b5c08d08bf
> Signed-off-by: Kevin Wolf
> ---
> qapi/char.json | 3 +--
> chardev/char-stdio.c | 4 +---
> 2 files changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/qapi/char.json b/qapi/char.json
> index b4d66ec90b..43486d1daa 100644
> --- a/qapi/char.json
> +++ b/qapi/char.json
> @@ -321,8 +321,7 @@
> # Configuration info for stdio chardevs.
> #
> # @signal: Allow signals (such as SIGINT triggered by ^C)
> -# be delivered to qemu. Default: true in -nographic mode,
> -# false otherwise.
> +# be delivered to qemu. Default: true.
> #
> # Since: 1.5
> ##
What does it mean to "allow signals to be delivered"?
To understand, I had to check what the QEMU code does with it (@signal
off clears termios c_lflag ISIG), and I had to look up what that means
(tcsetattr(3): "When any of the characters INTR, QUIT, SUSP, or DSUSP
are received, generate the corresponding signal").
Stealing that sentence verbatim from tcsetattr(3) would be a marked
improvement over what we have, in my opinion.
Nothing wrong with this patch, of course, but if you need to respin for
some other reason, consider improving the comment.
> diff --git a/chardev/char-stdio.c b/chardev/char-stdio.c
> index 82eaebc1db..403da308c9 100644
> --- a/chardev/char-stdio.c
> +++ b/chardev/char-stdio.c
> @@ -112,9 +112,7 @@ static void qemu_chr_open_stdio(Chardev *chr,
>
> qemu_chr_open_fd(chr, 0, 1);
>
> -if (opts->has_signal) {
> -stdio_allow_signal = opts->signal;
> -}
> +stdio_allow_signal = !opts->has_signal || opts->signal;
> qemu_chr_set_echo_stdio(chr, false);
> }
> #endif
Preferably with s/documentation/QMP documentation/ in the commit
message:
Reviewed-by: Markus Armbruster
Re: [PATCH 0/6] qemu-storage-daemon: QAPIfy --chardev
Am 23.10.2020 um 12:36 hat Daniel P. Berrangé geschrieben: > On Fri, Oct 23, 2020 at 12:12:16PM +0200, Kevin Wolf wrote: > > While the qemu-storage-daemon command line is still considered unstable, > > let's change --chardev from the old QemuOpts-based parser to QAPI, so > > that it becomes a simple mapping of chardev-add to the command line and > > will fit in a future fully QAPIfied command line without later > > incompatible changes or additional compatibility glue. > > NB, I am *not* objecting to this series in what I'm about to write, > but I want to point out that I don't think we should assume that > this proposed QAPIified CLI is neccessarily the long term end point. > Below I outline potential incompatible changes that would end up > turning -chardev into legacy / deprecated syntax. Well, when do we have an end point for anything anyway? Things will always evolve. > The chardev QAPI-ification is a good example of the gap between QOM > and QAPI. > > The chardev backends are QOM classes, but they lack the "user > creatable" interface. They aren't really proper QOM classes at this point. Technically they are, but they are pretty boring QOM classes that don't have any properties. Instead, they use ChardevBackend QAPI objects in the custom ChardevClass.open method. So they are in some weird in-between state. > IOW, if we're considering our ideal long term configuration goals > for QEMU, then I think we should be looking at a solution to bridge > the gap, so that we can get the best of both worlds, and have a single > way todo things. > > IMHO, ideally none of -netdev, -chardev, -device, etc would exist > in any "modern" CLI, they would just be considered legacy syntax. > Essentially everything would be created using the same -object arg > and object_add commands. I think we need to distinguish legacy options and convenience options (and this applies much more broadly than just here, and it's probably more important in other places). Even if we eventually make -object capable of configuring chardev and netdev backends, -chardev could still be more convenient to use for a human user. This equation of "legacy" and "convenience" is also on of the problems I have with the suggestion of having two binaries for "legacy" and "modern" interfaces. Humans are not legacy, and having a command line that is unfriendly for human users doesn't make it modern. We have a lot of legacy stuff in the command line, and the goal for it is to get rid of it and replace it with something modern (the goal is _not_ keeping it around for human users because compatibility with this legacy stuff often blocks us from moving forward). But convenience options for human users aren't bad at all, as long as they are based on a modern model of how things work. We'll always want to have some. Sorry for going on a tangent... Back to chardev: In the end, the preferred command line interface doesn't matter that much. The required structure will be the same. > What I think this means is that we need to use QAPI to represent > the properties of an object, and auto-generate the QOM code to > register properties, provider setters/getters, constructors, etc. Yes, I agree with this one. I expect we'll discuss this more when I send the first step for object-add (which is QMP, so for everything, not just qemu-storage-daemon), which will duplicate the QOM properties in the QAPI schema for user creatable objects. Generating QOM code from the schema should be the next step then. > For existing user creatable objects this would largely involve > deleting lots of existing code and defining some simple QAPI > schemas. This is probably quite easy, if tedious. > > For the chardev/netdev/etc objects we already have the QAPI schema, > but would need to add code genration parts to turn them into full > user creatable object classes. This is likely harder more involved > work, especially to keep backcompat working well. The challenging part is probably not keeping compatibility (there's no reason to break it if it represents everything well), but that these are union types in the QAPI schema that map to different QOM classes. It's not entirely obvious to me yet how we would deal with this, but the good thing is that this is all about internal interfaces which we can change relatively freely. Kevin
Re: [PATCH 0/6] qemu-storage-daemon: QAPIfy --chardev
On 23/10/20 12:36, Daniel P. Berrangé wrote: > IOW, if we're considering our ideal long term configuration goals > for QEMU, then I think we should be looking at a solution to bridge > the gap, so that we can get the best of both worlds, and have a single > way todo things. > > IMHO, ideally none of -netdev, -chardev, -device, etc would exist > in any "modern" CLI, they would just be considered legacy syntax. > Essentially everything would be created using the same -object arg > and object_add commands. I agree, especially for -chardev which should be the easiest of the three to make user-creatable. Devices have magic bus properties and netdevs aren't QOM objects at all, but character devices have neither of these issues. On the other hand, chardevs do not yet have properties, which means the transition to -object is not going to be trivial. But it is surely at least worth trying to see what the issues it introduces (perhaps QOM properties with struct and union types?), how they could be solved and whether it's worth it. Thanks, Paolo
Re: [PATCH 3/6] qapi: Remove wrapper struct for simple unions
Hi On Fri, Oct 23, 2020 at 2:40 PM Marc-André Lureau < marcandre.lur...@gmail.com> wrote: > Hi > > On Fri, Oct 23, 2020 at 2:14 PM Kevin Wolf wrote: > >> Variants of simple unions are always contained in a wrapper object >> called 'data' that serves no real use. When mapping a QAPI object to the >> command line, this becomes very visible to users because they have to >> add one or more useless 'data.' to many option names. >> >> As a first step towards avoiding this painful CLI experience, this gets >> rid of the 'data' indirection internally: The QAPI parser doesn't use a >> wrapper object as the variant type any more, so the indirection is >> removed from the generated C types. As a result, the C type looks the >> same for flat and simple unions now. A large part of this patch is >> mechanical conversion of C code to remove the 'data' indirection. >> >> Conceptually, the important change is that variants can now have flat >> and wrapped representations. For a transitioning period, we'll allow >> variants to support both representations in a later patch. Eventually, >> the plan is to deprecate and remove wrapped representations entirely, >> unifying simple and flat unions. >> >> The externally visible interfaces stay unchanged: Visitors still expect >> the 'data' wrappers, and introspection still shows it. >> >> Signed-off-by: Kevin Wolf >> > > breaks the build for me: > > Other than that, I like the change, and it looks quite straightforward. I hope it's acceptable and we are not missing the reasons that extra data indirection was necessary. Having to support both flat and wrapped versions in the externally visible interfaces might be tricky though. -- Marc-André Lureau
Re: [PATCH] block: End quiescent sections when a BDS is deleted
On Fri, 23 Oct 2020 03:48:39 -0700 wrote: > Patchew URL: > https://patchew.org/QEMU/160344969243.4091343.14371338409686732879.st...@bahia.lan/ > > > > Hi, > > This series seems to have some coding style problems. See output below for > more information: > > Type: series > Message-id: 160344969243.4091343.14371338409686732879.st...@bahia.lan > Subject: [PATCH] block: End quiescent sections when a BDS is deleted > > === TEST SCRIPT BEGIN === > #!/bin/bash > git rev-parse base > /dev/null || exit 0 > git config --local diff.renamelimit 0 > git config --local diff.renames True > git config --local diff.algorithm histogram > ./scripts/checkpatch.pl --mailback base.. > === TEST SCRIPT END === > > Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 > From https://github.com/patchew-project/qemu > * [new tag] > patchew/160344969243.4091343.14371338409686732879.st...@bahia.lan -> > patchew/160344969243.4091343.14371338409686732879.st...@bahia.lan > - [tag update] patchew/20201023101222.250147-1-kw...@redhat.com -> > patchew/20201023101222.250147-1-kw...@redhat.com > Switched to a new branch 'test' > f9501f8 block: End quiescent sections when a BDS is deleted > > === OUTPUT BEGIN === > ERROR: Use g_assert or g_assert_not_reached > #73: FILE: block/io.c:640: > +g_assert_cmpint(bs->quiesce_counter, >, 0); > Ah... sorry I wasn't aware of the story behind glib commit a6a875068779, I'll post a v2 what uses g_assert() instead. > ERROR: Use g_assert or g_assert_not_reached > #74: FILE: block/io.c:641: > +g_assert_cmpint(bs->refcnt, ==, 0); > > total: 2 errors, 0 warnings, 53 lines checked > > Commit f9501f846de1 (block: End quiescent sections when a BDS is deleted) has > style problems, please review. If any of these errors > are false positives report them to the maintainer, see > CHECKPATCH in MAINTAINERS. > === OUTPUT END === > > Test command exited with code: 1 > > > The full log is available at > http://patchew.org/logs/160344969243.4091343.14371338409686732879.st...@bahia.lan/testing.checkpatch/?type=message. > --- > Email generated automatically by Patchew [https://patchew.org/]. > Please send your feedback to patchew-de...@redhat.com
Re: [PATCH] block: End quiescent sections when a BDS is deleted
Patchew URL: https://patchew.org/QEMU/160344969243.4091343.14371338409686732879.st...@bahia.lan/ Hi, This series seems to have some coding style problems. See output below for more information: Type: series Message-id: 160344969243.4091343.14371338409686732879.st...@bahia.lan Subject: [PATCH] block: End quiescent sections when a BDS is deleted === TEST SCRIPT BEGIN === #!/bin/bash git rev-parse base > /dev/null || exit 0 git config --local diff.renamelimit 0 git config --local diff.renames True git config --local diff.algorithm histogram ./scripts/checkpatch.pl --mailback base.. === TEST SCRIPT END === Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384 From https://github.com/patchew-project/qemu * [new tag] patchew/160344969243.4091343.14371338409686732879.st...@bahia.lan -> patchew/160344969243.4091343.14371338409686732879.st...@bahia.lan - [tag update] patchew/20201023101222.250147-1-kw...@redhat.com -> patchew/20201023101222.250147-1-kw...@redhat.com Switched to a new branch 'test' f9501f8 block: End quiescent sections when a BDS is deleted === OUTPUT BEGIN === ERROR: Use g_assert or g_assert_not_reached #73: FILE: block/io.c:640: +g_assert_cmpint(bs->quiesce_counter, >, 0); ERROR: Use g_assert or g_assert_not_reached #74: FILE: block/io.c:641: +g_assert_cmpint(bs->refcnt, ==, 0); total: 2 errors, 0 warnings, 53 lines checked Commit f9501f846de1 (block: End quiescent sections when a BDS is deleted) has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS. === OUTPUT END === Test command exited with code: 1 The full log is available at http://patchew.org/logs/160344969243.4091343.14371338409686732879.st...@bahia.lan/testing.checkpatch/?type=message. --- Email generated automatically by Patchew [https://patchew.org/]. Please send your feedback to patchew-de...@redhat.com
[PATCH] block: End quiescent sections when a BDS is deleted
If a BDS gets deleted during blk_drain_all(), it might miss a
call to bdrv_do_drained_end(). This means missing a call to
aio_enable_external() and the AIO context remains disabled for
ever. This can cause a device to become irresponsive and to
disrupt the guest execution, ie. hang, loop forever or worse.
This scenario is quite easy to encounter with virtio-scsi
on POWER when punching multiple blockdev-create QMP commands
while the guest is booting and it is still running the SLOF
firmware. This happens because SLOF disables/re-enables PCI
devices multiple times via IO/MEM/MASTER bits of PCI_COMMAND
register after the initial probe/feature negotiation, as it
tends to work with a single device at a time at various stages
like probing and running block/network bootloaders without
doing a full reset in-between. This naturally generates many
dataplane stops and starts, and thus many drain sections that
can race with blockdev_create_run(). In the end, SLOF bails
out.
It is somehow reproducible on x86 but it requires to generate
articial dataplane start/stop activity with stop/cont QMP
commands. In this case, seabios ends up looping for ever,
waiting for the virtio-scsi device to send a response to
a command it never received.
Add a helper that pairs all previously called bdrv_do_drained_begin()
with a bdrv_do_drained_end() and call it from bdrv_close().
While at it, update the "/bdrv-drain/graph-change/drain_all"
test in test-bdrv-drain so that it can catch the issue.
BugId: https://bugzilla.redhat.com/show_bug.cgi?id=1874441
Signed-off-by: Greg Kurz
---
block.c |9 +
block/io.c | 13 +
include/block/block.h |6 ++
tests/test-bdrv-drain.c |1 +
4 files changed, 29 insertions(+)
diff --git a/block.c b/block.c
index 430edf79bb10..ddcb36dd48a8 100644
--- a/block.c
+++ b/block.c
@@ -4458,6 +4458,15 @@ static void bdrv_close(BlockDriverState *bs)
}
QLIST_INIT(&bs->aio_notifiers);
bdrv_drained_end(bs);
+
+/*
+ * If we're still inside some bdrv_drain_all_begin()/end() sections, end
+ * them now since this BDS won't exist anymore when bdrv_drain_all_end()
+ * gets called.
+ */
+if (bs->quiesce_counter) {
+bdrv_drained_end_quiesce(bs);
+}
}
void bdrv_close_all(void)
diff --git a/block/io.c b/block/io.c
index 54f0968aee27..8a0da06bbb14 100644
--- a/block/io.c
+++ b/block/io.c
@@ -633,6 +633,19 @@ void bdrv_drain_all_begin(void)
}
}
+void bdrv_drained_end_quiesce(BlockDriverState *bs)
+{
+int drained_end_counter = 0;
+
+g_assert_cmpint(bs->quiesce_counter, >, 0);
+g_assert_cmpint(bs->refcnt, ==, 0);
+
+while (bs->quiesce_counter) {
+bdrv_do_drained_end(bs, false, NULL, true, &drained_end_counter);
+}
+BDRV_POLL_WHILE(bs, qatomic_read(&drained_end_counter) > 0);
+}
+
void bdrv_drain_all_end(void)
{
BlockDriverState *bs = NULL;
diff --git a/include/block/block.h b/include/block/block.h
index d16c401cb44e..c0ce6e690081 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -779,6 +779,12 @@ void bdrv_drained_end(BlockDriverState *bs);
*/
void bdrv_drained_end_no_poll(BlockDriverState *bs, int *drained_end_counter);
+/**
+ * End all quiescent sections started by bdrv_drain_all_begin(). This is
+ * only needed when deleting a BDS before bdrv_drain_all_end() is called.
+ */
+void bdrv_drained_end_quiesce(BlockDriverState *bs);
+
/**
* End a quiescent section started by bdrv_subtree_drained_begin().
*/
diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index 1595bbc92e9e..8a29e33e004a 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -594,6 +594,7 @@ static void test_graph_change_drain_all(void)
g_assert_cmpint(bs_b->quiesce_counter, ==, 0);
g_assert_cmpint(b_s->drain_count, ==, 0);
+g_assert_cmpint(qemu_get_aio_context()->external_disable_cnt, ==, 0);
bdrv_unref(bs_b);
blk_unref(blk_b);
Re: [PATCH 2/6] char: Factor out qemu_chr_print_types()
On Fri, Oct 23, 2020 at 2:14 PM Kevin Wolf wrote:
> We'll want to call the same from a non-QemuOpts code path.
>
> Signed-off-by: Kevin Wolf
> ---
> include/chardev/char.h | 1 +
> chardev/char.c | 17 +++--
> 2 files changed, 12 insertions(+), 6 deletions(-)
>
> diff --git a/include/chardev/char.h b/include/chardev/char.h
> index db42f0a8c6..3b91645081 100644
> --- a/include/chardev/char.h
> +++ b/include/chardev/char.h
> @@ -212,6 +212,7 @@ void qemu_chr_be_update_read_handlers(Chardev *s,
> */
> void qemu_chr_be_event(Chardev *s, QEMUChrEvent event);
>
> +void qemu_chr_print_types(void);
> int qemu_chr_add_client(Chardev *s, int fd);
> Chardev *qemu_chr_find(const char *name);
>
> diff --git a/chardev/char.c b/chardev/char.c
> index 78553125d3..028612c333 100644
> --- a/chardev/char.c
> +++ b/chardev/char.c
> @@ -633,6 +633,16 @@ ChardevBackend *qemu_chr_parse_opts(QemuOpts *opts,
> Error **errp)
> return backend;
> }
>
> +void qemu_chr_print_types(void)
> +{
> +GString *str = g_string_new("");
>
Suggest g_auto here
+
> +chardev_name_foreach(help_string_append, str);
> +
> +qemu_printf("Available chardev backend types: %s\n", str->str);
> +g_string_free(str, true);
> +}
> +
> Chardev *qemu_chr_new_from_opts(QemuOpts *opts, GMainContext *context,
> Error **errp)
> {
> @@ -644,12 +654,7 @@ Chardev *qemu_chr_new_from_opts(QemuOpts *opts,
> GMainContext *context,
> char *bid = NULL;
>
> if (name && is_help_option(name)) {
> -GString *str = g_string_new("");
> -
> -chardev_name_foreach(help_string_append, str);
> -
> -qemu_printf("Available chardev backend types: %s\n", str->str);
> -g_string_free(str, true);
> +qemu_chr_print_types();
> return NULL;
> }
>
> --
> 2.28.0
>
>
>
anyway
Reviewed-by: Marc-André Lureau
--
Marc-André Lureau
Re: [PATCH 1/6] char/stdio: Fix QMP default for 'signal'
On Fri, Oct 23, 2020 at 2:14 PM Kevin Wolf wrote:
> Commit 02c4bdf1 tried to make signal=on the default for stdio chardevs
> except for '-serial mon:stdio', but it forgot about QMP and accidentally
> switched the QMP default from true (except for -nographic) to false
> (always). The documentation was kept unchanged and still describes the
> opposite of the old behaviour (which is an even older documentation
> bug).
>
> Fix all of this by making signal=true the default in ChardevStdio and
> documenting it as such.
>
> Fixes: 02c4bdf1d2ca8c02a9bae16398f260b5c08d08bf
> Signed-off-by: Kevin Wolf
>
Reviewed-by: Marc-André Lureau
---
> qapi/char.json | 3 +--
> chardev/char-stdio.c | 4 +---
> 2 files changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/qapi/char.json b/qapi/char.json
> index b4d66ec90b..43486d1daa 100644
> --- a/qapi/char.json
> +++ b/qapi/char.json
> @@ -321,8 +321,7 @@
> # Configuration info for stdio chardevs.
> #
> # @signal: Allow signals (such as SIGINT triggered by ^C)
> -# be delivered to qemu. Default: true in -nographic mode,
> -# false otherwise.
> +# be delivered to qemu. Default: true.
> #
> # Since: 1.5
> ##
> diff --git a/chardev/char-stdio.c b/chardev/char-stdio.c
> index 82eaebc1db..403da308c9 100644
> --- a/chardev/char-stdio.c
> +++ b/chardev/char-stdio.c
> @@ -112,9 +112,7 @@ static void qemu_chr_open_stdio(Chardev *chr,
>
> qemu_chr_open_fd(chr, 0, 1);
>
> -if (opts->has_signal) {
> -stdio_allow_signal = opts->signal;
> -}
> +stdio_allow_signal = !opts->has_signal || opts->signal;
> qemu_chr_set_echo_stdio(chr, false);
> }
> #endif
> --
> 2.28.0
>
>
>
--
Marc-André Lureau
Re: [PATCH 0/6] qemu-storage-daemon: QAPIfy --chardev
On Fri, Oct 23, 2020 at 12:12:16PM +0200, Kevin Wolf wrote: > While the qemu-storage-daemon command line is still considered unstable, > let's change --chardev from the old QemuOpts-based parser to QAPI, so > that it becomes a simple mapping of chardev-add to the command line and > will fit in a future fully QAPIfied command line without later > incompatible changes or additional compatibility glue. NB, I am *not* objecting to this series in what I'm about to write, but I want to point out that I don't think we should assume that this proposed QAPIified CLI is neccessarily the long term end point. Below I outline potential incompatible changes that would end up turning -chardev into legacy / deprecated syntax. The chardev QAPI-ification is a good example of the gap between QOM and QAPI. The chardev backends are QOM classes, but they lack the "user creatable" interface. Thus instead of configuring them using -object / object_add, we have a QAPI model, custom args/commands -chardev / chardev_add, and then code that populates the object instance properties from the QAPI model. Except we've not really exposed them as object instance properties - they are mostly just struct fields directly accessed. The benefit of the chardev approach is that we have an introspectable QAPI model. The downside is that we have a bunch of conversion code for QAPI to QOM mapping. The benefit of the user creatable object approach is that we dont have to add custom CLI args for different types of object, nor write code to populate QOM from QAPI. The downside is that we're divorced from the QAPI schema, so loose introspection, and have a different type of tedious boilerplate code to write. The fact that we have this custom -chardev arg, instead of just using -object is related to the ordering problems we have in main() in the system emulators. We have some -object types that need creating before the chardev types, and some -object types that need creating after the chardevs. If -chardev didn't exist, and we used -object for chardev creation, then the ordering problem would trivially not exist, as we would simply be processing all -object args in order that we receive them. IOW, if we're considering our ideal long term configuration goals for QEMU, then I think we should be looking at a solution to bridge the gap, so that we can get the best of both worlds, and have a single way todo things. IMHO, ideally none of -netdev, -chardev, -device, etc would exist in any "modern" CLI, they would just be considered legacy syntax. Essentially everything would be created using the same -object arg and object_add commands. What I think this means is that we need to use QAPI to represent the properties of an object, and auto-generate the QOM code to register properties, provider setters/getters, constructors, etc. For existing user creatable objects this would largely involve deleting lots of existing code and defining some simple QAPI schemas. This is probably quite easy, if tedious. For the chardev/netdev/etc objects we already have the QAPI schema, but would need to add code genration parts to turn them into full user creatable object classes. This is likely harder more involved work, especially to keep backcompat working well. Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
[PATCH 5/6] tests/qapi-schema: Flat representation of simple unions
This adds some test cases related to flat representation of simple
unions and the 'allow-flat' option for union branches.
Signed-off-by: Kevin Wolf
---
tests/qapi-schema/flat-union-allow-flat.err| 2 ++
tests/qapi-schema/flat-union-allow-flat.json | 10 ++
tests/qapi-schema/flat-union-allow-flat.out| 0
tests/qapi-schema/meson.build | 4
tests/qapi-schema/qapi-schema-test.json| 10 ++
tests/qapi-schema/qapi-schema-test.out | 14 ++
tests/qapi-schema/union-allow-flat-bad.err | 2 ++
tests/qapi-schema/union-allow-flat-bad.json| 5 +
tests/qapi-schema/union-allow-flat-bad.out | 0
.../qapi-schema/union-allow-flat-builtin-type.err | 2 ++
.../qapi-schema/union-allow-flat-builtin-type.json | 5 +
.../qapi-schema/union-allow-flat-builtin-type.out | 0
tests/qapi-schema/union-clash-member.err | 2 ++
tests/qapi-schema/union-clash-member.json | 6 ++
tests/qapi-schema/union-clash-member.out | 0
15 files changed, 62 insertions(+)
create mode 100644 tests/qapi-schema/flat-union-allow-flat.err
create mode 100644 tests/qapi-schema/flat-union-allow-flat.json
create mode 100644 tests/qapi-schema/flat-union-allow-flat.out
create mode 100644 tests/qapi-schema/union-allow-flat-bad.err
create mode 100644 tests/qapi-schema/union-allow-flat-bad.json
create mode 100644 tests/qapi-schema/union-allow-flat-bad.out
create mode 100644 tests/qapi-schema/union-allow-flat-builtin-type.err
create mode 100644 tests/qapi-schema/union-allow-flat-builtin-type.json
create mode 100644 tests/qapi-schema/union-allow-flat-builtin-type.out
create mode 100644 tests/qapi-schema/union-clash-member.err
create mode 100644 tests/qapi-schema/union-clash-member.json
create mode 100644 tests/qapi-schema/union-clash-member.out
diff --git a/tests/qapi-schema/flat-union-allow-flat.err
b/tests/qapi-schema/flat-union-allow-flat.err
new file mode 100644
index 00..f3f3d2384b
--- /dev/null
+++ b/tests/qapi-schema/flat-union-allow-flat.err
@@ -0,0 +1,2 @@
+flat-union-allow-flat.json: In union 'MyUnion':
+flat-union-allow-flat.json:6: 'allow-flat' requires simple union
diff --git a/tests/qapi-schema/flat-union-allow-flat.json
b/tests/qapi-schema/flat-union-allow-flat.json
new file mode 100644
index 00..114acaad86
--- /dev/null
+++ b/tests/qapi-schema/flat-union-allow-flat.json
@@ -0,0 +1,10 @@
+# Flat unions must not use allow-flat
+{ 'enum': 'Enum', 'data': [ 'one', 'two' ] }
+{ 'struct': 'Base',
+ 'data': { 'switch': 'Enum' } }
+{ 'struct': 'Branch', 'data': { 'name': 'str' } }
+{ 'union': 'MyUnion',
+ 'base': 'Base',
+ 'discriminator': 'switch',
+ 'data': { 'one': { 'type': 'Branch', 'allow-flat': true },
+'two': 'Branch' } }
diff --git a/tests/qapi-schema/flat-union-allow-flat.out
b/tests/qapi-schema/flat-union-allow-flat.out
new file mode 100644
index 00..e69de29bb2
diff --git a/tests/qapi-schema/meson.build b/tests/qapi-schema/meson.build
index 304ef939bd..4973a9b57a 100644
--- a/tests/qapi-schema/meson.build
+++ b/tests/qapi-schema/meson.build
@@ -103,6 +103,7 @@ schemas = [
'features-name-bad-type.json',
'features-no-list.json',
'features-unknown-key.json',
+ 'flat-union-allow-flat.json',
'flat-union-array-branch.json',
'flat-union-bad-base.json',
'flat-union-bad-discriminator.json',
@@ -184,12 +185,15 @@ schemas = [
'unclosed-list.json',
'unclosed-object.json',
'unclosed-string.json',
+ 'union-allow-flat-bad.json',
+ 'union-allow-flat-builtin-type.json',
'union-base-empty.json',
'union-base-no-discriminator.json',
'union-branch-case.json',
'union-branch-if-invalid.json',
'union-branch-invalid-dict.json',
'union-clash-branches.json',
+ 'union-clash-member.json',
'union-empty.json',
'union-invalid-base.json',
'union-optional-branch.json',
diff --git a/tests/qapi-schema/qapi-schema-test.json
b/tests/qapi-schema/qapi-schema-test.json
index 63f92adf68..1c29576ae9 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -110,6 +110,16 @@
{ 'struct': 'UserDefC',
'data': { 'string1': 'str', 'string2': 'str' } }
+{ 'struct': 'UserDefD',
+ 'data': { 'type': 'str' } }
+
+{ 'union': 'UserDefSimpleUnion',
+ 'data': { 'value1' : {'type': 'UserDefA'},
+'value2' : 'UserDefB',
+'value3' : 'UserDefB',
+'value4' : { 'type': 'UserDefD', 'allow-flat': false }
+ } }
+
# for testing use of 'number' within alternates
{ 'alternate': 'AltEnumBool', 'data': { 'e': 'EnumOne', 'b': 'bool' } }
{ 'alternate': 'AltEnumNum', 'data': { 'e': 'EnumOne', 'n': 'number' } }
diff --git a/tests/qapi-schema/qapi-schema-test.out
b/tests/qapi-schema/qapi-schema-test.out
index a4e4051c61..7521e86e9f 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -105,6 +105,20 @@
[PATCH 6/6] qemu-storage-daemon: Use qmp_chardev_add() for --chardev
While this makes the code quite a bit longer and arguably isn't very
elegant, it removes the dependency on QemuOpts from the --chardev option
of the storage daemon.
Going through qmp_chardev_add() already now ensures that semantics and
accessible features won't change incompatibly once we QAPIfy it fully.
Note that there are a few differences between the command line option
-chardev in the system emulator and chardev-add in QMP. The --chardev
option of qemu-storage-daemon will follow QMP in these respects:
* All chardev types using ChardevHostdev accept a "path" option on the
command line, but QMP renamed it to "device"
* ChardevSocket:
- Intentionally different defaults (documented as such): server=false
and wait=true (if server=true) on the command line, but server=true
and wait=false in QMP.
- Accidentally different defaults: tight=true on the command line, but
tight=false in QMP (this is a bug in commit 776b97d3).
- QMP has a nested SocketAddressLegacy field, whereas the command line
accepts "path"/"host"/"port"/"fd"/etc. on the top level.
- The command line has a "delay" option, but QMP has "nodelay"
* ChardevUdp has two SocketAddressLegacy fields, the command line has
"host"/"port"/"localaddr"/"localport" on the top level with defaults
for values that are mandatory in SocketAddressLegacy
Signed-off-by: Kevin Wolf
---
storage-daemon/qemu-storage-daemon.c | 47 ++--
1 file changed, 38 insertions(+), 9 deletions(-)
diff --git a/storage-daemon/qemu-storage-daemon.c
b/storage-daemon/qemu-storage-daemon.c
index e419ba9f19..b543c30951 100644
--- a/storage-daemon/qemu-storage-daemon.c
+++ b/storage-daemon/qemu-storage-daemon.c
@@ -37,6 +37,7 @@
#include "qapi/error.h"
#include "qapi/qapi-visit-block-core.h"
#include "qapi/qapi-visit-block-export.h"
+#include "qapi/qapi-visit-char.h"
#include "qapi/qapi-visit-control.h"
#include "qapi/qmp/qdict.h"
#include "qapi/qmp/qstring.h"
@@ -207,18 +208,46 @@ static void process_options(int argc, char *argv[])
}
case OPTION_CHARDEV:
{
-/* TODO This interface is not stable until we QAPIfy it */
-QemuOpts *opts = qemu_opts_parse_noisily(&qemu_chardev_opts,
- optarg, true);
-if (opts == NULL) {
-exit(EXIT_FAILURE);
-}
+QDict *args;
+Visitor *v;
+ChardevBackend *chr_options;
+char *id;
+bool help;
-if (!qemu_chr_new_from_opts(opts, NULL, &error_fatal)) {
-/* No error, but NULL returned means help was printed */
+args = keyval_parse(optarg, "type", &help, &error_fatal);
+if (help) {
+if (qdict_haskey(args, "type")) {
+/* TODO Print help based on the QAPI schema */
+qemu_opts_print_help(&qemu_chardev_opts, true);
+} else {
+qemu_chr_print_types();
+}
exit(EXIT_SUCCESS);
}
-qemu_opts_del(opts);
+
+/*
+ * TODO Flattened version of chardev-add in the QAPI schema
+ *
+ * While it's not there, parse 'id' manually from the QDict
+ * and treat everything else as the 'backend' option for
+ * chardev-add.
+ */
+id = g_strdup(qdict_get_try_str(args, "id"));
+if (!id) {
+error_report("Parameter 'id' is missing");
+exit(EXIT_FAILURE);
+}
+qdict_del(args, "id");
+
+v = qobject_input_visitor_new_keyval(QOBJECT(args));
+visit_set_flat_simple_unions(v, true);
+visit_type_ChardevBackend(v, NULL, &chr_options, &error_fatal);
+visit_free(v);
+
+qmp_chardev_add(id, chr_options, &error_fatal);
+qapi_free_ChardevBackend(chr_options);
+g_free(id);
+qobject_unref(args);
break;
}
case OPTION_EXPORT:
--
2.28.0
[PATCH 4/6] qapi: Optionally parse simple unions as flat
This extends the Visitor interface with an option that can enable flat
representation (without the 'data' wrapper) for simple unions. This way,
a command line parser can enable a more user friendly syntax while QMP
doesn't enable the option and uses the same representation as before.
We need to disable flat representation for ChardevSpiceChannel, which
has a 'type' option that conflicts with the ChardevBackend 'type'. This
variant will get nesting even when flat parsing is enabled in the
Visitor.
Signed-off-by: Kevin Wolf
---
qapi/char.json | 3 ++-
docs/devel/qapi-code-gen.txt | 11 ++-
include/qapi/visitor-impl.h | 3 +++
include/qapi/visitor.h | 10 ++
qapi/qapi-visit-core.c | 10 ++
scripts/qapi/expr.py | 7 ++-
scripts/qapi/schema.py | 25 ++---
scripts/qapi/visit.py| 20 ++--
8 files changed, 73 insertions(+), 16 deletions(-)
diff --git a/qapi/char.json b/qapi/char.json
index 43486d1daa..57ec18220b 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -414,7 +414,8 @@
'stdio': 'ChardevStdio',
'console': 'ChardevCommon',
'spicevmc': { 'type': 'ChardevSpiceChannel',
- 'if': 'defined(CONFIG_SPICE)' },
+ 'if': 'defined(CONFIG_SPICE)',
+ 'allow-flat': false },
'spiceport': { 'type': 'ChardevSpicePort',
'if': 'defined(CONFIG_SPICE)' },
'vc': 'ChardevVC',
diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index 9722c1a204..ee34d39a20 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -295,7 +295,9 @@ Syntax:
'*features': FEATURES }
BRANCHES = { BRANCH, ... }
BRANCH = STRING : TYPE-REF
- | STRING : { 'type': TYPE-REF, '*if': COND }
+ | STRING : { 'type': TYPE-REF,
+'*if': COND,
+'*allow-flat': BOOL }
Member 'union' names the union type.
@@ -334,6 +336,13 @@ values to data types like in this example:
'data': { 'file': 'BlockdevOptionsFile',
'qcow2': 'BlockdevOptionsQcow2' } }
+Simple unions can support both wrapped and flat representation of
+branches that have a struct type, unless it is explicitly disabled in
+the schema with 'allow-flat': false. Branches of other types are
+always wrapped. Which representation is used in the generated visitor
+C code can be configured per visitor. Flat representation is
+appropriate when parsing command line options.
+
In the Client JSON Protocol, all simple union branches have wrapped
representation, as in these examples:
diff --git a/include/qapi/visitor-impl.h b/include/qapi/visitor-impl.h
index 7362c043be..f628b6eb36 100644
--- a/include/qapi/visitor-impl.h
+++ b/include/qapi/visitor-impl.h
@@ -121,6 +121,9 @@ struct Visitor
/* Must be set */
void (*free)(Visitor *v);
+
+/* Set to true to make simple unions look like flat unions */
+bool flat_simple_unions;
};
#endif
diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index ebc19ede7f..d41be4df48 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -496,6 +496,16 @@ bool visit_is_input(Visitor *v);
*/
bool visit_is_dealloc(Visitor *v);
+/*
+ * Check if simple unions should be treated as flat.
+ */
+bool visit_flat_simple_unions(Visitor *v);
+
+/*
+ * Set if simple unions should be treated as flat.
+ */
+void visit_set_flat_simple_unions(Visitor *v, bool flat);
+
/*** Visiting built-in types ***/
/*
diff --git a/qapi/qapi-visit-core.c b/qapi/qapi-visit-core.c
index 7e5f40e7f0..dc6fd78b8c 100644
--- a/qapi/qapi-visit-core.c
+++ b/qapi/qapi-visit-core.c
@@ -145,6 +145,16 @@ bool visit_is_dealloc(Visitor *v)
return v->type == VISITOR_DEALLOC;
}
+bool visit_flat_simple_unions(Visitor *v)
+{
+return v->flat_simple_unions;
+}
+
+void visit_set_flat_simple_unions(Visitor *v, bool flat)
+{
+v->flat_simple_unions = flat;
+}
+
bool visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp)
{
assert(obj);
diff --git a/scripts/qapi/expr.py b/scripts/qapi/expr.py
index 2fcaaa2497..a39092e4a9 100644
--- a/scripts/qapi/expr.py
+++ b/scripts/qapi/expr.py
@@ -248,7 +248,12 @@ def check_union(expr, info):
for (key, value) in members.items():
source = "'data' member '%s'" % key
check_name_str(key, info, source)
-check_keys(value, info, source, ['type'], ['if'])
+check_keys(value, info, source, ['type'], ['if', 'allow-flat'])
+if 'allow-flat' in value:
+if discriminator is not None:
+raise QAPISemError(info, "'allow-flat' requires simple union")
+if not isinstance(value['allow-flat'], bool):
+raise QAPISemError(info, "'allow-flat' must be a boolean")
check_if(value, info, source)
[PATCH 1/6] char/stdio: Fix QMP default for 'signal'
Commit 02c4bdf1 tried to make signal=on the default for stdio chardevs
except for '-serial mon:stdio', but it forgot about QMP and accidentally
switched the QMP default from true (except for -nographic) to false
(always). The documentation was kept unchanged and still describes the
opposite of the old behaviour (which is an even older documentation
bug).
Fix all of this by making signal=true the default in ChardevStdio and
documenting it as such.
Fixes: 02c4bdf1d2ca8c02a9bae16398f260b5c08d08bf
Signed-off-by: Kevin Wolf
---
qapi/char.json | 3 +--
chardev/char-stdio.c | 4 +---
2 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/qapi/char.json b/qapi/char.json
index b4d66ec90b..43486d1daa 100644
--- a/qapi/char.json
+++ b/qapi/char.json
@@ -321,8 +321,7 @@
# Configuration info for stdio chardevs.
#
# @signal: Allow signals (such as SIGINT triggered by ^C)
-# be delivered to qemu. Default: true in -nographic mode,
-# false otherwise.
+# be delivered to qemu. Default: true.
#
# Since: 1.5
##
diff --git a/chardev/char-stdio.c b/chardev/char-stdio.c
index 82eaebc1db..403da308c9 100644
--- a/chardev/char-stdio.c
+++ b/chardev/char-stdio.c
@@ -112,9 +112,7 @@ static void qemu_chr_open_stdio(Chardev *chr,
qemu_chr_open_fd(chr, 0, 1);
-if (opts->has_signal) {
-stdio_allow_signal = opts->signal;
-}
+stdio_allow_signal = !opts->has_signal || opts->signal;
qemu_chr_set_echo_stdio(chr, false);
}
#endif
--
2.28.0
[PATCH 2/6] char: Factor out qemu_chr_print_types()
We'll want to call the same from a non-QemuOpts code path.
Signed-off-by: Kevin Wolf
---
include/chardev/char.h | 1 +
chardev/char.c | 17 +++--
2 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/include/chardev/char.h b/include/chardev/char.h
index db42f0a8c6..3b91645081 100644
--- a/include/chardev/char.h
+++ b/include/chardev/char.h
@@ -212,6 +212,7 @@ void qemu_chr_be_update_read_handlers(Chardev *s,
*/
void qemu_chr_be_event(Chardev *s, QEMUChrEvent event);
+void qemu_chr_print_types(void);
int qemu_chr_add_client(Chardev *s, int fd);
Chardev *qemu_chr_find(const char *name);
diff --git a/chardev/char.c b/chardev/char.c
index 78553125d3..028612c333 100644
--- a/chardev/char.c
+++ b/chardev/char.c
@@ -633,6 +633,16 @@ ChardevBackend *qemu_chr_parse_opts(QemuOpts *opts, Error
**errp)
return backend;
}
+void qemu_chr_print_types(void)
+{
+GString *str = g_string_new("");
+
+chardev_name_foreach(help_string_append, str);
+
+qemu_printf("Available chardev backend types: %s\n", str->str);
+g_string_free(str, true);
+}
+
Chardev *qemu_chr_new_from_opts(QemuOpts *opts, GMainContext *context,
Error **errp)
{
@@ -644,12 +654,7 @@ Chardev *qemu_chr_new_from_opts(QemuOpts *opts,
GMainContext *context,
char *bid = NULL;
if (name && is_help_option(name)) {
-GString *str = g_string_new("");
-
-chardev_name_foreach(help_string_append, str);
-
-qemu_printf("Available chardev backend types: %s\n", str->str);
-g_string_free(str, true);
+qemu_chr_print_types();
return NULL;
}
--
2.28.0
[PATCH 3/6] qapi: Remove wrapper struct for simple unions
Variants of simple unions are always contained in a wrapper object
called 'data' that serves no real use. When mapping a QAPI object to the
command line, this becomes very visible to users because they have to
add one or more useless 'data.' to many option names.
As a first step towards avoiding this painful CLI experience, this gets
rid of the 'data' indirection internally: The QAPI parser doesn't use a
wrapper object as the variant type any more, so the indirection is
removed from the generated C types. As a result, the C type looks the
same for flat and simple unions now. A large part of this patch is
mechanical conversion of C code to remove the 'data' indirection.
Conceptually, the important change is that variants can now have flat
and wrapped representations. For a transitioning period, we'll allow
variants to support both representations in a later patch. Eventually,
the plan is to deprecate and remove wrapped representations entirely,
unifying simple and flat unions.
The externally visible interfaces stay unchanged: Visitors still expect
the 'data' wrappers, and introspection still shows it.
Signed-off-by: Kevin Wolf
---
docs/devel/qapi-code-gen.txt | 32
backends/tpm/tpm_emulator.c| 3 +-
backends/tpm/tpm_passthrough.c | 4 +-
block/crypto.c | 3 +-
block/qcow2.c | 9 ++--
block/vmdk.c | 14 ++
blockdev.c | 36 ++---
chardev/char-file.c| 5 +-
chardev/char-mux.c | 5 +-
chardev/char-parallel.c| 5 +-
chardev/char-pipe.c| 7 ++-
chardev/char-ringbuf.c | 5 +-
chardev/char-serial.c | 5 +-
chardev/char-socket.c | 13 ++---
chardev/char-stdio.c | 5 +-
chardev/char-udp.c | 11 ++--
chardev/char.c | 10 ++--
chardev/msmouse.c | 4 +-
chardev/wctablet.c | 4 +-
hw/core/numa.c | 6 +--
hw/display/xenfb.c | 8 +--
hw/input/hid.c | 8 +--
hw/input/ps2.c | 6 +--
hw/input/virtio-input-hid.c| 8 +--
hw/mem/pc-dimm.c | 18 +++
hw/virtio/virtio-mem-pci.c | 3 +-
hw/virtio/virtio-pmem-pci.c| 3 +-
monitor/hmp-cmds.c | 14 +++---
qemu-keymap.c | 2 +-
replay/replay-input.c | 46 +++--
tests/test-char.c | 12 ++---
tests/test-clone-visitor.c | 14 +++---
tests/test-qmp-cmds.c | 2 +-
tests/test-qobject-input-visitor.c | 24 -
tests/test-qobject-output-visitor.c| 24 -
ui/console.c | 5 +-
ui/input-keymap.c | 12 ++---
ui/input-legacy.c | 12 ++---
ui/input.c | 43
util/qemu-sockets.c| 8 +--
scripts/qapi/introspect.py | 7 ++-
scripts/qapi/schema.py | 48 ++
scripts/qapi/visit.py | 28 +++
tests/qapi-schema/doc-good.out | 8 +--
tests/qapi-schema/qapi-schema-test.out | 70 +++---
45 files changed, 308 insertions(+), 311 deletions(-)
diff --git a/docs/devel/qapi-code-gen.txt b/docs/devel/qapi-code-gen.txt
index c6438c6aa9..9722c1a204 100644
--- a/docs/devel/qapi-code-gen.txt
+++ b/docs/devel/qapi-code-gen.txt
@@ -310,7 +310,20 @@ The BRANCH's STRING name is the branch name.
The BRANCH's value defines the branch's properties, in particular its
type. The form TYPE-REF is shorthand for { 'type': TYPE-REF }.
-A simple union type defines a mapping from automatic discriminator
+Each branch can have flat or wrapped representation in the Client JSON
+Protocol.
+
+Wrapped representation of a branch means that it is represented by a
+'data' member that is of the specified data type corresponding to the
+discriminator value.
+
+Flat representation means that the union forms a single object that
+contains both common members (from the base type) and the selected
+branch's members, without any additional nesting. The two sets of
+member names must be disjoint.
+
+A simple union type automatically contains the 'type' member as a
+discriminator and defines a mapping from automatic discriminator
values to data types like in this example:
{ 'struct': 'BlockdevOptionsFile', 'data': { 'filename': 'str' } }
@@ -321,10 +334,8 @@ values to data types like in this example:
'data': { 'file': 'BlockdevOptionsFile',
'qcow2': 'BlockdevOptionsQcow2' } }
-In the Client JSON Protocol, a simple union is represented by an
-object that contains the 'type' member as a discriminator
[PATCH 0/6] qemu-storage-daemon: QAPIfy --chardev
While the qemu-storage-daemon command line is still considered unstable, let's change --chardev from the old QemuOpts-based parser to QAPI, so that it becomes a simple mapping of chardev-add to the command line and will fit in a future fully QAPIfied command line without later incompatible changes or additional compatibility glue. Kevin Wolf (6): char/stdio: Fix QMP default for 'signal' char: Factor out qemu_chr_print_types() qapi: Remove wrapper struct for simple unions qapi: Optionally parse simple unions as flat tests/qapi-schema: Flat representation of simple unions qemu-storage-daemon: Use qmp_chardev_add() for --chardev qapi/char.json| 6 +- docs/devel/qapi-code-gen.txt | 43 +++--- include/chardev/char.h| 1 + include/qapi/visitor-impl.h | 3 + include/qapi/visitor.h| 10 +++ backends/tpm/tpm_emulator.c | 3 +- backends/tpm/tpm_passthrough.c| 4 +- block/crypto.c| 3 +- block/qcow2.c | 9 +- block/vmdk.c | 14 ++-- blockdev.c| 36 chardev/char-file.c | 5 +- chardev/char-mux.c| 5 +- chardev/char-parallel.c | 5 +- chardev/char-pipe.c | 7 +- chardev/char-ringbuf.c| 5 +- chardev/char-serial.c | 5 +- chardev/char-socket.c | 13 ++- chardev/char-stdio.c | 9 +- chardev/char-udp.c| 11 +-- chardev/char.c| 27 +++--- chardev/msmouse.c | 4 +- chardev/wctablet.c| 4 +- hw/core/numa.c| 6 +- hw/display/xenfb.c| 8 +- hw/input/hid.c| 8 +- hw/input/ps2.c| 6 +- hw/input/virtio-input-hid.c | 8 +- hw/mem/pc-dimm.c | 18 ++-- hw/virtio/virtio-mem-pci.c| 3 +- hw/virtio/virtio-pmem-pci.c | 3 +- monitor/hmp-cmds.c| 14 ++-- qapi/qapi-visit-core.c| 10 +++ qemu-keymap.c | 2 +- replay/replay-input.c | 46 +- storage-daemon/qemu-storage-daemon.c | 47 +-- tests/test-char.c | 12 ++- tests/test-clone-visitor.c| 14 ++-- tests/test-qmp-cmds.c | 2 +- tests/test-qobject-input-visitor.c| 24 +++--- tests/test-qobject-output-visitor.c | 24 +++--- ui/console.c | 5 +- ui/input-keymap.c | 12 +-- ui/input-legacy.c | 12 +-- ui/input.c| 43 +- util/qemu-sockets.c | 8 +- scripts/qapi/expr.py | 7 +- scripts/qapi/introspect.py| 7 +- scripts/qapi/schema.py| 63 +++--- scripts/qapi/visit.py | 36 tests/qapi-schema/doc-good.out| 8 +- tests/qapi-schema/flat-union-allow-flat.err | 2 + tests/qapi-schema/flat-union-allow-flat.json | 10 +++ tests/qapi-schema/flat-union-allow-flat.out | 0 tests/qapi-schema/meson.build | 4 + tests/qapi-schema/qapi-schema-test.json | 10 +++ tests/qapi-schema/qapi-schema-test.out| 84 +++ tests/qapi-schema/union-allow-flat-bad.err| 2 + tests/qapi-schema/union-allow-flat-bad.json | 5 ++ tests/qapi-schema/union-allow-flat-bad.out| 0 .../union-allow-flat-builtin-type.err | 2 + .../union-allow-flat-builtin-type.json| 5 ++ .../union-allow-flat-builtin-type.out | 0 tests/qapi-schema/union-clash-member.err | 2 + tests/qapi-schema/union-clash-member.json | 6 ++ tests/qapi-schema/union-clash-member.out | 0 66 files changed, 484 insertions(+), 336 deletions(-) create mode 100644 tests/qapi-schema/flat-union-allow-flat.err create mode 100644 tests/qapi-schema/flat-union-allow-flat.json create mode 100644 tests/qapi-schema/flat-union-allow-flat.out create mode 100644 tests/qapi-schema/union-allow-flat-bad.err create mode 100644 tests/qapi-schema/union-allow-flat-bad.json create mode 100644 tests/qapi-schema/union-allow-flat-bad.out create mode 100644 tests/qapi-schema/union-allow-flat-builtin-type.err create mode 100644 tests/qapi-schema/union-allow-flat-builtin-type.json create mode 100644 tests/qapi-schema/union-allow-flat
Re: [PATCH v2 08/20] block/block-copy: add block_copy_cancel
22.10.2020 23:50, Vladimir Sementsov-Ogievskiy wrote:
22.07.2020 14:28, Max Reitz wrote:
On 01.06.20 20:11, Vladimir Sementsov-Ogievskiy wrote:
Add function to cancel running async block-copy call. It will be used
in backup.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/block/block-copy.h | 7 +++
block/block-copy.c | 22 +++---
2 files changed, 26 insertions(+), 3 deletions(-)
diff --git a/include/block/block-copy.h b/include/block/block-copy.h
index d40e691123..370a194d3c 100644
--- a/include/block/block-copy.h
+++ b/include/block/block-copy.h
@@ -67,6 +67,13 @@ BlockCopyCallState *block_copy_async(BlockCopyState *s,
void block_copy_set_speed(BlockCopyState *s, BlockCopyCallState *call_state,
uint64_t speed);
+/*
+ * Cancel running block-copy call.
+ * Cancel leaves block-copy state valid: dirty bits are correct and you may use
+ * cancel + to emulate pause/resume.
+ */
+void block_copy_cancel(BlockCopyCallState *call_state);
+
BdrvDirtyBitmap *block_copy_dirty_bitmap(BlockCopyState *s);
void block_copy_set_skip_unallocated(BlockCopyState *s, bool skip);
diff --git a/block/block-copy.c b/block/block-copy.c
index 851d9c8aaf..b551feb6c2 100644
--- a/block/block-copy.c
+++ b/block/block-copy.c
@@ -44,6 +44,8 @@ typedef struct BlockCopyCallState {
bool failed;
bool finished;
QemuCoSleepState *sleep_state;
+ bool cancelled;
+ Coroutine *canceller;
/* OUT parameters */
bool error_is_read;
@@ -582,7 +584,7 @@ block_copy_dirty_clusters(BlockCopyCallState *call_state)
assert(QEMU_IS_ALIGNED(offset, s->cluster_size));
assert(QEMU_IS_ALIGNED(bytes, s->cluster_size));
- while (bytes && aio_task_pool_status(aio) == 0) {
+ while (bytes && aio_task_pool_status(aio) == 0 && !call_state->cancelled) {
BlockCopyTask *task;
int64_t status_bytes;
@@ -693,7 +695,7 @@ static int coroutine_fn
block_copy_common(BlockCopyCallState *call_state)
do {
ret = block_copy_dirty_clusters(call_state);
- if (ret == 0) {
+ if (ret == 0 && !call_state->cancelled) {
ret = block_copy_wait_one(call_state->s, call_state->offset,
call_state->bytes);
}
@@ -707,13 +709,18 @@ static int coroutine_fn
block_copy_common(BlockCopyCallState *call_state)
* 2. We have waited for some intersecting block-copy request
* It may have failed and produced new dirty bits.
*/
- } while (ret > 0);
+ } while (ret > 0 && !call_state->cancelled);
Would it be cleaner if block_copy_dirty_cluster() just returned
-ECANCELED? Or would that pose a problem for its callers or the async
callback?
I'd prefer not to merge io ret with block-copy logic: who knows what underlying
operations may return.. Can't it be _another_ ECANCELED?
And it would be just a sugar for block_copy_dirty_clusters() call, I'll have to
check ->cancelled after block_copy_wait_one() anyway.
Also, for the next version I try to make it more obvious that finished
block-copy call is in one of thee states:
- success
- failed
- cancelled
Hmm. Also, cancelled should be OK for copy-on-write operations in filter, it
just mean that we don't need to care anymore.
This is unrelated: actually only async block-copy call may be cancelled.
if (call_state->cb) {
call_state->cb(ret, call_state->error_is_read,
call_state->s->progress_opaque);
}
+ if (call_state->canceller) {
+ aio_co_wake(call_state->canceller);
+ call_state->canceller = NULL;
+ }
+
call_state->finished = true;
return ret;
--
Best regards,
Vladimir
Re: [PULL 22/23] hw/sd: Fix incorrect populated function switch status data structure
On 10/23/20 4:02 AM, Bin Meng wrote: Hi Niek, On Thu, Oct 22, 2020 at 11:20 PM Niek Linnenbank wrote: Hi Bin, Philippe, If im correct the acceptance tests for orange pi need to be run with a flag ARMBIAN_ARTIFACTS_CACHED set that explicitly allows them to be run using the armbian mirror. So if you pass that flag on the same command that Philippe gave, the rests should run. Thank you for the hints. Actually I noticed the environment variable ARMBIAN_ARTIFACTS_CACHED when looking at the test codes, but after I turned on the flag it still could not download the test asset from the apt.armbian.com website. This patch could help you, but it use a different image [*]: https://www.mail-archive.com/qemu-devel@nongnu.org/msg737760.html (I haven't tested the image yet but I assume the same bug is present). I have a follow up question and Im interested to hear your opinion on that Philippe. Should we perhaps update the orange pi tests (and maybe others) so they use a reliable mirror that we can control, for example a github repo? I would be happy to create a repo for that, at least for the orange pi tests. But maybe there is already something planned as a more general solution for artifacts of other machines as well? This is a technical debt between the Avocado and QEMU communities (or a misunderstanding?). QEMU community understood the Avocado caching would be an helpful feature, but it ended being more of a problem. I.e. here Niek, Michael Roth and myself still have the image cached, so we can keep running the tests committed to the repository. You Bin can not, as the armbian mirror is not stable and remove the old image. The old image (Armbian_19.11.3_Orangepipc_bionic_current_5.3.9.img) has been manually tested by Niek and myself, I enabled tracing and look at the SD packets for some time, was happy how the model worked and decided we should keep running a test with this particular image. Armbian released new images, in particular the one Cleber sent in [*]: Armbian_20.08.1_Orangepipc_bionic_current_5.8.5.img.xz. While it might work similarly, I haven't tested it, and don't have time to do again the same audit. From my experience with the Raspberry Pi, these images never work the same way, as the Linux kernel evolves quickly with these kinda recent embedded boards. The QEMU raspi models have to emulate new devices (or complete current ones) every years, because Linux started to use a device differently, or started to use a part of the SoC that was not used before. Either the kernel doesn't boot, or there are side-effect later when userspace program is executed. PITA to debug TBH. For this reason I disagree with updating test images to the latest releases. It would be nice to know QEMU works with the latest Armbian, but nobody popped up on the mailing list asking for it. I am personally happy enough using the 19.11 release for now.
